Camera two-dimensional moving target tracking method and system based on loopback detection

Abstract

The invention discloses a camera two-dimensional moving target tracking method and system based on loopback detection. The method is suitable for a dynamic operation scene of mechanical equipment, target appearance features are extracted through a structural modeling strategy, and dynamic estimation and noise reduction are performed on the camera pose by using equipment motion estimation and Kalman filtering. A TOPIC matching algorithm is introduced to realize high-precision tracking of motion and appearance feature fusion, and meanwhile, a long-term tracking error is corrected through loopback detection. Finally, a target information map which can be updated in real time is constructed, target state management and visual output are supported, and the accuracy and stability of continuous tracking in an application scene are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a camera two-dimensional motion target tracking method and system based on loop detection, which is particularly suitable for precise tracking in a complex environment under camera two-dimensional motion. BACKGROUND

[0002] Most of the multi-target tracking algorithms in actual applications are based on static or relatively unchanged background scenes, and typical applications include monitoring video analysis, traffic target detection, etc. Such methods generally rely on the movement of targets in a fixed background, and thus match and track the targets by their position information, appearance features, motion trajectories, etc. in consecutive frames. In such scenarios, the targets usually stay in the field of view for a long time, and the algorithm can achieve high-precision continuous tracking by constructing appearance models and motion prediction models. However, in order to accurately model the motion state of multiple targets and extract the appearance features, existing methods often have problems such as large computational overhead, slow processing speed, high algorithm complexity, etc., which makes it difficult to meet the requirements of high processing efficiency and real-time performance in application environments.

[0003] In a complex environment with camera two-dimensional motion, the target and the background environment change simultaneously, showing dynamic coupling characteristics, making target detection and tracking more difficult. In such scenarios, the target stays in the field of view for a short time, is generated and disappears frequently, and is disturbed by strong background information, etc., further exacerbating the shortcomings of traditional tracking algorithms in terms of detection accuracy and stability. In addition, due to the irregular appearance frequency of targets in complex scenes, even a large span may appear in adjacent frames, existing methods are difficult to maintain the identity consistency of the target in long time intervals and multiple re-identification scenarios, affecting the tracking continuity and accuracy.

[0004] More seriously, in actual applications, the system often needs to deploy a target tracking module on a resource-limited embedded platform or unmanned device, so higher requirements are put forward for the computational efficiency, energy consumption control and real-time response capability of the algorithm. Traditional tracking algorithms based on the assumption of static background are difficult to adapt to such mobile visual platforms, and cannot effectively meet the needs. Therefore, there is an urgent need for a target tracking method that is highly adaptable, computationally efficient and robust in camera two-dimensional motion, to solve the problem of limited application of existing technology in complex environments. SUMMARY

[0005] In order to solve the problem that the existing multi-target tracking algorithm is limited in the application of two-dimensional camera motion, especially the technical bottlenecks such as short target stay time, strong background dynamic interference, frequent target generation and disappearance, and long time span tracking difficulty in complex environment, the application provides a camera two-dimensional motion target tracking method and system based on loop detection, which can realize efficient, stable and continuous tracking of targets under the condition of parallel motion of multiple cameras, effectively improve the continuity of target tracking, reduce the dependence on computing resources, and meet the comprehensive needs of real-time, accuracy and deployment flexibility in the scene.

[0006] According to an aspect of the present application, a camera two-dimensional motion target tracking method based on loop detection is provided, comprising: obtaining the image sequence of the external environment collected by each perception device during the motion process and performing preprocessing; based on the preprocessed image, target region recognition and candidate extraction are performed to obtain the target in each image; extracting the local key points and their direction invariance descriptors of each target to form the local structure representation of the target; introducing a loop detection mechanism to identify the spatial overlap relationship between the current observation and the historical observation, constructing a closed loop constraint, and correcting the system cumulative error through trajectory optimization; introducing a Kalman filter to estimate and correct the spatial state of the perception device at consecutive time points in real time.

[0007] As a further technical solution, the method further comprises: spatially initializing a plurality of perception devices deployed on a movable platform, obtaining the relative position, direction and field of view boundary of each perception device, and then constructing a coordinate transformation relationship framework between multiple perspectives.

[0008] As a further technical solution, obtaining the image sequence of the external environment collected by each perception device during the motion process and performing preprocessing comprises: based on the image sequence of the external environment continuously obtained by each perception device during the motion process, performing image preprocessing including resolution adjustment, image enhancement and noise suppression.

[0009] As a further technical solution, based on the preprocessed image, target region recognition and candidate extraction are performed, comprising: using a pre-trained target detection network to extract deep features from the input image, generating multi-level feature maps, and through a region candidate generation module or an anchor frame mechanism, multiple potential candidate frames are generated on the feature map, and these candidate frames are processed through classification and regression branches, respectively outputting confidence and boundary box coordinate parameters corresponding to the target category; The target in each image is obtained by screening the candidate frame according to the confidence threshold and merging overlapping regions through non-maximum suppression.

[0010] As a further technical solution, the local key points of each target and their direction invariance descriptors are extracted, including: Based on the obtained target in the image, the local key points and their direction invariance descriptors in the target recognition frame are extracted through the pose recognition neural network or using the ORB algorithm to form the local structure representation of the target.

[0011] As a further technical solution, a loop detection mechanism is introduced to identify the spatial overlap relationship between the current observation and the historical observation, construct a closed loop constraint, and correct the system cumulative error through trajectory optimization, including: Based on the inter-frame position estimation, the spatial pose change of each perception device at adjacent time points is predicted; Based on the spatial pose change of each perception device, the TOPIC is combined for tracking matching to determine whether the current frame and the historical frame constitute an effective loop; After identifying the loop frame, the loop frame and the current frame are constructed into a closed loop edge constraint and added to the entire perception device trajectory graph; The entire trajectory graph is optimized by using a graph optimization method, and the goal is to minimize the re-projection error or relative pose error of all edges.

[0012] As a further technical solution, based on the spatial pose change of each perception device, the TOPIC is combined for tracking matching to determine whether the current frame and the historical frame constitute an effective loop, including: According to the calculated position of the perception device in the target information map, the historical frame tracking information of the current frame is calculated according to the field of view projection; The motion feature loss and the appearance feature loss of the two targets are calculated, and then the motion loss matrix and the appearance loss matrix between the historical frame tracking information and the current frame information are calculated; The TOPIC algorithm is applied through the two loss matrices for matching to determine whether the current frame and the historical frame constitute an effective loop.

[0013] As a further technical solution, a Kalman filter is introduced to estimate and correct the spatial state of the perception device at consecutive time points in real time, including: The motion state of the perception device pose on a two-dimensional plane is modeled; Based on the modeled motion state, a linear or quasi-linear motion prediction model is established to predict the state value of the next frame at the current frame; The high-confidence inter-frame matching result output by the loop detection is introduced as an observation value to correct the prediction error.

[0014] As a further technical solution, after completing the current frame target matching and camera pose correction, the method further comprises: According to the matching result, the target information map is dynamically updated, including adding the position information of the target, extending the trajectory of the identified target, and maintaining the state; Assign a unique identifier to all tracking targets and record their tracking history in consecutive frames; Visualize the category, location and motion trajectory information of the target.

[0015] According to an aspect of the present application, a camera two-dimensional motion target tracking system based on loop detection is provided, comprising: The first main module is used for acquiring the image sequence of the external environment collected by each perception device during the motion process and performing preprocessing; The second main module is used for target region identification and candidate extraction based on the preprocessed image to obtain the target in each image; The third main module is used for extracting the local key points and their direction invariance descriptors of each target to form the local structure representation of the target; The fourth main module introduces a loop detection mechanism to identify the spatial overlap relationship between the current observation and the historical observation, construct a closed loop constraint, and correct the system cumulative error through trajectory optimization; The fifth main module is used for introducing a Kalman filter to estimate and correct the spatial state of the perception device at consecutive time points in real time.

[0016] The present application proposes a target tracking method based on a loop detection mechanism in a camera two-dimensional motion environment, which is designed for mechanical operation in a complex two-dimensional motion scene, and has the following remarkable beneficial effects: 1. Improve the accuracy and robustness of target tracking This method constructs a multi-dimensional target description mechanism, which effectively enhances the system's adaptability to target appearance changes, posture differences and local occlusion through structured perception and representation of the region of interest in the image. Combined with the multi-feature fusion matching mechanism, stable identification can still be maintained in complex scenes with multiple perspectives and multiple targets, reducing problems such as misidentification, mismatching and target loss. Compared with traditional tracking schemes that rely on single features or simple trajectory reasoning, this method has obvious improvement in continuous tracking accuracy.

[0017] 2. Enhance system stability and long-term tracking ability The method constructs a dynamic state prediction and feedback adjustment mechanism, estimates the spatial motion state of the device through the device's own sensing information, and combines historical tracking data to complete state correction. Especially in the process of continuous operation of the device or long-time tracking of the target, the state drift and cumulative error caused by noise, occlusion or posture change can be effectively inhibited. The stability, continuity and recoverability of the system in long-time operation are significantly better than those of existing schemes.

[0018] 3. Improve processing efficiency and adapt to edge computing deployment The present application fully considers the computing power condition and response requirement of the deployment platform, adopts lightweight design and module decoupling mode, so that the system can also run efficiently on resource-limited embedded devices. On a typical edge computing hardware platform, millisecond-level response can be achieved, with fast startup and stable execution capability, which is suitable for the high real-time requirement of industrial automation scenarios.

[0019] 4. Support target map construction and intelligent linkage control Based on the loop detection, the method can construct a target information map and update it in real time, record the state information such as target position, trajectory and life cycle, and output in a visual form, which is convenient for the upper work system to call. The test shows that the target tracking accuracy of the matching system is improved significantly, the repetition rate and frame loss rate are reduced, and the environmental protection and economic benefits are improved.

[0020] In summary, the present application solves the problem of target detection and tracking in the two-dimensional motion background of multiple cameras through multi-module collaborative design, and is superior to the prior art in terms of accuracy, efficiency, stability, resource consumption and practicability, and has good popularization and application prospect, especially suitable for precision operation scenarios. BRIEF DESCRIPTION OF DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0022] Figure 1 A camera two-dimensional motion target tracking method based on loop detection is provided for the embodiments of the present application.

[0023] Figure 2 The TOPIC algorithm pseudocode diagram provided for the embodiments of the present application. DETAILED DESCRIPTION

[0024] The application provides a target tracking method based on loop detection in a camera two-dimensional motion environment, which is suitable for a dynamic operation scene of a mechanical device.

[0025] The terms "comprise", "comprising", "include", "including", "contain", "containing", "have", "having", "composed of", "comprised of", "consist of", "consisting of", and any variations thereof in the specification and in the claims, and the above-mentioned drawings, are intended to cover not only the inclusion of steps or units but also the exclusion of not clearly listed steps or units, and can include other steps or units that are not clearly listed or inherent to the process, method, product or device.

[0026] To make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the present application. In addition, the technical features in each embodiment or single embodiment provided by the present application can be combined with each other to form new technical solutions, and such combination is not restricted by the order of steps and / or structure composition mode, but must be based on the realization by those skilled in the art. When the combination of technical solutions contradicts each other or cannot be realized, it should be considered that the combination of technical solutions does not exist and is not within the protection scope of the present application.

[0027] The present application provides a target tracking method based on loop detection in a multi-camera two-dimensional motion environment, aiming to solve the problem of difficult stable tracking of targets in the prior art under the condition of multi-camera two-dimensional dynamic motion, thereby improving the accurate identification and continuous tracking ability of targets in application scenarios.

[0028] As shown in Figure 1 The technical solution of the present application comprises the following steps: Step 1, initialization of visual perception device and space modeling.

[0029] The method first initializes a plurality of visual perception modules deployed on a movable platform (such as a mechanical device) in space, obtains the relative position, direction and field of view boundary of each device. In this process, a coordinate conversion relationship framework between multiple perspectives is constructed to support data fusion and target space unified identification under different perspectives, and to solve the problem of inconsistent visual information caused by relative motion between devices.

[0030] Step 2, image acquisition and preprocessing.

[0031] Each perception device continuously acquires image sequences of the external environment during motion, providing raw data support for subsequent identification and analysis. In order to improve the robustness of the system under various complex working conditions such as light changes, environmental disturbances and imaging blur, the system performs image preprocessing operations including resolution adjustment, image enhancement, noise suppression and the like on the collected images, thereby improving the overall quality and target saliency of the images.

[0032] Step 3, target region identification and candidate extraction.

[0033] In the processed image, the system uses a set of identification mechanisms with fast response capability to detect and locate possible target regions. This mechanism is based on a combination of deep feature extraction and region evaluation to preliminarily screen potential targets in the image and output their spatial position, bounding box and other basic information. This step not only improves the efficiency of subsequent processing, but also provides an initial reference for maintaining target identity between multiple frames.

[0034] In the processed image, the system uses a target detection algorithm based on convolutional neural network (CNN) to efficiently detect and accurately locate possible weed target regions. This identification mechanism mainly includes the following steps: First, a pre-trained target detection network (such as Faster R-CNN, DetectoRS, etc.) is used to extract deep features from the input image, generating multi-level feature maps to capture edge information, texture structure and spatial distribution features of the weed target. The system generates multiple potential candidate regions on the feature map through a region proposal generation module (Region Proposal Network, RPN) or anchor box mechanism. These candidate boxes are processed through classification and regression branches to output the confidence of the corresponding target class and the bounding box coordinate parameters.

[0035] Then, the system filters the candidate boxes according to the confidence threshold, removes low-probability false detection regions, and merges overlapping regions through non-maximum suppression (NMS) to finally obtain the weed target with accurate position and clear boundary in each image.

[0036] The CNN-based detection process provides reliable initialization information and spatial reference for the subsequent multi-frame tracking module, ensuring the continuous tracking effect of weeds in the background of camera motion.

[0037] Step 4, target structure feature modeling.

[0038] Considering that the appearance of the target in consecutive frames may differ significantly due to factors such as changes in viewing angle, occlusion, or pose transformation, the system introduces a structural appearance feature modeling strategy to extract deep features from the key regions of each target. Specifically, the system uses the target detection module in the above steps to obtain the target identification frame in the image, and on this basis, extracts key point information through a pose recognition neural network (such as OpenPose, etc.), or uses the ORB (Oriented FAST and Rotated BRIEF) algorithm to extract local key points and their direction-invariant descriptors within the identification frame. These feature points and their spatial relationships constitute the local structure representation of the target. This modeling process not only covers the geometric outline and spatial layout features of the target, but also retains stable structure reference points as appearance feature information of the target, enhancing the robustness and accuracy of the system in the cross-frame matching process. Even under complex motion or occlusion conditions, the system can maintain continuous tracking of the target identity.

[0039] Step 5, loop detection.

[0040] In a multi-camera two-dimensional motion environment, the continuous movement of visual perception devices inevitably introduces cumulative drift errors, especially in long-running or closed path scenarios, which will significantly affect the consistency and positioning accuracy of target tracking. Therefore, the system designs and introduces a loop detection (Loop Closure Detection) mechanism to identify the spatial overlap relationship between the current observation and the historical observation, thereby constructing a closed loop constraint and correcting the system's cumulative error through trajectory graph optimization techniques, improving the global consistency of the overall tracking system.

[0041] The system triggers the loop detection module in each key frame (or frame that meets certain motion distance and time interval). To improve efficiency, the system uses a pose pre-screening mechanism based on the camera's preliminary pose provided by the optical flow estimation module to quickly eliminate historical frames with too large spatial position differences from the current frame, reducing the search space.

[0042] The specific steps are as follows: Step 5.1 Device motion estimation and relative position perception.

[0043] In view of the relatively fixed motion mode of each perception device in a two-dimensional plane, the system designs an inter-frame position estimation mechanism to infer the spatial pose change of each device at adjacent time points. This mechanism does not rely on external positioning devices, but is based on the image sequence collected by the device itself, and has the advantages of lightweight and low delay. In specific implementation, the system can use a neural network based on optical flow estimation (such as RAFT or PWC-Net) to extract a dense motion field between consecutive frames, and then calculate the translation and rotation changes of key structural regions in the image; in the context of limited computing resources, traditional optical flow algorithms such as Lucas-Kanade can also be used to estimate local or global motion parameters by tracking the pixel-level displacement of feature points. Combined with the relative position relationship between devices determined in the initial modeling stage, this estimation mechanism can real-time correct and update the spatial coordination state between multiple devices, providing accurate support for the fusion of multi-source observation information and target trajectory consistency.

[0044] On the basis of preliminary screening, the system performs fine matching operation on the candidate frames to verify the accuracy of the loop relationship. In this step, the TOPIC method is used to match the target position information and structural feature information extracted from the historical frame information. If the number of feature matches between images, spatial consistency and similarity all meet the set threshold, it is determined that the pair of frames constitutes an effective loop. The TOPIC method is as follows: Step 5.2 Tracking matching based on TOPIC.

[0045] According to the position of the camera in the target position information map calculated in the previous step, the historical frame tracking information of the current frame is calculated according to the field of view projection. The motion feature loss (composed of IOU and center point normalized distance) and appearance feature loss (based on the average distance of the pose point center correction) between two targets are calculated respectively. From this, the motion loss matrix and appearance loss matrix between the historical frame tracking information and the current frame information are calculated. Through the two loss matrices, the TOPIC algorithm is applied for matching. The core idea of TOPIC is to use both motion features and appearance features for matching, and to solve possible matching conflicts through a two-round matching mechanism. In the first round of matching, the Hungarian algorithm is used to calculate the matching results based on appearance and motion respectively. Then, for the conflicting matches, TOPIC adaptively selects the more reliable features for the final matching according to the motion level. This mechanism effectively reduces false negatives (FN) and improves the overall tracking accuracy.

[0046] Once the loop frame is identified, the system constructs a loop edge with the current frame and adds it to the entire camera trajectory graph (Pose Graph). This graph contains the pose transformation relationship between all key frames, including: Sequential edges between adjacent frames: obtained based on motion estimation; Loop edges between non-adjacent frames: obtained based on image content overlap matching.

[0047] After the graph is constructed, the system performs a non-linear optimization on the whole trajectory graph using a graph optimization method (e.g., g2o, Ceres Solver, or iSAM2), aiming to minimize the re-projection error or relative pose error of all edges. This optimization process automatically corrects the accumulated pose errors caused by drift, achieving consistent reconstruction of the global trajectory of the system. Considering that the system is deployed on a multi-camera platform with spatial overlap or field-of-view intersection between cameras, to enhance the comprehensiveness and robustness of loop detection, the system introduces a cross-view collaborative detection mechanism: mapping image features from multiple cameras into a global coordinate system; using the relative pose relationship between each camera to infer the image projection area, assisting in determining whether historical frames constitute loop candidates; introducing multi-camera joint consistency judgment rules in the loop verification stage, establishing a closed loop edge only when there is a high-confidence match from multiple views. This strategy effectively improves the accuracy of loop detection in dynamic environments and avoids false loops caused by false matches.

[0048] Step 6, actual camera pose state correction based on Kalman filter.

[0049] To accurately perceive and dynamically track the real pose state of each visual perception device (i.e., camera) in a two-dimensional motion environment, the system introduces a Kalman filter to estimate and correct the spatial state of the camera at consecutive time points in real time. This module works collaboratively with the loop detection, motion estimation, and other subsystems, effectively suppressing local positioning errors caused by drift, occlusion, image blur, and other factors, and improving the stability and spatial consistency of the overall target tracking system.

[0050] Step 6.1, state modeling and prediction process The system takes the camera pose in a two-dimensional plane as the research object, modeling its motion state as a five-dimensional vector as follows:

[0051] Where: represents the two-dimensional spatial position of the camera in the current frame; represents the heading angle (orientation) of the camera in the plane; represents the linear velocity; represents the angular velocity (rotation rate).

[0052] Based on this state definition, the system establishes a simplified linear or quasi-linear motion prediction model for predicting the state value of the next frame from the current frame, using the following motion model:

[0053] where F is the state transition matrix; Q is the process noise covariance matrix; denotes the predicted state at the kth time instant.

[0054] The prediction stage combines the previous frame state estimation and the optical flow information between image frames to provide a priori inference for the camera pose change.

[0055] Step 6.2, observation modeling and error correction.

[0056] To correct the prediction error, the system introduces the high-confidence inter-frame matching results output by the loop detection module as the observation value :

[0057] The observation value is obtained through the loop detection module and has high global consistency. The system establishes the following observation model:

[0058] where H is the observation matrix; R is the observation noise covariance matrix; is the pose observation provided by the current frame loop or detection module.

[0059] The Kalman filter uses the following formula for state correction and covariance update:

[0060] where: is the Kalman gain, which measures the credibility of prediction and observation; is the estimated covariance of the current state; is the identity matrix.

[0061] Step 7, update the target information map and perform target state management and visual output.

[0062] After completing the current frame target matching and camera pose correction, the target information map is dynamically updated according to the matching results, including the position information of new targets, the extension of the trajectories of recognized targets, and the state maintenance (such as confidence, existence state, etc.). At the same time, all tracked targets are assigned a unique identifier and their tracking history in consecutive frames is recorded to maintain the life cycle management of the targets. Finally, the information such as the category, position, and motion trajectory of the targets is visualized and output, which can be called by the upper job control system and provides accurate and real-time perception support for downstream tasks.

[0063] The implementation basis of each embodiment of the present application is achieved by programmed processing of a device with processor function. Therefore, in engineering practice, the technical solutions and functions of each embodiment of the present application are packaged into various modules. Based on this actual situation, on the basis of each embodiment described above, an embodiment of the present application provides a camera two-dimensional motion target tracking system based on loop detection, which is used to execute the camera two-dimensional motion target tracking method based on loop detection in the method embodiment described above.

[0064] The system comprises: a first main module for acquiring and preprocessing image sequences of an external environment collected by each perception device during motion; a second main module for target region identification and candidate extraction based on the preprocessed images, to obtain targets in each image; a third main module for extracting local key points of each target and their direction invariance descriptors, to constitute a local structure representation of the target; a fourth main module for introducing a loop detection mechanism, identifying the spatial overlap relationship between current observation and historical observation, constructing a closed loop constraint, and correcting system cumulative error through trajectory graph optimization; and a fifth main module for introducing a Kalman filter to perform real-time estimation and error correction on the spatial state of the perception device at consecutive time points.

[0065] The camera two-dimensional motion target tracking system based on loop detection provided by the embodiment of the present application faces the problem that existing multi-target tracking algorithms are limited in application under the camera two-dimensional motion background, especially the technical bottlenecks such as short target stay time, strong background dynamic interference, frequent target generation and disappearance, and long-time span tracking difficulty in complex environments. By using the foregoing several modules and through cooperation of the modules, efficient, stable and continuous tracking of the target can be achieved under the condition of parallel motion of multiple cameras, the continuity of target tracking is effectively improved, the dependence on computing resources is reduced, and the comprehensive needs for real-time performance, accuracy and deployment flexibility in the scene are met.

[0066] It should be noted that the system embodiments provided by the present application are used to implement the methods in the method embodiments described above, and are also used to implement the methods in other method embodiments provided by the present application. The difference is only that the corresponding function modules are set, and the principle is basically the same as that of the above-mentioned system embodiments provided by the present application. As long as the person skilled in the art improves the modules in the above-mentioned system embodiments by combining technical features to obtain corresponding technical means and technical solutions composed of these technical means on the basis of the above-mentioned system embodiments, and the technical solutions have practicality, the corresponding system class embodiments are obtained, which are used to implement the methods in other method class embodiments.

[0067] In summary of the above embodiments, the present application proposes an efficient tracking scheme fusing loop detection mechanism and TOPIC multi-feature matching for the problem of target continuous tracking under the background of two-dimensional motion of the camera, and the technical key points of the scheme are as follows: 1. Target appearance structure feature extraction and optimization mechanism A feature construction method combining target stable structure information is proposed, which realizes the improvement of the distinguishing ability of the target under continuous views by modeling and vector coding the significant areas in the image. The method does not depend on the variable information such as color and texture of the target, has stronger occlusion robustness and time consistency, provides reliable support for subsequent multi-frame matching, and is the key foundation for realizing the identity maintenance of the target.

[0068] 2. Device space state estimation and dynamic position perception strategy The present application establishes a space motion estimation method without external positioning system and only relying on the observation data of the perception device itself. The strategy is based on the initialization information of the relative position relationship between devices, and completes the inference of the spatial pose change through the image change trend, provides necessary support for the spatial consistency of the targets in the system, and is the key guarantee for realizing the consistent tracking across devices.

[0069] 3. Camera pose correction mechanism combining Kalman filter and loop detection The Kalman filter is used to dynamically smooth and suppress abnormal jumps of the device space state estimation results, the matching information obtained by the TOPIC is used for loop detection, and the pose information obtained by the loop detection is used as an observation value for Kalman filter pose correction operation, which effectively reduces the cumulative error, avoids the deviation of target tracking, improves the long-term stability and positioning accuracy of the system, and reduces the cumulative error in long-time tracking. This module is crucial for the long-term stable operation of the system, and is the guarantee for improving the trajectory continuity and reducing the misrecognition.

[0070] 4. Mechanism for constructing and dynamically updating target information map The target position, trajectory and confidence in the map are continuously updated based on the tracking results after the pose correction, the target life cycle management (birth, disappearance and existence state maintenance) is realized, and the map provides data support for the subsequent decision module and is the key bridge for the application of the achievements.

[0071] It should be understood that the above contents not elaborated in detail can be regarded as the conventional means in the art.

[0072] It should be noted that the above examples are only used to illustrate the technical solutions of the present application, and are not intended to limit the present application; although the present application has been described in detail with reference to the foregoing examples, those skilled in the art should understand that the technical solutions recorded in the foregoing examples can still be modified, or some or all of the technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the technical solutions of the embodiments of the present application.

Claims

1. A camera two-dimensional motion object tracking method based on loop detection, characterized in that, The method comprises the following steps: acquiring image sequences of an external environment collected by each sensing device during movement and pre-processing the image sequences; performing target region identification and candidate extraction based on the pre-processed images to obtain targets in each image; extracting local key points and their direction invariance descriptors of each target to form a local structure representation of the target; introducing a loop detection mechanism to identify spatial overlap between current observations and historical observations, build a closed-loop constraint, and correct system cumulative errors through trajectory graph optimization; introducing a Kalman filter to estimate and correct errors in real time for the spatial state of the sensing device at consecutive time points.

2. The camera two-dimensional motion object tracking method based on loop detection according to claim 1, wherein, The method further comprises: spatially initializing a plurality of sensing devices deployed on a movable platform, acquiring the relative position, direction, and field of view boundary of each sensing device, and then building a coordinate conversion relationship framework between multiple perspectives.

3. The camera two-dimensional motion object tracking method based on loop detection according to claim 1, wherein, acquiring image sequences of an external environment collected by each sensing device during movement and pre-processing the image sequences, comprising: based on the image sequences of the external environment continuously acquired by each sensing device during movement, performing image preprocessing including resolution adjustment, image enhancement, and noise suppression.

4. The camera two-dimensional motion object tracking method based on loop detection according to claim 1, wherein, performing target region identification and candidate extraction based on the pre-processed images, comprising: using a pre-trained target detection network to extract deep features from the input image, generating multi-level feature maps, and generating multiple potential candidate boxes on the feature maps through a region candidate generation module or an anchor box mechanism. These candidate boxes are processed through classification and regression branches to output confidence and bounding box coordinate parameters for corresponding target classes; screening the candidate boxes according to the confidence threshold, and merging overlapping regions through non-maximum suppression to obtain targets in each image.

5. The camera two-dimensional motion object tracking method based on loop detection according to claim 1, wherein, extracting local key points and their direction invariance descriptors of each target, comprising: based on the targets in the acquired images, extracting local key points and their direction invariance descriptors within the target recognition box through a pose recognition neural network or using the ORB algorithm to form a local structure representation of the target.

6. The camera two-dimensional motion object tracking method based on loop detection according to claim 1, wherein, introducing a loop detection mechanism to identify spatial overlap between current observations and historical observations, build a closed-loop constraint, and correct system cumulative errors through trajectory graph optimization, comprising: based on inter-frame position estimation, predicting the spatial pose change of each sensing device at adjacent time points; based on the spatial pose change of each sensing device, combining TOPIC for tracking matching to determine whether the current frame and the historical frame constitute an effective loop; after identifying the loop frame, building a closed-loop edge constraint between the loop frame and the current frame and adding it to the entire sensing device trajectory graph; using graph optimization to perform non-linear optimization on the entire trajectory graph, with the goal of minimizing the reprojection error or relative pose error of all edges.

7. The camera two-dimensional motion object tracking method based on loop detection according to claim 6, wherein, based on the spatial pose change of each sensing device, combining TOPIC for tracking matching to determine whether the current frame and the historical frame constitute an effective loop, comprising: according to the calculated position of the sensing device in the target information map, calculating the tracking information of the historical frame of the current frame according to the field of view projection; The motion feature loss and the appearance feature loss are calculated for the two targets respectively, and then the motion loss matrix and the appearance loss matrix between the historical frame tracking information and the current frame information are calculated; TOPIC algorithm is applied through the two loss matrices to match the current frame and the historical frame to determine whether they constitute an effective loop.

8. The camera two-dimensional motion object tracking method based on loop detection according to claim 1, wherein, A Kalman filter is introduced to estimate and correct the error of the spatial state of the perception device at consecutive time points, including: The motion state of the perception device pose on a two-dimensional plane is modeled; Based on the modeled motion state, a linear or quasi-linear motion prediction model is established to predict the state value of the next frame at the current frame; The high-confidence inter-frame matching result output by the loop detection is introduced as an observation value to correct the prediction error.

9. The camera two-dimensional motion object tracking method based on loop detection according to claim 1, wherein, After completing the target matching of the current frame and the camera pose correction, the method further includes: According to the matching result, the target information map is dynamically updated, including the position information of the new target, the trajectory extension and state maintenance of the identified target; All tracking targets are assigned a unique identifier and their tracking history in consecutive frames is recorded; The category, position and motion trajectory information of the target are visualized and output.

10. A camera two-dimensional motion object tracking system based on loop detection, characterized in that, It includes: The first main module is used to acquire the image sequence of the external environment collected by each perception device during the motion process and to pre-process it; The second main module is used to identify the target area and extract the candidate based on the pre-processed image, to obtain the target in each image; The third main module is used to extract the local key points and their direction invariance descriptors of each target, to constitute the local structure representation of the target; The fourth main module introduces a loop detection mechanism to identify the spatial overlap relationship between the current observation and the historical observation, to build a closed loop constraint, and to correct the system cumulative error through trajectory graph optimization; The fifth main module introduces a Kalman filter to estimate and correct the error of the spatial state of the perception device at consecutive time points.

Images