Railway catenary foreign object tracking method based on global-local attention mechanism

Abstract

A railway catenary foreign object tracking method based on global-local attention mechanism, comprising: using a global perception module to decouple background motion and shield train interference for a monitoring video stream, generating a saliency motion field, and then screening out initial foreign object targets; using a deep neural network tracker based on an attention mechanism to locally focus on tracking the targets, outputting position prediction results and tracking confidence; adaptively fusing the tracker output and Kalman filter output to obtain the final tracking position; when the tracking confidence is lower than a preset loss threshold, starting a recovery mechanism to realize automatic recovery of the target. The present application solves the problems in the prior art of relying on manual initialization, being easily affected by train interference drift, having a high false tracking rate, and being difficult to recover after target loss, and realizes unmanned tracking of railway catenary foreign objects.

Description

Technical Field

[0001] This invention relates to the field of railway safety monitoring technology, specifically to a method for tracking foreign objects in railway overhead contact lines based on a global-local attention mechanism. Background Technology

[0002] As a core component of the high-speed train power supply system, the operational stability of the overhead contact system is directly related to railway transportation safety. However, in the open external environment, lightweight floating objects such as balloons, water surface mulch, and kites often intrude into the overhead contact system area due to wind. Once these objects become entangled in high-voltage transmission lines, they can easily cause major accidents such as short circuits, train stoppages, and even sudden fires, posing significant safety hazards and economic losses to railway operations.

[0003] Currently, detection and tracking technologies for foreign objects in railway overhead contact lines can be broadly categorized into two types based on sensing methods: contact and non-contact. Contact detection is mainly used in perimeter protection and rockfall monitoring, with typical examples including photoelectric fence monitoring systems and distributed fiber optic perimeter security systems. While these technologies offer advantages such as strong anti-interference capabilities and low false alarm rates, they are essentially passive triggering mechanisms, only triggering backend alarms when a foreign object comes into contact with the sensor. They cannot provide early warnings, nor can they obtain semantic information such as the type and size of the foreign object. Non-contact detection, such as infrared thermal imaging or lidar scanning, involves active detection. However, in the complex environment of high voltage, high electromagnetic interference, and varying lighting conditions along railway lines, they are prone to high false alarm rates and have limited coverage. Coupled with high construction and maintenance costs, their large-scale deployment is severely constrained.

[0004] Given the aforementioned limitations of sensor-based solutions, computer vision-based non-contact solutions have become a key research focus due to their low cost and rich information content. However, existing vision-based Single Object Tracking (SOT) algorithms still suffer from problems in unattended railway scenarios, such as inability to effectively achieve fully automatic initialization and detection of foreign objects, high error tracking rate, and lack of loss recovery capabilities. Summary of the Invention

[0005] The purpose of this invention is to provide a railway overhead contact line foreign object tracking method based on a global-local attention mechanism, in order to solve the problems of existing technologies that cannot effectively decouple background motion, cannot actively shield train interference, have a high error tracking rate, and lack loss recovery capabilities.

[0006] A method for tracking foreign objects in railway overhead contact lines based on a global-local attention mechanism is applied to a tracking system. The system includes a global sensing module and a local focusing module. The method includes: Step S1: Obtain the railway catenary monitoring video stream through the global perception module, calculate the original optical flow field of the scene in the monitoring video stream using dense optical flow technology, decouple the original optical flow field from background motion using the multi-scale motion estimator based on hybrid sparse features built into the global perception module to eliminate camera motion interference, then remove the motion component of the train area to obtain the salient motion field, extract candidate target areas based on the salient motion field, and then select the initial foreign object target from the candidate target areas using a multi-factor scoring function; Step S2: Input the initial foreign object target's location information into the local focusing module, use a deep neural network tracker based on the attention mechanism to perform local focusing tracking of the initial foreign object target, extract target features within the search area of ​​the current frame, interact the target features with the template features, and output the target's location prediction result and tracking confidence. Step S3: The motion state of the target is predicted using a Kalman filter. The position prediction result of the deep neural network tracker is adaptively fused with the motion state prediction result of the Kalman filter. Based on the tracking confidence of the target, the fusion weight of the adaptive fusion is dynamically adjusted through a linear mapping function to obtain the final tracking position of the target. The final tracking position is used to update the state estimate of the Kalman filter. Step S4: When the tracking confidence is lower than the preset loss threshold, the recovery mechanism is activated. The recovery mechanism is as follows: a search range is generated within the prediction region of the Kalman filter, the semantic features of the candidate target are matched with the dynamic historical high confidence feature library, and interference verification is performed in combination with the interference region mask to achieve automatic target recovery.

[0007] The railway overhead contact line foreign object tracking method based on a global-local attention mechanism provided by the present invention has the following beneficial effects: 1. Fully automated unattended tracking is achieved: This invention breaks through the limitation of traditional tracking algorithms that rely on manual labeling of targets in the first frame. By decoupling the background motion of the original optical flow field through a multi-scale motion estimator based on hybrid sparse features, multi-scale motion detection is achieved. By removing the motion components of the train area, a salient motion field is obtained, and salientity assessment is achieved. Candidate target regions are extracted based on the salient motion field, and initial foreign object targets are selected from the candidate target regions through a multi-factor scoring function. Thus, fully automated initialization and discovery of foreign object targets is effectively achieved without manual intervention.

[0008] 2. Effectively eliminates background motion interference: For camera vibration and background motion issues in train operation scenarios, background motion decoupling technology is used to eliminate pseudo-motion interference. For trains, which are frequently encountered large dynamic interference objects, a significant motion field is calculated from the original optical flow field by combining background motion compensation and interference object shielding. This achieves an active interference shielding mechanism, effectively preventing the tracker from drifting onto trains and other interference objects, and significantly reducing the error tracking rate.

[0009] 3. This invention combines the powerful feature representation capability of deep neural network trackers with the robust motion estimation capability of Kalman filters: In scenarios such as foreign objects being briefly obscured by trains or sudden changes in motion patterns, the adaptive fusion mechanism can dynamically adjust the weights of the tracker and filter according to the tracking confidence, ensuring stable tracking performance under various working conditions.

[0010] 4. Intelligent loss recovery capability: When the tracking confidence level is lower than the preset loss threshold, the present invention realizes automatic target recovery by activating the recovery mechanism. It can automatically enter the search recovery mode after foreign objects are blocked, which significantly improves the trajectory survival probability and effectively avoids the common failure mode of the tracker drifting to strong interference objects such as trains. Attached Figure Description

[0011] Figure 1 A flowchart illustrating the foreign object tracking method for railway overhead contact lines based on a global-local attention mechanism provided by this invention. Figure 2 A comparison image of the original input image of an exemplary railway catenary scene and the target feature focusing heatmap of the deep neural network tracker based on the attention mechanism of this invention; Figure 3 This is a comparison chart of the tracking accuracy curves of the present invention and existing mainstream tracking algorithms on the OTB100 general dataset; Figure 4 This is a comparison chart of the tracking success rate curves of the present invention and existing mainstream tracking algorithms on the OTB100 general dataset; Figure 5 This is a visual comparison chart of the tracking performance of this invention and existing mainstream tracking algorithms in a real railway overhead contact system scenario. Detailed Implementation

[0012] To facilitate understanding of the present invention, a more complete description will be given below with reference to various embodiments. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.

[0013] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0014] This invention provides a method for tracking foreign objects in railway overhead contact lines based on a global-local attention mechanism. The method is applied to a tracking system comprising two core components: a Global Perception Module (GPM) and a Local Focus Module (LFM), which work collaboratively in a cascaded manner. The video stream first enters the GPM, which autonomously detects abnormal moving targets from a complex dynamic background and completes tracking initialization by decoupling background motion and shielding against train interference. Subsequently, the LFM takes over the initialized target, performing high-precision focusing tracking and status maintenance. When the target is occluded or lost, it automatically recaptures the target using deep semantic memory matching and interference verification mechanisms. The organic integration of the two modules creates a closed-loop system encompassing autonomous detection, precise tracking, and loss recovery.

[0015] Please see Figure 1 The method of the present invention includes steps S1-S4: Step S1: Obtain the railway catenary monitoring video stream through the global perception module, calculate the original optical flow field of the scene in the monitoring video stream using dense optical flow technology, and decouple the original optical flow field from background motion by using the multi-scale motion estimator based on hybrid sparse features built into the global perception module to eliminate camera motion interference. Then, remove the motion component of the train area to obtain the salient motion field. Based on the salient motion field, extract candidate target areas, and then use a multi-factor scoring function to select the initial foreign object target from the candidate target areas.

[0016] The core task of GPM is to automatically detect abnormal moving targets in a scene from consecutive video frames without human intervention. GPM mainly performs three core steps: background motion decoupling, dynamic interference masking, and candidate target scoring and initialization.

[0017] First, let's explain the background motion decoupling process of GPM. This requires acquiring the railway overhead contact line monitoring video stream first, and then using dense optical flow technology to calculate the original optical flow field of the scene. .

[0018] In dynamic camera scenarios, directly using the raw optical flow field to detect moving targets is severely hampered by background motion. To eliminate background spurious motion introduced by camera vibration and obstacles, this invention designs a robust multi-scale camera motion compensation (CMC) module. This module incorporates a multi-scale motion estimator based on hybrid sparse features. It employs the computationally efficient ORB (Oriented Fast and Rotated BRIEF) feature extraction algorithm and combines it with the Lucas-Kanade (LK) sparse optical flow estimation algorithm for feature tracking.

[0019] Specifically, the process by which the multi-scale motion estimator decouples the original optical flow field from background motion to eliminate camera motion interference includes steps S101-S103: Step S101: Extract image feature points from the current frame and the previous frame using a multi-scale motion estimator, and calculate the consistency of forward and backward optical flow to screen stable matching points. Specifically, firstly, highly saliency corner points are extracted at multiple scale levels of the image pyramid, and then a rigorous forward-backward optical flow consistency verification is performed on each feature point, expressed as:

[0020] in, for The first extracted from the time frame The pixel coordinate vector of each feature point For the feature points calculated by the forward optical flow operator based on the sparse optical flow estimation algorithm, from Time frame to Optical flow mapping of time frames, For feature points calculated by the backward optical flow operator based on the sparse optical flow estimation algorithm, from Time frame to Optical flow mapping of time frames, The preset forward-backward consistency threshold, This represents the L2 norm.

[0021] This two-way verification mechanism effectively eliminates erroneous matches caused by changes in lighting or occlusion. Only stable matching points that pass verification are used for subsequent motion estimation.

[0022] Step S102: Use a robust estimation algorithm to fit an affine transformation matrix based on the stable matching point to describe the global motion of the background.

[0023] In this embodiment, the robust estimation algorithm is preferably the Random Sample Consensus Algorithm (RANSAC). The RANSAC algorithm is used to fit an affine transformation matrix from these stationary points, which describes the global motion of the background. Compared to dense matching across the entire image, this sparse feature tracking strategy reduces the computational load by two orders of magnitude while maintaining the accuracy of background estimation, thereby supporting the stable operation of the system at high frame rates.

[0024] Step S103: Calculate the background compensation optical flow field using the affine transformation matrix, perform differential calculation between the calculated field and the original optical flow field, eliminate the background optical flow caused by camera vibration or motion, and decouple the background motion from the original optical flow field.

[0025] The following describes the dynamic interference shielding process of GPM. In a railway scenario, there are predictable large dynamic interference objects, namely trains. If left untreated, the optical flow of the train's motion will mask the minute movements of the foreign object. Therefore, this invention designs an active, semantic-based visual interference shielding mechanism.

[0026] First, an interference detection model is introduced, responsible for detecting train-related objects in each frame. To balance the real-time performance and high recall of railway monitoring, this detection model employs a lightweight, anchor-free detection architecture. Its core architecture utilizes a cross-stage local connectivity (CSP) backbone and a decoupled head mechanism to ensure effective capture of high-speed moving targets even at low inference speeds. To further improve the model's sensitivity to specific interference sources, supervised transfer learning is performed on a dedicated railway dataset, focusing on generating highly reliable train binarization masking codes. .

[0027] Finally, by combining background motion compensation and interference shielding, the original optical flow field is obtained. Calculate the saliency of the motion field The expression is:

[0028] in, Indicates a salient sports field. For the original optical flow field, To compensate for the optical flow field in the background, For train binary shielding mask, It represents the Hadamaji.

[0029] In addition, the interference detection model also performed optical flow attenuation processing on the track region in the lower half of the image, further suppressing radial optical flow interference caused by the lens movement.

[0030] Through the above processing, motion interference at multiple levels can be effectively suppressed: background optical flow is decoupled and eliminated through affine transformation, the train area is actively shielded through a shielding mask, and the track area is suppressed through optical flow attenuation processing. In the end, the salient motion field that is retained only contains real foreign object motion information.

[0031] Finally, the candidate target scoring and initialization process of GPM will be explained.

[0032] Based on the above significant sports field Candidate target regions are extracted, and a multi-factor scoring function is designed for automatic screening. The specific process is as follows: First, for salient sports fields Amplitude calculation is performed, regions with optical flow amplitudes exceeding a preset motion threshold are binarized, noise is eliminated through morphological operations, and the contours of connected regions are extracted as candidate target regions.

[0033] Then, for each candidate target region, a comprehensive score is calculated based on three dimensions: area appropriateness, centrality, and motion salience. Specifically, the expression for the multi-factor scoring function is:

[0034] in, For the overall score, The area is rated as moderate. For centrality scoring, For the significance score of the movement, , , These are the preset weighting coefficients.

[0035] This multi-factor scoring function comprehensively considers the geometric, positional, and motion characteristics of the target, ensuring that the bounding box corresponding to the highest-scoring contour has a very high probability of being the object of interest. Finally, the selected... The bounding box corresponding to the highest contour is used as the initial foreign object target. This bounding box is used as the final output of GPM to initialize and start LFM.

[0036] After the initial foreign object target is selected in step S1, the template initialization stage needs to be entered first to prepare for the local focusing tracking in step S2.

[0037] Specifically, using the bounding box of the initial foreign object target as a reference, a target region image is cropped from the current frame as a reference template image. Specifically, the reference template image and the search region of the current frame are segmented into a P×P patch sequence, which is then sequentially input into a position encoder and a Transformer encoder to store the initial target's appearance feature representation. Simultaneously, the target's deep semantic features are extracted as initial appearance descriptors and stored in a dynamic feature library for potential subsequent loss recovery.

[0038] Furthermore, the state vector of the motion prediction model is initialized based on the initial position of the target and the bounding box size. In this embodiment, the motion prediction model is a Kalman filter (KF), and its state vector... , , , , These represent the x-coordinate of the target center, the y-coordinate of the target center, the width, and the height, respectively. , , , respectively , , , The corresponding first derivative, This indicates transpose. In the initial state, the position component is determined by the bounding box of the initial foreign object, and the velocity component is initialized to zero.

[0039] After template initialization is completed, the local focus tracking stage of step S2 is entered.

[0040] Step S2: Input the initial foreign object target's location information into the local focusing module, use a deep neural network tracker based on the attention mechanism to perform local focusing tracking of the initial foreign object target, extract target features within the search area of ​​the current frame, interact the target features with the template features, and output the target's location prediction result and tracking confidence.

[0041] To address the drastic visual changes of the target during flight, this invention employs a discriminative core tracking engine based on the Vision Transformer. Its advantage lies in its self-attention mechanism, which adaptively focuses on the most information-rich and stable parts of the target, while the cross-attention mechanism effectively models the long-range dependencies between the template and the search area.

[0042] In this embodiment, step S2 specifically includes steps S201-S204: Step S201: Input the initial foreign object target position information into the local focusing module. Using the initial foreign object target position information as a reference, crop out the region where the initial foreign object target is located from the initial frame as a reference template image. Complete the template initialization of the deep neural network tracker based on the attention mechanism. Store the features of the reference template image into the template cache of the deep neural network tracker. In the current frame, with the target tracking position output from the previous frame as the center, delineate a search area of ​​a fixed size and crop out the search area of ​​the current frame. Step S202: Divide the reference template image and the search region of the current frame into fixed-size template Patch sequences and search region Patch sequences, respectively. Add learnable position codes to each Patch sequence to preserve the spatial position information of the target and the background. Then, concatenate the template Patch sequence and the search region Patch sequence in the sequence dimension to obtain the concatenated Patch sequence. Step S203: Input the spliced ​​Patch sequence into the Transformer encoder and output the enhanced feature map of the search region that incorporates template prior information through the self-attention mechanism; Specifically, the self-attention mechanism is used to perform the following operations: 1. Extract the target stability features of the template region and the deep semantic and texture features of the search region respectively, while suppressing background interference information such as contact wires and tracks; 2. Model the long-range dependency between the template region and the search region, allowing each position in the search region to adaptively focus on the key features of the foreign object target in the template, thus completing the deep interaction between the template and the search features.

[0043] In step S204, the enhanced feature map of the search region is then used as input to complete the decoding through the prediction head of the deep neural network tracker, outputting a target boundary regression map and a target classification score map. The position prediction result of the output target is obtained by mapping from the target boundary regression map, and the tracking confidence is calculated based on the peak value of the target classification score map. The value of the tracking confidence reflects the tracker's confidence in the current prediction result.

[0044] Step S3: The motion state of the target is predicted using a Kalman filter. The position prediction result of the deep neural network tracker is adaptively fused with the motion state prediction result of the Kalman filter. Based on the tracking confidence of the target, the fusion weight of the adaptive fusion is dynamically adjusted through a linear mapping function to obtain the final tracking position of the target. The final tracking position is used to update the state estimate of the Kalman filter.

[0045] Among these, appearance-based tracking strategies are prone to failure when the target is occluded or exhibits rapid motion blur. Therefore, this invention introduces a Kalman filter for motion state prediction. Its state vector... This includes the center coordinates, width, height, and first derivative of the bounding box. Based on this, the present invention introduces an adaptive fusion strategy, adaptively fusing the position prediction results of the deep neural network tracker with the motion state prediction results of the Kalman filter. Furthermore, according to the target's tracking confidence, the fusion weights of the adaptive fusion are dynamically adjusted through a linear mapping function to obtain the final tracking position of the target, satisfying the following equation:

[0046]

[0047] in, For the final tracking position, The position prediction results of the deep neural network tracker The motion state prediction results of the Kalman filter; The fusion weights are positively correlated with the tracking confidence; when the tracking confidence is high... Increase when tracking confidence is low Decrease; Track the confidence level for the current frame. , The preset mapping parameters, and satisfy the following conditions: , .

[0048] Linear mapping ensures that when the tracking confidence is high, Increase, tracker position prediction results Dominant; when tracking confidence is low, Reduce the motion state prediction results of the Kalman filter This design ensures that even at low confidence levels, the tracker's appearance information retains a certain weight, avoiding inertial errors that may result from relying solely on motion prediction. It achieves an intelligent collaborative strategy that relies on appearance for effective tracking and on motion to handle occlusion interference.

[0049] Using the final tracking position after fusion The state estimate of the Kalman filter is updated to provide updated historical trajectory information for motion prediction in the next frame.

[0050] Step S4: When the tracking confidence is lower than the preset loss threshold, the recovery mechanism is activated. The recovery mechanism is as follows: a search range is generated within the prediction region of the Kalman filter, the semantic features of the candidate target are matched with the dynamic historical high confidence feature library, and interference verification is performed in combination with the interference region mask to achieve automatic target recovery.

[0051] When tracking confidence When the target value is below the preset loss threshold, it indicates that the target may have been lost due to reasons such as being obscured by a train or briefly leaving the field of vision. In this case, the system will automatically activate the recovery mechanism, which specifically involves: Maintain a dynamic historical high-confidence feature library to store the target's deep semantic features under the most recent K high-confidence frames; Extract the semantic features of the candidate target, calculate the time-weighted average cosine similarity between the semantic features of the candidate target and all historical features in the dynamic historical high-confidence feature library, take the maximum similarity as the embedding re-identification score, and at the same time perform template matching on the candidate target to obtain the template matching score. Specifically, this invention performs target re-identification based on deep semantic features. To overcome the limitations of lightweight trackers in handling drastic target deformation, this invention constructs a semantic matching mechanism based on deep memory features (M-Feat). This mechanism independently utilizes the powerful representational capabilities of the visual Transformer backbone network in the tracking engine to directly extract the feature map of the search region encoded in the last layer of the target. Global Average Pooling (GAP) compresses the high-dimensional feature map into a fixed-dimensional feature vector, eliminating the need for an additional feature extraction network. This ensures the isomorphism between the features and the tracking model, avoids semantic bias caused by feature heterogeneity, and maintains the lightweight nature of the system. Furthermore, a dynamic historical high-confidence feature library is maintained during the tracking process. This feature library is used to store the target depth semantic features at high confidence levels in the most recent K frames in real time. In this embodiment, K=20. The feature library is dynamically updated using a first-in, first-out (FIFO) queue structure. Specifically, the semantic features of the current frame are only stored in the feature library when the tracking confidence is higher than a preset storage threshold of 0.5, ensuring that only high-confidence features are stored in the memory. All features stored in the feature library, as well as subsequently extracted candidate features, undergo L2 normalization to unify the vector scale and ensure the effectiveness of cosine similarity calculation.

[0052] In this embodiment, the following equation is satisfied during the calculation of the time-weighted average cosine similarity between the semantic features of the candidate target and all historical features in the dynamic historical high-confidence feature library:

[0053] in, The semantic features representing candidate targets. Indicates the first in the dynamic historical high-confidence feature library A historical characteristic, This represents the total number of historical features in the dynamic historical high-confidence feature library. express and Time-weighted average cosine similarity; For the first The weights corresponding to each historical feature increase with the storage time, meaning that the more recently stored historical vectors are assigned higher weights, and the sum of all weights is 1.

[0054] The consistency between candidate targets and historical memories in the semantic space was quantified. After calculating the time-weighted average cosine similarity corresponding to each historical feature, the maximum similarity was used as the embedding re-identification score. .

[0055] To improve the robustness of the recovery, this invention introduces template matching for auxiliary verification based on semantic re-identification. For each candidate target region, normalized cross-correlation template matching is performed using the most recent target template images stored in the memory bank to obtain a template matching score. .

[0056] Then, the template matching score and the embedding re-identification score are weighted and fused to obtain the comprehensive re-identification score, expressed as:

[0057] in, This represents the overall re-identification score. Scoring based on template matching. To embed the re-identification score, These are preset weighting coefficients. In this embodiment... Template matching provides local appearance consistency at the pixel level, while deep embedding provides global feature consistency at the semantic level. Their complementary fusion effectively improves the accuracy of the reconstruction. Only when the overall re-identification score is... It is the highest among all candidates, and A candidate is considered a valid recovery target only if its similarity to the pre-set re-identification threshold is exceeded.

[0058] Next, the overlap rate between the candidate target with the highest overall re-identification score and the current frame's train binarized mask is calculated. This is the last line of defense to ensure the robustness of target recovery. Interference verification is performed on the candidate target with the highest overall re-identification score. The system calls the interference detection model from step S103 to calculate its overlap rate with the current frame's train binarized mask. The overlap rate is defined as the proportion of pixels within the candidate target's bounding box that belong to the train binarized mask region to the total number of pixels in the candidate target's bounding box.

[0059] Only candidate targets with an overlap rate lower than a preset interference threshold will be ultimately adopted. If the overlap rate is lower than the preset interference threshold, the candidate target is determined to be a valid foreign object target, the system resumes tracking the target, and updates the template and Kalman filter state; if the overlap rate is greater than or equal to the preset interference threshold, the candidate area is considered to be affected by train interference, the system refuses to resume tracking, and continues to maintain the search state to wait for the next frame's judgment.

[0060] Through the multi-level recovery mechanism of feature memory, semantic re-identification, template-assisted verification, and interference verification, and by combining deep semantic memory and active anti-occlusion semantic interference verification, the recovery success rate under long-term occlusion and complex scenarios is significantly improved, and the common failure mode of tracker drifting to strong interference objects such as trains is effectively avoided.

[0061] Figure 2 A comparison is shown between the original input image of an exemplary railway overhead contact line scene and the target feature focusing heatmap of the deep neural network tracker based on the attention mechanism of this invention. In the heatmap, brighter areas represent higher feature response intensity. Figure 2 As can be seen, the tracker of this invention can adaptively focus on the main body area of ​​the kite or foreign object, accurately capture the core stable features of the target, and effectively suppress the feature response of background areas such as contact wires, towers, and signal machines. It solves the problems of traditional algorithms being easily interfered with in complex backgrounds and having insufficient discriminative feature extraction, laying the foundation for subsequent accurate tracking.

[0062] Figure 3 This image shows a comparison of the tracking accuracy curves of the present invention and existing mainstream tracking algorithms on the OTB100 general dataset. Figure 4 The graph shows a comparison of the tracking success rate curves of the present invention and existing mainstream tracking algorithms on the OTB100 general dataset. Both the present invention and existing mainstream tracking algorithms adopt the OPE (One-Pass Evaluation) protocol, which is common in the field of target tracking. Figure 3 and Figure 4 In this context, GLA represents the present invention. Existing mainstream tracking algorithms include SiamRPN++, Ocean, OStrack, DaSiamRPN, GradNet, SiamRPN, SRDCF, SiamFC, and fDSST.

[0063] from Figure 3 and Figure 4 As can be seen, the tracking accuracy of the present invention reaches 0.901 and the tracking success rate reaches 0.691. Both indicators are better than existing mainstream tracking algorithms such as SiamRPN++, Ocean, and OSTrack. This proves that the present invention not only has excellent performance in the specific scenario of railway catenary, but also has extremely strong general tracking capability and generalization, and can adapt to the target tracking needs under different deformation and interference scenarios.

[0064] Figure 5 The image shows a visual comparison of the tracking performance of this invention and existing mainstream tracking algorithms in a real railway overhead contact system scenario, after a kite or foreign object undergoes rapid movement and large non-rigid deformation. Figure 5In this context, GLA represents the present invention. Existing mainstream tracking algorithms include KCF, Crst, SiamMask, SiamRPN, and OSTrack. Figure 5 The scene includes typical high-interference elements in railway scenarios such as overhead contact lines, metal towers, and signal lights.

[0065] from Figure 5 It can be seen that correlation filtering algorithms such as KCF and Crst, lacking scale adaptation mechanisms and deep semantic feature extraction capabilities, cannot adapt to the rapid deformation and scale changes of the target. This leads to the introduction of significant background noise into the tracking box, resulting in severe scale distortion and positional shift. Siam network algorithms such as SiamRPN and SiamMask suffer from feature similarity measurement mechanisms that are easily interfered with by the high-frequency textured background of overhead contact lines and signaling systems, misidentifying background areas as targets and causing the tracking box to completely detach from the target and drift into the background signaling area. While the OSTrack algorithm achieves accurate tracking in the initial stage thanks to the feature extraction capabilities of the Transformer architecture, its lack of background semantic suppression mechanisms specific to railway scenarios leads to tracking box shrinkage after target deformation, only covering a localized area of ​​the target, resulting in a significant decrease in tracking accuracy. In contrast, this invention, through the anti-deformation feature extraction of the local focusing module, the background semantic suppression mechanism, and the train interference shielding design, can still accurately fit the complete outline of the foreign object target under conditions of severe target deformation and complex background interference, without offset, drift, or scale distortion. It effectively solves the tracking failure problem of existing algorithms in railway scenarios and has extremely strong scene adaptability and tracking robustness.

[0066] In summary, the foreign object tracking method for railway overhead contact lines based on the global-local attention mechanism described above has the following beneficial effects: 1. Fully automated unattended tracking is achieved: This invention breaks through the limitation of traditional tracking algorithms that rely on manual labeling of targets in the first frame. By decoupling the background motion of the original optical flow field through a multi-scale motion estimator based on hybrid sparse features, multi-scale motion detection is achieved. By removing the motion components of the train area, a salient motion field is obtained, and salientity assessment is achieved. Candidate target regions are extracted based on the salient motion field, and initial foreign object targets are selected from the candidate target regions through a multi-factor scoring function. Thus, fully automated initialization and discovery of foreign object targets is effectively achieved without manual intervention.

[0067] 2. Effectively eliminates background motion interference: For camera vibration and background motion issues in train operation scenarios, background motion decoupling technology is used to eliminate pseudo-motion interference. For trains, which are frequently encountered large dynamic interference objects, a significant motion field is calculated from the original optical flow field by combining background motion compensation and interference object shielding. This achieves an active interference shielding mechanism, effectively preventing the tracker from drifting onto trains and other interference objects, and significantly reducing the error tracking rate.

[0068] 3. This invention combines the powerful feature representation capability of deep neural network trackers with the robust motion estimation capability of Kalman filters: In scenarios such as foreign objects being briefly obscured by trains or sudden changes in motion patterns, the adaptive fusion mechanism can dynamically adjust the weights of the tracker and filter according to the tracking confidence, ensuring stable tracking performance under various working conditions.

[0069] 4. Intelligent loss recovery capability: When the tracking confidence level is lower than the preset loss threshold, the present invention realizes automatic target recovery by activating the recovery mechanism. It can automatically enter the search recovery mode after foreign objects are blocked, which significantly improves the trajectory survival probability and effectively avoids the common failure mode of the tracker drifting to strong interference objects such as trains.

Claims

1. A method for tracking foreign objects in railway overhead contact lines based on a global-local attention mechanism, characterized in that, Applied to a tracking system, the system including a global sensing module and a local focusing module, the method includes: Step S1: Obtain the railway catenary monitoring video stream through the global perception module, calculate the original optical flow field of the scene in the monitoring video stream using dense optical flow technology, decouple the original optical flow field from background motion using the multi-scale motion estimator based on hybrid sparse features built into the global perception module to eliminate camera motion interference, then remove the motion component of the train area to obtain the salient motion field, extract candidate target areas based on the salient motion field, and then select the initial foreign object target from the candidate target areas using a multi-factor scoring function; Step S2: Input the initial foreign object target's location information into the local focusing module, use a deep neural network tracker based on the attention mechanism to perform local focusing tracking of the initial foreign object target, extract target features within the search area of ​​the current frame, interact the target features with the template features, and output the target's location prediction result and tracking confidence. Step S3: The motion state of the target is predicted using a Kalman filter. The position prediction result of the deep neural network tracker is adaptively fused with the motion state prediction result of the Kalman filter. Based on the tracking confidence of the target, the fusion weight of the adaptive fusion is dynamically adjusted through a linear mapping function to obtain the final tracking position of the target. The final tracking position is used to update the state estimate of the Kalman filter. Step S4: When the tracking confidence is lower than the preset loss threshold, the recovery mechanism is activated. The recovery mechanism is as follows: generate a search range within the prediction area of ​​the Kalman filter, match the semantic features of the candidate target with the dynamic historical high confidence feature library, and perform interference verification by combining the interference area mask to achieve automatic target recovery. In step S1, the expression for the multi-factor scoring function is: in, For the overall score, The area is rated as moderate. For centrality scoring, For the significance score of the movement, , , These are preset weighting coefficients; In step S4, the recovery mechanism is initiated as follows: Maintain a dynamic historical high-confidence feature library to store the target's deep semantic features under the most recent K high-confidence frames; Extract the semantic features of the candidate target, calculate the time-weighted average cosine similarity between the semantic features of the candidate target and all historical features in the dynamic historical high-confidence feature library, take the maximum similarity as the embedding re-identification score, and at the same time perform template matching on the candidate target to obtain the template matching score. The template matching score and the embedded re-identification score are weighted and fused to obtain the comprehensive re-identification score; Calculate the overlap rate between the candidate target with the highest comprehensive re-identification score and the current frame's train binarized masking mask; If the overlap rate is lower than the preset interference threshold, the candidate target is determined to be a valid target and tracking is resumed.

2. The method for tracking foreign objects in railway overhead contact lines based on a global-local attention mechanism according to claim 1, characterized in that, In step S1, the original optical flow field is decoupled from background motion by the multi-scale motion estimator based on hybrid sparse features built into the global perception module to eliminate camera motion interference. Specifically, this includes: Image feature points of the current frame and the previous frame are extracted by a multi-scale motion estimator, and stable matching points are selected by calculating the consistency of forward and backward optical flow. A robust estimation algorithm is used to fit an affine transformation matrix based on the stable matching point to describe the global motion of the background. The background compensated optical flow field is calculated using the affine transformation matrix, and then differentially calculated with the original optical flow field to achieve background motion decoupling.

3. The railway overhead contact line foreign object tracking method based on global-local attention mechanism according to claim 2, characterized in that, During the process of calculating the consistency between forward and backward optical flows to select stable matching points, the following equation is satisfied: in, for The first extracted from the time frame The pixel coordinate vector of each feature point For the feature points calculated by the forward optical flow operator based on the sparse optical flow estimation algorithm, from Time frame to Optical flow mapping of time frames, For feature points calculated by the backward optical flow operator based on the sparse optical flow estimation algorithm, from Time frame to Optical flow mapping of time frames, The preset forward-backward consistency threshold, This represents the L2 norm.

4. The railway overhead contact line foreign object tracking method based on global-local attention mechanism according to claim 3, characterized in that, In step S1, the motion components of the train region are removed using a train binarization mask, and the resulting salient motion field satisfies the following equation: in, Indicates a salient sports field. For the original optical flow field, To compensate for the optical flow field in the background, For train binary shielding mask, It represents the Hadamaji.

5. The method for tracking foreign objects in railway overhead contact lines based on a global-local attention mechanism according to claim 4, characterized in that, The train binarized shielding mask is generated by a pre-trained interference detection model, which adopts a detection model architecture based on a cross-stage local connectivity backbone network.

6. The method for tracking foreign objects in railway overhead contact lines based on a global-local attention mechanism according to claim 5, characterized in that, Step S2 specifically includes: The initial foreign object target's location information is input into the local focusing module. Based on the initial foreign object target's location information, the region where the initial foreign object target is located is cropped from the initial frame as a reference template image. This completes the template initialization of the deep neural network tracker based on the attention mechanism. The features of the reference template image are stored in the template cache of the deep neural network tracker. In the current frame, a search region of a fixed size is defined with the target tracking position output from the previous frame as the center, and the search region of the current frame is cropped. The baseline template image and the search region of the current frame are respectively divided into a template patch sequence and a search region patch sequence of fixed size. Learnable position codes are added to each patch sequence to preserve the spatial position information of the target and the background. The template patch sequence and the search region patch sequence are then concatenated in the sequence dimension to obtain the concatenated patch sequence. The concatenated Patch sequence is input into the Transformer encoder, and the search region enhanced feature map, which is fused with template prior information, is output through the self-attention mechanism. Then, using the enhanced feature map of the search region as input, the prediction head of the deep neural network tracker completes the decoding, outputting a target boundary regression map and a target classification score map. The position prediction result of the output target is obtained by mapping from the target boundary regression map, and the tracking confidence is calculated based on the peak value of the target classification score map.

7. The railway overhead contact line foreign object tracking method based on global-local attention mechanism according to claim 6, characterized in that, Step S3 satisfies the following formula: in, For the final tracking position, The position prediction results of the deep neural network tracker The motion state prediction results of the Kalman filter, To integrate weights, Track the confidence level for the current frame. , These are the preset mapping parameters.

8. The method for tracking foreign objects in railway overhead contact lines based on a global-local attention mechanism according to claim 7, characterized in that, In calculating the time-weighted average cosine similarity between the semantic features of the candidate target and all historical features in the dynamic historical high-confidence feature library, the following equation is satisfied: in, The semantic features representing candidate targets. Indicates the first in the dynamic historical high-confidence feature library A historical characteristic, This represents the total number of historical features in the dynamic historical high-confidence feature library. express and Time-weighted average cosine similarity For the first One historical characteristic; In the process of weightedly fusing the template matching score and the embedded re-identification score to obtain the comprehensive re-identification score, the following equation must be satisfied: in, This represents the overall re-identification score. Scoring based on template matching. To embed the re-identification score, These are the preset weighting coefficients.

Images