A railway overhead line system anomaly detection method, device, equipment and medium

Abstract

The application discloses a railway catenary anomaly detection method, device, equipment and medium, and relates to the field of intelligent detection of railway catenary. The method comprises the following steps: performing image segmentation processing on an original inspection image of a target railway catenary support component to obtain a target image; inputting the target image into an anomaly detection network to obtain an anomaly detection result (an image-level anomaly classification score and a pixel-level anomaly positioning heat map); the anomaly detection network is constructed based on a RAF-CLIP model; the RAF-CLIP model performs text encoding on a prompt word library comprising multiple groups of double-branch text semantic prompts to output text prototype features, performs multi-level feature extraction, multi-scale space aggregation and lightweight anomaly focusing on the original inspection image to obtain a visual feature set, and performs cross-modal semantic alignment on the text prototype features and the visual feature set to output the anomaly detection result. The application can accurately and efficiently realize railway catenary anomaly detection based on a small amount of normal samples.

Description

Technical Field

[0001] This application relates to the field of intelligent detection of railway catenary, and in particular to a method, device, equipment and medium for detecting anomalies in railway catenary. Background Technology

[0002] In electrified railway pantograph-catenary systems, the health of the overhead contact line support components directly determines the quality of power transmission and the safety of train operation. Because trains operate at high speeds for extended periods, and the external environment is complex and variable (e.g., strong winds, vibration, corrosion, electrical loads), the support components are highly susceptible to structural damage or functional failure, such as insulator breakage, dropper cable breakage, loose bolts, or missing cotter pins. Failure to detect and eliminate these defects in a timely manner can potentially lead to major train accidents, causing severe economic losses and social impact. Therefore, research on the abnormal detection of overhead contact line support components has significant engineering implications.

[0003] Traditional overhead contact line inspection relies primarily on manual patrols, which suffers from significant drawbacks such as low efficiency, high subjectivity, and difficulty adapting to massive data growth and complex, harsh operating environments. In recent years, with the development of drones, intelligent inspection vehicles, and artificial intelligence algorithms, overhead contact line operation and maintenance has gradually moved towards intelligentization. Mainstream overhead contact line inspection algorithms fall into two main categories: machine learning and deep learning. The former follows a feature extraction and template matching process, such as corner detection frameworks based on Harris-SSDA or machine learning methods incorporating domain knowledge. These methods are computationally efficient, but their generalization performance is limited by the manual setting of hyperparameters. The latter achieves component localization and anomaly identification through deep feature modeling, such as the CSRM-MIM method based on masked image modeling and the asymmetric decoupling head detector under the YOLO framework. These studies have achieved good results under conditions of abundant data sources, but in actual lines, abnormal samples are extremely scarce, and faults are very rare events compared to normal samples. Traditional supervised learning struggles to collect complete datasets covering all fault types, resulting in severely insufficient generalization ability of the model for unknown anomalies.

[0004] Furthermore, the background of overhead contact line images is complex, containing steel frame structures, vegetation, sky, and varying lighting conditions. Detection schemes based on autoencoders or generative adversarial networks often struggle to accurately reconstruct subtle features in high-resolution, highly complex images. Anomaly detection for small components such as pins and washers is prone to numerous false positives or false negatives. At the same time, existing models typically require separate training and parameter tuning for each type of component. Faced with hundreds or thousands of different specifications of components in the overhead contact line, this "one model per type" approach lacks scalability in actual inspection operations and is ill-suited to the rapidly iterating detection demands.

[0005] Therefore, how to accurately and efficiently detect anomalies in railway catenary based on a small number of normal samples (Few-shot) has become an urgent problem to be solved. Summary of the Invention

[0006] The purpose of this application is to provide a method, apparatus, equipment and medium for detecting anomalies in railway catenary, which can accurately and efficiently detect anomalies in railway catenary based on a small number of normal samples.

[0007] To achieve the above objectives, this application provides the following solution: Firstly, this application provides a method for detecting anomalies in railway overhead contact lines, including: Obtain the original inspection images of the target railway catenary support components; The original inspection image is processed using an image segmentation algorithm to obtain the target image; The target image is input into the anomaly detection network to obtain the image-level anomaly classification score and pixel-level anomaly localization heatmap of the target railway catenary support component. The anomaly detection network is built based on the RAF-CLIP model; the RAF-CLIP model includes: a semantic prompting module, a CLIP text encoder, a CLIP image encoder, a multi-scale spatial aggregation module, a lightweight anomaly focusing adaptation module, and a cross-modal semantic alignment module; The semantic prompting module is used to construct a prompting lexicon, which includes: multiple sets of bi-branch text semantic prompts; the CLIP text encoder is used to encode the prompting lexicon and output text prototype features; the CLIP image encoder is used to extract multi-level features from the original inspection image to obtain multi-level feature maps with different depths and receptive fields; the multi-scale spatial aggregation module is used to fuse neighborhood context information on each level feature map using sliding windows of different sizes to obtain multi-scale fused features; the lightweight anomaly focusing adaptation module is used to enhance the multi-scale fused features based on a learnable self-attention mechanism to obtain a visual feature set; the cross-modal semantic alignment module is used to output image-level anomaly classification scores and pixel-level anomaly localization heatmaps based on the text prototype features and the visual feature set.

[0008] Secondly, this application provides a railway catenary anomaly detection device, comprising: The image acquisition module is used to acquire the original inspection images of the target railway catenary support components; The image processing module is used to process the original inspection image using an image segmentation algorithm to obtain the target image; An anomaly detection module is used to input the target image into an anomaly detection network to obtain an image-level anomaly classification score and a pixel-level anomaly location heatmap of the target railway catenary support component. The anomaly detection network is built based on the RAF-CLIP model; the RAF-CLIP model includes: a semantic prompting module, a CLIP text encoder, a CLIP image encoder, a multi-scale spatial aggregation module, a lightweight anomaly focusing adaptation module, and a cross-modal semantic alignment module; The semantic prompting module is used to construct a prompting lexicon, which includes: multiple sets of bi-branch text semantic prompts; the CLIP text encoder is used to encode the prompting lexicon and output text prototype features; the CLIP image encoder is used to extract multi-level features from the original inspection image to obtain multi-level feature maps with different depths and receptive fields; the multi-scale spatial aggregation module is used to fuse neighborhood context information on each level feature map using sliding windows of different sizes to obtain multi-scale fused features; the lightweight anomaly focusing adaptation module is used to enhance the multi-scale fused features based on a learnable self-attention mechanism to obtain a visual feature set; the cross-modal semantic alignment module is used to output image-level anomaly classification scores and pixel-level anomaly localization heatmaps based on the text prototype features and the visual feature set.

[0009] Thirdly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor executes the computer program to implement the railway catenary anomaly detection method described in any one of the above.

[0010] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the railway catenary anomaly detection method described in any one of the above descriptions.

[0011] According to the specific embodiments provided in this application, this application has the following technical effects: This application provides a method, device, equipment, and medium for detecting anomalies in railway overhead contact lines. An anomaly detection network is constructed based on the RAF-CLIP model. The semantic prompting module in the RAF-CLIP model can automatically learn the semantic decision boundaries between normal overhead contact line features and various anomaly defect features, completely eliminating reliance on manual annotation experience. The CLIP text encoder only needs fine-tuning a small number of learnable prompt words to achieve efficient domain transfer, making it suitable for small-sample overhead contact line scenarios. Subsequently, from three progressive dimensions—multi-scale spatial aggregation, lightweight anomaly focusing, and cross-modal feature projection—the model's ability to capture and represent various subtle defects in the overhead contact line is comprehensively improved. Based on learnable general state prompts and visual enhancement of anomaly focusing, the RAF-CLIP model in this application has stronger generalization and accuracy, achieving superior detection accuracy and positioning performance under limited sample conditions, and adapting to rapidly iterative detection needs. It effectively addresses the challenges of detecting complex backgrounds and subtle defects in overhead contact lines, providing reliable technical support for intelligent railway operation and maintenance. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 A flowchart illustrating a method for detecting anomalies in a railway overhead contact system, as provided in this application embodiment; Figure 2 A detailed implementation process diagram of a railway catenary anomaly detection method provided in this application embodiment; Figure 3 This is a real-shot image of the original collected sample provided in the embodiments of this application; Figure 4 This is a schematic diagram illustrating the visualization effect of various component samples provided in the embodiments of this application; Figure 5 The diagram shows the anomaly detection and localization effect of the RAF-CLIP model provided in this application embodiment on various key components such as insulators, brackets, and fasteners. Figure 6 Anomaly localization effect diagram of the baseline model and the complete RAF-CLIP model provided in the embodiments of this application on the same contact wire component image; Figure 7 This is a schematic diagram of the functional modules of a railway catenary anomaly detection device provided in an embodiment of this application; Figure 8 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0014] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0015] In recent years, large-scale pre-trained basic vision models (such as DINOv2 and CLIP) have demonstrated excellent general feature representation capabilities, enabling them to capture deep distribution patterns in natural images. However, how to effectively adapt these general models to the highly specialized, complex, and rare-sample-scarce field of railway catenary inspection, and achieve accurate identification of various defects with only a very small number of normal samples (Few-shot), remains a key technical problem that urgently needs to be solved in current intelligent railway operation and maintenance.

[0016] The purpose of this application is to provide a method, device, equipment and medium for detecting anomalies in railway catenary, which can accurately and efficiently detect anomalies in railway catenary based on a small number of normal samples.

[0017] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0018] In one exemplary embodiment, such as Figure 1 As shown, a method for detecting anomalies in railway overhead contact lines is provided. This method utilizes semantic prompting enhancement to detect anomalies in high-speed railway overhead contact lines with few samples. The method includes: Step 101: Obtain the original inspection image of the target railway catenary support assembly.

[0019] Step 102: The original inspection image is processed using an image segmentation algorithm to obtain the target image.

[0020] Step 103: Input the target image into the anomaly detection network to obtain the image-level anomaly classification score and pixel-level anomaly localization heatmap of the target railway catenary support component; the anomaly detection network is constructed based on the RAF-CLIP model; the RAF-CLIP model includes: a semantic prompting module, a CLIP text encoder, a CLIP image encoder, a multi-scale spatial aggregation module, a lightweight anomaly focusing adaptation module, and a cross-modal semantic alignment module.

[0021] The RAF-CLIP model is constructed based on a cross-modal visual-language pre-training framework with semantic prompt enhancement. The semantic prompt module constructs a prompt lexicon, which includes multiple sets of bi-branch text semantic prompts. The CLIP text encoder encodes the prompt lexicon to output text prototype features. The CLIP image encoder extracts multi-level features from the original inspection image, obtaining multi-level feature maps with different depths and receptive fields. The multi-scale spatial aggregation module fuses neighborhood context information on each level of feature map using sliding windows of different sizes, obtaining multi-scale fused features. The lightweight anomaly focusing adaptation module enhances the multi-scale fused features based on a learnable self-attention mechanism to obtain a visual feature set. The cross-modal semantic alignment module outputs image-level anomaly classification scores and pixel-level anomaly localization heatmaps based on the text prototype features and the visual feature set.

[0022] In another exemplary embodiment of this application, step 102 specifically includes: segmenting the original inspection image using the SAM (SegmentAnything Model) algorithm to obtain segmented regions; and filtering the segmented regions to obtain a target image.

[0023] In another exemplary embodiment of this application, constructing a prompt lexicon specifically includes: constructing a learnable prefix word embedding sequence of a set length; the learnable prefix word embedding sequence is used to capture the contextual features of the contact wire surface texture, structural morphology, and defects; concatenating the learnable prefix word embedding sequence with fault-free fixed state suffixes and faulty fixed state suffixes respectively to obtain a set of bi-branch text semantic prompts; and constructing a prompt lexicon based on multiple sets of bi-branch text semantic prompts and orthogonality constraints.

[0024] In another exemplary embodiment of this application, text encoding is performed on the prompt word library to output text prototype features, specifically including: text encoding is performed on each group of bi-branch text semantic prompts in the prompt word library to obtain text features of each group of bi-branch text semantic prompts; mean aggregation is performed on multiple groups of text features to obtain text prototype features; the text prototype features include: normal text prototype features and abnormal text prototype features.

[0025] In another exemplary embodiment of this application, multi-level feature extraction is performed on the original inspection image to obtain multi-level feature maps with different depths and receptive fields. Specifically, this includes: dividing the original inspection image into a sequence of image blocks using a patch embedding operation, encoding the image block sequence, extracting multi-level feature representations, and obtaining multi-level feature maps with different depths and receptive fields; the multi-level feature maps include: a global class token and a local patch token.

[0026] By employing sliding windows of different sizes to fuse neighborhood context information on each level of feature map, multi-scale fusion features are obtained. Specifically, this includes: reshaping each layer of map tile tokens into a two-dimensional spatial feature map; using three different sizes of sliding windows, calculating the neighborhood weights of each position in the two-dimensional spatial feature map using a Gaussian kernel function, and performing weighted summation to obtain multi-scale spatial aggregation features; and concatenating the multi-scale spatial aggregation features with the global category token to form multi-scale fusion features.

[0027] The lightweight anomaly focusing adaptation module includes an adapter and a CLIP projection layer. The adapter is used to guide global category tokens to aggregate global anomaly information and guide local tile tokens to enhance defect perception capabilities based on multi-scale fusion features, thereby obtaining enhanced features. The CLIP projection layer is used to map the enhanced features to a joint embedding space to obtain a set of visual features.

[0028] In another exemplary embodiment of this application, outputting an image-level anomaly classification score and a pixel-level anomaly localization heatmap based on the text prototype features and the visual feature set specifically includes: using the text prototype features as semantic anchors, calculating and accumulating cosine similarities with the visual features in the visual feature set to obtain the probability that each tile token belongs to an anomaly category. Reshaping the patch probability sequence formed by the probabilities of each tile token belonging to an anomaly category into a two-dimensional probability map, and upsampling and Gaussian smoothing filtering the two-dimensional probability map to obtain a pixel-level anomaly localization heatmap. Fusing the global category token probability with the maximum probability value in the pixel-level anomaly localization heatmap to obtain an image-level anomaly classification score.

[0029] In another exemplary embodiment of this application, the multi-task joint loss optimization function used when training the RAF-CLIP model is constructed based on image-level classification loss, pixel-level segmentation loss, and patch alignment loss; the image-level classification loss is determined based on the probability corresponding to the image-level ground truth label and the global category token; the pixel-level segmentation loss is determined based on the probability that each visual feature belongs to the abnormal category and the pixel-level ground truth mask; the patch alignment loss is determined based on the normal patch token and the abnormal patch token at each level and scale.

[0030] In practical applications, the overall implementation process of this embodiment is as follows: Step A: Use an autonomous inspection vehicle to acquire images of the overhead contact line support and suspension device of the high-speed railway, obtaining original inspection images. Step B: Establish a sample dataset of inspected overhead contact line support components. After precise segmentation by the SAM algorithm and manual screening, a standard experimental dataset containing multiple key components is finally formed, providing high-quality data support for subsequent model training and testing. Step C: Construct a learnable normal / abnormal semantic prompt module, generate dual-branch learnable text prompts and expand them into a prompt vocabulary, and extract text prototype features through a frozen CLIP text encoder. Step D: Input the images of overhead contact line components sequentially into the RAF-CLIP image encoder. After multi-level feature extraction and multi-scale spatial aggregation module weighted fusion of neighborhood context, and then through a lightweight anomaly focusing adapter and frozen projection layer, obtain joint embedding spatial features and fine-grained image patch features. Step E: Input the text prototype features and joint embedding spatial features into the cross-modal semantic alignment module, calculate cosine similarity and anomaly probability, and combine the joint constraints of classification loss, segmentation loss and patch alignment loss to generate a pixel-level anomaly detection map. Step F: Use a small number of image samples to train and test the model, and save the optimal model for actual inspection.

[0031] Step C specifically includes: Step C1: Design length is The learnable prefix word embedding sequence is concatenated with the fault-free fixed state suffix "without defect" and the faulty fixed state suffix "with defect" to form a two-branch semantic cue. The two share the learnable prefix parameters to ensure semantic consistency and enhance the binary distinction between normal and abnormal.

[0032] Step C2: To further enhance semantic diversity, construct a system containing... The system employs a set of independently learnable prefix suggestion vocabularies, each set of prefixes being independently initialized and optimized. This allows the system to learn complementary and diverse normal / abnormal semantic patterns from auxiliary data. Simultaneously, orthogonality constraints are imposed on all text embedding features within the vocabulary, encouraging feature vectors of different suggestions to be orthogonal to each other in the semantic space, thereby effectively avoiding semantic redundancy and feature coupling.

[0033] Step C3: Input the expanded cue word library into the frozen CLIP text encoder and output the text features of each cue.

[0034] Step C4: To eliminate potential random fluctuations in single-set features and improve the stability and robustness of the final semantic prototype, [the following steps are taken]. The text features are aggregated using mean calculation to obtain the final normal and abnormal text prototype features used for cross-modal alignment. This aggregated prototype can accurately characterize fine-grained semantic features such as minor cracks, localized corrosion, structural deformation, surface wear, and loose fasteners in the contact network, providing stable and reliable semantic anchors for subsequent cross-modal matching.

[0035] Step D specifically includes: Step D1: The input image is divided into a regular, non-overlapping sequence of image patches through Patch Embedding and then fed into the CLIP image encoder. This encoder contains four sequentially stacked feature extraction blocks, outputting multi-level feature representations with different depths and receptive fields, including global category tokens and local detail information patch tokens, which respectively carry the global semantics and local texture detail information of the image.

[0036] Step D2: Reshape the tile tokens at each level into two-dimensional spatial feature maps, and process the feature maps using three different sliding windows (sizes 1, 3, and 5). For each center location, use a Gaussian kernel function to calculate the weights of each location in the neighborhood, and perform a weighted summation of all features in the neighborhood to achieve multi-scale spatial aggregation. After aggregation, the features are unfolded back into a sequence form and concatenated with the global category tokens of the same level to form multi-scale fused features.

[0037] Step D3: Input the multi-scale fused features into the lightweight anomaly focusing adapter. This adapter contains a learnable projection matrix that adjusts the feature distribution through a self-attention mechanism. After fine-tuning a small number of parameters of the adapter end-to-end on an auxiliary anomaly dataset, this module can guide the global category token to actively focus on all distant patch regions in the image that are related to anomalies, while guiding each local patch token to focus its attention on subtle anomaly-related patterns in itself and its surroundings, suppressing responses to normal backgrounds, thereby significantly enhancing the ability to perceive subtle defects.

[0038] Step D4: Map the enhanced features to the joint embedding space shared with the text through the frozen CLIP projection layer to obtain a multi-scale, exceptionally sensitive, and detail-rich set of visual features.

[0039] Step E specifically includes: Step E1: Using normal and abnormal text prototype features as stable semantic anchors, perform fine-grained cosine similarity calculations with visual features at each level and scale. Then, accumulate and fuse the similarities across all levels and scales to obtain a global matching score for each tile token. Based on these scores, calculate the probability that each tile token belongs to an abnormal category. The probability of the global category token is used for image-level anomaly classification, while the probability of each tile token is used for pixel-level anomaly segmentation.

[0040] Step E2: A multi-task joint loss optimization model is employed, including image-level classification loss, pixel-level segmentation loss, and patch alignment loss. Focal Loss is used for classification to alleviate the imbalance between normal and abnormal samples; the segmentation loss combines Focal Loss and L1 loss to balance the sparsity and boundary accuracy of abnormal regions; the patch alignment loss forces the model to learn a more compact and separable feature space by constraining the similarity between normal and abnormal patches to be lower than the similarity between patches of the same type. The total loss is a weighted sum of the three losses.

[0041] Step E3: The patch probability sequence is reshaped into a two-dimensional probability map, which is then bilinearly upsampled to restore the original image size. Gaussian smoothing filtering is then applied to suppress isolated noise points, ultimately generating a smooth, continuous, and spatially coherent pixel-level anomaly detection map. Simultaneously, the image-level anomaly score is obtained by fusing the probability of the global category token with the maximum probability value in the anomaly map. This allows the model to simultaneously output image-level classification results and pixel-level localization heatmaps, fully meeting the practical inspection and detection requirements of high-speed rail catenary systems, which demand low sample size, high precision, and fine-grained RAF-CLIP.

[0042] In step F, the ratio of normal to abnormal images for each type of contact wire component sample is 1:1, which is used for training, and the trained model is saved for testing.

[0043] This embodiment fills the gap in anomaly detection of high-speed railway catenary support devices under limited sample conditions. Compared with traditional manual prompting methods, the RAF-CLIP model, based on learnable general state prompts and anomaly focusing visual enhancement, exhibits stronger generalization and accuracy. This architecture consists of a learnable normal / abnormal semantic prompting module, a multi-scale spatial aggregation module, a lightweight anomaly focusing adapter, and a patch alignment loss strategy. Experiments show that this method can achieve superior detection accuracy and positioning performance under limited sample conditions, effectively addressing the challenges of detecting complex backgrounds and subtle defects in the catenary, and providing reliable technical support for intelligent railway operation and maintenance.

[0044] The following is combined with Figure 2 A more detailed description of the implementation process of the railway catenary anomaly detection method in this embodiment is provided, and the effectiveness of this embodiment is verified by experimental methods.

[0045] This embodiment focuses on few-shot anomaly detection for high-speed railway catenary components based on semantic cue enhancement. The anomaly detection network used is the RAF-CLIP model, constructed using a cross-modal vision-language pre-training framework based on semantic cue enhancement. Specifically, a multi-scale spatial aggregation module is designed to enhance the feature extraction capability for catenary defects of different sizes by weighted fusion of neighborhood context information through multi-scale sliding windows, effectively alleviating the problems of missed detection of small-sized anomalies and inaccurate localization of large-sized anomalies. A lightweight anomaly focusing adaptation module is introduced to dynamically adjust the attention distribution of category tokens and tile tokens, enabling the model to accurately focus on local defect areas of the catenary and strengthen the expression of anomaly-sensitive features. In addition, a learnable normal / abnormal semantic cue module is constructed to generate universal state text representations and align them with visual features. Combined with patch alignment loss, the distinguishability between normal and abnormal features is enhanced, ultimately achieving efficient and accurate few-shot anomaly detection and localization at the high-speed railway catenary component level.

[0046] The implementation process of this embodiment is as follows: An autonomous overhead contact line inspection vehicle collects images of the overhead contact line and constructs a dataset; a learnable normal / abnormal semantic prompting module is used to construct two-branch text prompt words with learnable prefixes, describing the states of no fault and presence of fault respectively, which are then input into the CLIP text encoder of the RAF-CLIP model to obtain text prototype features. and The images of the overhead contact system components are sequentially subjected to block embedding, multi-layer block feature extraction from the CLIP image encoder in the RAF-CLIP model, multi-scale spatial aggregation module multi-receptive field weighted summation, and learnable adapter and frozen projection layer cross-modal mapping to obtain joint embedded spatial features. Fusion features with fine-grained image patches Text prototype features , and joint embedding spatial features Input the cross-modal semantic alignment module to obtain refined text embedding features. , ; for the first Layered image patch-level features With refined text embedding features , ( Perform alignment operations, combined with image-level classification loss. Compared with pixel-level segmentation loss The joint constraint optimization model parameters are used to finally generate pixel-level anomaly detection maps for overhead contact line components. (i.e., anomaly location heatmap).

[0047] The railway catenary-specific dataset required for model training and performance testing in this embodiment was constructed by collecting data on-site using an autonomous catenary inspection vehicle. During actual operation, the inspection vehicle strictly followed a pre-set standardized inspection path, moving smoothly on the track while maintaining a constant operating height and fixed imaging parameters such as focal length, exposure, and frame rate. It conducted comprehensive image acquisition of various catenary support structures along the line, using multi-view and multi-directional shooting methods to obtain massive amounts of raw real-scene images of individual components. The actual shooting effect of the raw collected samples is as follows: Figure 3 As shown.

[0048] After completing the batch acquisition of on-site images, the SAM precise image segmentation algorithm was introduced to carry out systematic data preprocessing. Relying on the algorithm's powerful ability to accurately segment target regions, redundant background interference information was automatically removed from large-scale global inspection images, efficiently and accurately selecting and locating local target images of clearly defined and complete contact network core components. After sample screening, category sorting, and abnormal sample enrichment, six typical contact network key component samples with a higher proportion of fault and abnormal samples were finally selected to form a standard experimental dataset. The visualization effects of various component samples are shown below. Figure 4 As shown.

[0049] This embodiment presents the RAF-CLIP model for pixel-level anomaly detection of high-speed railway catenary components, which comprehensively realizes three core aspects: text modal feature extraction, image modal feature extraction, and anomaly map and anomaly score calculation.

[0050] 1. Text modality feature extraction.

[0051] In the task of detecting anomalies in high-speed railway catenary components with few samples, traditional text-based prompting methods based on manual design typically rely on human experience to construct fixed combinations of "component name + defect type," such as "insulator crack," "cantilever corrosion," and "contact wire wear." These prompts have three fundamental limitations: First, they heavily depend on the prior knowledge of professional annotators regarding various catenary defects, making it difficult to cover the diverse and varied subtle defects encountered in actual inspections, such as cracks, corrosion, deformation, wear, flashover marks, and loose fasteners. Second, the semantic expression of manual prompts is static and discrete, lacking adaptability to changes in surface texture, structural morphology, and lighting conditions of different components, resulting in extremely poor generalization performance. Third, given the inherent scarcity of anomaly samples, highly uneven category distribution, and large defect scale spans in catenary inspection data, manual prompts are prone to semantic bias, leading to insufficient robustness of the detection model and an inability to adapt to real and complex high-speed railway catenary inspection scenarios.

[0052] To address the aforementioned issues, the RAF-CLIP model abandons the traditional categorical and discretized prompt design approach, instead focusing on the essential and universal semantic differences between normal and abnormal states, constructing a fully learnable universal state prompt mechanism. This mechanism, through a data-driven approach, automatically learns the semantic decision boundaries between normal surface texture, structural morphology, and various abnormal defect features of the overhead contact system on an auxiliary abnormal dataset, completely eliminating reliance on manual annotation experience. The resulting textual semantic representation possesses strong generalization ability, high robustness, and adaptability across components and defect types, providing stable, reliable, and transferable semantic anchors for subsequent cross-modal feature alignment.

[0053] In its implementation, the RAF-CLIP model first constructs a dedicated text prompting system for overhead contact line anomaly detection tasks. Each prompt is broken down into two parts: a learnable prefix and a fixed-state suffix.

[0054] (1) Learnable prefix: Let it be a prefix of length 1. Word embedding sequence ,in ( (For word embedding dimension). These prefix parameters are learned end-to-end on the auxiliary dataset and are responsible for capturing the contextual features of general texture, structural morphology, and subtle defects on the contact wire surface without any manual design.

[0055] (2) Fixed state suffixes: The fixed state is described in semantically clear and unambiguous natural language, namely the fixed state suffix "without defect" for no fault and the fixed state suffix "with defect" for faulty. These two suffixes accurately anchor the binary semantic boundary between no fault and faulty, ensuring that the model always learns the essential distinction between normal and abnormal.

[0056] The resulting bi-branch text semantic prompts are as follows.

[0057] (1) In the formula, , These are designated as normal and abnormal prompts, respectively, sharing learnable prefix parameters to ensure semantic consistency while enhancing the binary distinction between normal and abnormal. To further enhance semantic diversity and improve the model's adaptability to complex defect morphologies, a prompt lexicon containing independent learnable prefixes of type B is constructed. The details are as follows.

[0058] Each set of prefixes is initialized and optimized independently, enabling it to learn complementary and diverse normal / abnormal semantic patterns from auxiliary data. Furthermore, to prevent semantic redundancy and feature coupling between multiple sets of prompts, this embodiment applies orthogonality constraints to all text embedding features within the lexicon, encouraging feature vectors of different prompts to be orthogonal to each other in the semantic space. This effectively enhances the distinguishability between normal and abnormal features and avoids semantic confusion.

[0059] The completed cue word library is then fed into the frozen CLIP text encoder built into RAF-CLIP. This encoder relies on the general semantic priors learned by CLIP on massive text-image pairs, and only requires fine-tuning of a small number of learnable prefix parameters to achieve efficient domain transfer, making it highly suitable for contact network scenarios with limited sample sizes. The text encoder outputs the text features of each set of prompts. .

[0060] To eliminate potential random fluctuations in single-set features and improve the consistency, stability, and robustness of the final semantic prototype, this embodiment... The text features are aggregated by mean to obtain the text prototype features, as follows.

[0061] In the formula, These are the normal and abnormal text prototype features ultimately used for cross-modal alignment. The resulting text prototype features can accurately characterize fine-grained semantic features such as minor cracks, local corrosion, structural deformation, surface wear, and loose fasteners in the contact network. This provides high-quality, highly generalizable, and extremely stable semantic anchors for subsequent cross-modal matching, fundamentally solving the core pain points of traditional manual prompts, such as weak generalization ability, large semantic bias under small sample conditions, and poor detection robustness.

[0062] 2. Image modal feature extraction.

[0063] The images of the high-speed rail overhead contact system are collected by autonomous inspection vehicles in complex outdoor environments. The image data presents the following significant challenges: strong background interference (such as sky, forests, and other cables), drastic changes in lighting (backlighting, shadows, strong light reflection), dense arrangement of towers and cable structures, severe mutual obstruction between components, varied shooting angles and inconsistent distances, and the ease with which noise is introduced during the imaging process. Meanwhile, the types of defects in the overhead contact system components are diverse, encompassing surface micro-cracks, localized rust patches, structural deformation, loose fasteners, contact wire surface wear, flashover marks, etc., and the scale of these defects ranges greatly, from millimeter-level micro-cracks to centimeter-level large-area rust areas. Furthermore, most defective areas occupy a very small portion of the overall image, have low contrast, and blurred edges, making them easily confused with normal textures.

[0064] The native CLIP visual encoder (ViT architecture) is pre-trained on massive amounts of natural images through image-text comparison learning. Its design goal is primarily global object recognition and classification, rather than refined perception of localized minor defects. Therefore, its visual feature extraction focuses more on overall contours, global semantic information, and the main target structure, while its ability to represent localized, small, low-contrast, and structurally blurred defect regions is severely inadequate. Attention weights are excessively concentrated on background textures or the global structure of components, easily "submerging" defect features, frequently losing small-scale defects, and blurring detailed information. Directly using these features for refined anomaly detection and localization in overhead contact lines will inevitably lead to a large number of missed and false detections.

[0065] To address this fundamental contradiction, the RAF-CLIP model constructs a complete anomaly-sensitive visual enhancement system from three progressive dimensions: multi-scale spatial aggregation, lightweight anomaly focusing adapter, and cross-modal feature projection. This system gradually transforms the original coarse-grained global visual features into fine-grained visual representations that focus on local defects, are rich in detailed information, and are adapted to multi-scale anomalies, thereby comprehensively improving the model's ability to capture and represent various subtle defects in the overhead contact system.

[0066] (1) Multi-level feature extraction.

[0067] First, input the overhead contact line inspection image. The image is divided into a regular, non-overlapping sequence of image blocks through a block embedding operation, and then fed into the CLIP image encoder. This encoder contains four sequentially stacked feature extraction blocks, capable of outputting multi-level feature representations with different depths and receptive fields. Shallow features retain rich details such as texture, edges, and grayscale variations, while deeper features contain higher-level semantic abstractions and global structural information. Formally, the first... layer( The output is as follows.

[0068] In the formula, It serves as a global category token, used to encode the global semantics of the entire image; ( Each image patch is represented by a local tile token, carrying information about local texture, edges, grayscale variations, and potential defects. The combined use of multiple feature layers provides a feature foundation for subsequent enhancement processes that includes both global structural awareness and local details.

[0069] (2) Multi-scale spatial aggregation mechanism.

[0070] To address the issues of large scale differences in catenary defects and the lack of local neighborhood generalization bias in the ViT structure, RAF-CLIP introduces a multi-scale spatial aggregation module. This module, located before the adapter, is specifically designed to fuse local neighborhood context information at each level. Specifically, it first fuses the first... Layered tile tokens are reshaped into two-dimensional spatial feature maps. ,in Then, three different sizes of sliding windows were used. The feature maps are processed separately. For the center location... Each location within its neighborhood weight The Gaussian kernel function is used for calculation, and the formula is as follows.

[0071] In the formula, The standard deviation of the Gaussian kernel controls the extent of neighborhood influence. This Gaussian weighting strategy adaptively strengthens the feature responses in the central region while weakening irrelevant background information at greater distances. (Aggregated features) (Multi-scale spatial aggregation features) is the weighted sum of all features within the neighborhood, as shown in the following formula.

[0072] This operation ensures that each location feature after aggregation retains its original central details while incorporating contextual relationships across different scales, effectively enhancing the feature discrimination between small defective regions and surrounding normal regions. After aggregation, the features are re-expanded into a sequence form and compared with global category tokens at the same level. splicing together to form multi-scale fusion features The formula is as follows.

[0073] This feature set comprehensively covers the contextual information requirements of defects of different sizes, providing a multi-scale, context-rich feature foundation for subsequent anomaly focusing.

[0074] (3) Lightweight anomaly focusing adapter.

[0075] Despite the rich neighborhood context inherent in the features obtained through multi-scale spatial aggregation, the self-attention distribution in the native CLIP visual encoder still tends to focus on global structure rather than local anomalies. RAF-CLIP introduces a lightweight, trainable self-attention-based adapter specifically designed to correct this attention bias. This adapter contains three sets of learnable projection matrices. And an output projection The calculation process is as follows.

[0076] in, , and They represent the first The query matrix, key matrix, and value matrix obtained by mapping the multi-scale fusion features of the layers through a learnable projection matrix are used to calculate the self-attention weights and guide the model to focus on abnormal regions. This indicates the enhanced features of the adapter output.

[0077] By fine-tuning a small number of adapter parameters end-to-end on an auxiliary anomaly dataset, this module achieves two key objectives. The first is to guide global category tokenization. The first method actively focuses on all distant patch regions in the image that are related to anomalies, thereby effectively aggregating global anomaly information for image-level classification. Secondly, it guides each local tile token to focus its attention on subtle anomaly-related patterns, such as crack edges or rust textures, in itself and its surroundings, while suppressing responses to the normal background, thus significantly enhancing the perception of subtle defects.

[0078] This dual-direction enables enhanced characteristics of the adapter output. It possesses both global anomaly detection and fine-grained local defect localization capabilities. Finally, through the frozen CLIP projection layer... The enhanced features are mapped to a joint embedding space shared with the text, as shown in the following formula.

[0079] In the formula, For the dimension of the final unified space, This represents the total number of local patches obtained after patch embedding of a single input image. All layers. With scale The outputs together constitute a multi-scale, exceptionally sensitive, and detail-rich set of visual features. This provides high-quality visual representations for subsequent cross-modal alignment and anomaly inference, and completely solves problems such as ignoring local defects in native features, susceptibility to background interference, and loss of small targets.

[0080] 3. Anomaly graph and anomaly score calculation.

[0081] In obtaining the enhanced visual feature set and stable text prototype features generated through learnable text prompts. and Subsequently, RAF-CLIP constructed a complete cross-modal matching, multi-task joint optimization, and anomaly inference process. This process can simultaneously output image-level anomaly classification scores and pixel-level anomaly localization heatmaps, fully meeting the integrated inspection needs of high-speed rail catenary components from global defect identification to local fine-grained localization, and is especially suitable for the detection of minute defects in key components such as insulators, cantilever arms, contact wires, and fasteners.

[0082] (1) Cross-modal similarity matching.

[0083] RAF-CLIP uses textual prototype features and As a stable and transferable semantic anchor, it is associated with visual features at each level and scale. (in Fine-grained cosine similarity calculation is performed on the index of the category token or tile token to establish an accurate visual-semantic correspondence, as shown in the following formula.

[0084] In the formula, For cosine similarity, This is a temperature coefficient used to control the sharpness of the similarity distribution and avoid extreme values ​​from having an excessive impact on the matching results. , They represent the first Layer The first scale The cosine similarity between each tile token and the prototype features of abnormal and normal text is used to measure the degree of matching between the corresponding local regions of the image and the abnormal and normal semantics.

[0085] Then, all levels With all scales The similarity scores are accumulated and fused to obtain the global anomaly matching score for each tile token. Match score with normal The formula is as follows.

[0086] Based on the scores above, the probability that each tile token belongs to the anomaly category can be calculated. The formula is as follows.

[0087] In the formula, The probability of an anomaly corresponding to the global category token, i.e., taking the index. When the position corresponding to the global category token is specified, this is the probability value that the visual token belongs to the anomaly category. This probability integrates cross-modal semantic alignment information across all levels and scales and is directly used for image-level anomaly classification. The closer the value is to 1, the higher the probability that there is a defect in the contact wire component image.

[0088] The closer the value is to 1, the higher the probability of defects in the catenary component image; while the probability of each block token... These are used for pixel-level anomaly segmentation, which precisely reflects the confidence level that each local region in the image belongs to an anomaly, providing a fine-grained basis for subsequent defect localization.

[0089] (2) Multi-task joint loss optimization.

[0090] To fully explore the essential differences between normal and anomalous features under small sample conditions and alleviate feature confusion caused by the scarcity of anomalous samples, RAF-CLIP constructs a multi-task joint optimization objective with three complementary losses, simultaneously learning image-level classification, pixel-level segmentation, and feature discriminativity end-to-end. The entire training process is completed on an auxiliary anomaly detection dataset, with only the adapter and learnable text prefix parameters fine-tuned, while the remaining CLIP main parameters are completely frozen.

[0091] Image-level classification loss Because the number of normal and abnormal samples in the auxiliary dataset is often highly imbalanced, directly using cross-entropy loss will cause the model to favor the majority class. Therefore, RAF-CLIP uses Focal Loss as the classification loss, as shown in the following formula.

[0092] In the formula, Image-level ground truth labels. Focal Loss is a focal loss function that reduces the weight of easily distinguishable samples by introducing a modulation factor, causing the model to pay more attention to those difficult-to-distinguish samples during training, thereby effectively alleviating the class imbalance problem.

[0093] Pixel-level segmentation loss To achieve accurate anomaly localization, patch-level probabilistic graphs need to be supervised. First, the patch probability sequence... Reconstructed into a two-dimensional probability graph and pixel-level real mask Scale to the same size Considering that anomalous pixels usually only account for a very small portion of the image, and the boundaries of anomalous regions need to be accurately characterized, the segmentation loss combines Focal Loss and L1 Loss, as shown in the following formula.

[0094] Focal Loss addresses pixel-level positive-negative imbalance. It represents the mean absolute error, calculates the absolute deviation of the predicted value from the true mask pixel by pixel, complements Focal Loss, and improves the accuracy of outlier region boundaries through direct regression supervision.

[0095] Patch alignment loss To further enhance the discriminative power between normal and abnormal tile tokens, RAF-CLIP innovatively proposes a patch alignment loss. Its core idea is that for any set of both normal and abnormal patches existing simultaneously in a batch, the similarity between normal-abnormal patch pairs should be lower than the similarity between normal-normal patch pairs and abnormal-abnormal patch pairs. The formal definition is as follows.

[0096] In the formula, These represent the number of normal patches and the number of abnormal patches in the current batch. For hierarchical sets, For scale sets, For the first Layer The first scale A normal patch feature For the first The r-th scale of the layer Anomalous patch features For the first Layer The first scale A normal patch feature For the first Layer The first scale The loss function uses a patch feature to identify anomalous features. By increasing the distance between features of different classes and decreasing the distance between features of the same class, the model is forced to learn a more compact and separable anomalous feature space, thereby significantly improving the detection robustness under small sample conditions.

[0097] Ultimately, the total loss function (the final multi-task joint loss optimization function) The sum of the three losses mentioned above is given by the following formula.

[0098] In the formula, and These are hyperparameters used to balance the importance of the three tasks. Through joint optimization, RAF-CLIP can simultaneously improve image-level classification accuracy, pixel-level segmentation quality, and feature discriminativity, achieving optimal overall performance.

[0099] (3) Anomaly map generation and image-level anomaly score calculation.

[0100] After model training is complete, the inference phase begins. For any test image, RAF-CLIP first calculates the anomaly probability of each tile token. And reshape it into a two-dimensional probability graph Since the resolution of the original patch is usually much smaller than that of the input image, bilinear upsampling is required to reduce the resolution. Restore to the original Figure 1 To further suppress isolated outliers caused by local noise or minor misjudgments, a Gaussian smoothing filter is applied to the recovered probability map, ultimately resulting in a smooth, continuous, and spatially coherent pixel-level outlier map. This anomaly diagram can accurately pinpoint the exact location and outline of minute defects such as insulator breakage, bracket corrosion, structural deformation, loose fasteners, and contact wire wear.

[0101] Meanwhile, image-level anomaly classification scores It is obtained by combining the probability of the global category token with the maximum probability value in the anomaly graph, as shown in the following formula.

[0102] In the formula, For nearest neighbor scores based on the memory bank, This is the fusion coefficient. In practical zero-shot inference, it can also be used directly. As an image-level score, this design enables the model to quickly filter out fault-free normal images, while also providing fine-grained anomaly localization results for defective images, fully meeting the practical inspection and detection needs of high-speed rail catenary systems, which require fewer samples, higher precision, finer granularity, and integration.

[0103] 4. To verify the effectiveness of the proposed method, comparative experiments and model ablation experiments were conducted in Tables 1-3, as shown below.

[0104] Table 1. Image-level quantitative analysis results of the inspection of overhead contact line support components.

[0105] Table 2. Pixel-level quantitative analysis results of the inspection of overhead contact line support components.

[0106] Table 3 Ablation Experiment Analysis

[0107] Table 4. Quantitative Analysis Results of Small Samples of Contact Line Support Components

[0108] It's important to note that 0-shot (zero samples) refers to a model that has no training samples on the target dataset and relies entirely on adapters trained on industrial anomaly detection datasets (such as MVTec-AD or Visa, alternately used as auxiliary domains) and textual prompts for direct inference. 1-shot refers to providing one normal sample (without anomalies) for each category in the target dataset to build the memory. 2-shot refers to providing two normal samples for each category. 4-shot refers to providing four normal samples for each category.

[0109] (1) Quantitative comparison experiment with the SOTA method.

[0110] To verify the superiority and effectiveness of the proposed method, quantitative comparative experiments were conducted on a dataset of contact network images captured by detection equipment. Specifically, one representative method from each of the common methods in the anomaly detection field was selected, including UniAD based on a unified detection framework, SimpleNet based on a lightweight reconstruction framework, InvAD based on a diffusion model, and AA-CLIP based on semantic prompts, and these were compared with the RAF-CLIP proposed in this application. It is important to emphasize that, to ensure fairness, all comparison methods used the same backbone network, input image resolution, and experimental settings. The final experimental results are shown in Tables 1 and 2.

[0111] Based on the quantitative experimental results in Tables 1 and 2, the RAF-CLIP proposed in this application demonstrates significant and comprehensive advantages in anomaly detection and localization of overhead contact line support components. In image-level anomaly detection, RAF-CLIP achieves average I-AUROC, I-AP, and I-F1_max of 97.4%, 97.4%, and 97.0% across six component categories, respectively. This result is significantly higher than current mainstream methods such as UniAD, SimpleNet, InvAD, and AA-CLIP, with an overall performance lead of approximately 3 to 16 percentage points. Particularly in categories such as catenary bases, bracing sleeves, positioning connectors, and double-ear sleeves, RAF-CLIP's various image-level indicators remain consistently above 98%, demonstrating extremely high detection reliability and robustness. Although RAF-CLIP's I-AUROC is slightly lower than InvAD and AA-CLIP in the insulator category, its I-AP and I-F1_max remain excellent, without any substantial performance degradation.

[0112] In pixel-level anomaly localization, RAF-CLIP also achieved the best average among all comparison methods, with P-AUROC, P-AP, and P-F1_max of 95.6%, 37.8%, and 52.9%, respectively, representing improvements of approximately 2 to 18 percentage points compared to other methods. Because pixel-level localization requires extremely high precision at defect boundaries, and some anomaly areas are visually similar to the normal background, the P-AP index of all methods is generally low. However, RAF-CLIP's average P-AP remains significantly higher, showing particularly outstanding improvements in categories such as braced sleeves and double-ear sleeves. More notably, for components with complex geometries and small anomaly features, such as positioning clamps and positioning connectors, the P-AUROC of other comparison methods often drops below 80%, while RAF-CLIP consistently maintains above 88%, demonstrating stronger fine-grained localization capabilities and stability.

[0113] By combining image-level and pixel-level results, the RAF-CLIP proposed in this application, through an effective multimodal representation learning strategy, can capture global semantic information and accurately identify local defect details, thereby achieving a comprehensive and stable performance improvement in the task of anomaly detection and localization of overhead contact line support components.

[0114] (2) Ablation experiment analysis.

[0115] The ablation experiments in Table 3 demonstrate that each core module in RAF-CLIP is indispensable and complementary. Starting from the baseline, the experiments progressively added learnable cues, hierarchical features, spatial aggregation, and patch alignment loss to verify the contributions of each component.

[0116] The baseline model's I-AUROC / P-AUROC ratios were 89.3% and 87.4%, respectively, indicating limited performance. Introducing learnable cues improved both metrics to 91.5% and 89.6%, an increase of approximately 2-3 percentage points, demonstrating that data-driven cues effectively capture the common semantic boundaries between normal and abnormal features. After adding hierarchical feature extraction, I-AUROC reached 94.1% and P-AUROC reached 93.7%, improvements of 2.6 and 4.1 percentage points respectively compared to using only cues, indicating that multi-level features are crucial for cross-scale defect localization. Further addition of a spatial aggregation module further improved performance, reaching I-AUROC 96.3% and P-AUROC 94.8%, improvements of approximately 2.2 and 1.1 percentage points respectively, validating the enhancing effect of local contextual information on the perception of small-sized defects. Finally, by adding the patch alignment loss, the complete model achieves the best results: I-AUROC / I-AP 97.4% / 97.4%, P-AUROC / P-AP 95.6% / 37.8%, which is an improvement of about 1.1 and 0.8 percentage points respectively compared to not adding this loss.

[0117] From the removal perspective, both metrics decreased after removing the patch alignment loss; the decrease was even more pronounced after removing the spatial aggregation module; if both hierarchical features and spatial aggregation were removed simultaneously, I-AUROC dropped to 91.5% and P-AUROC to 89.6%, with a degradation much greater than the sum of the effects of removing them individually, demonstrating a significant synergistic gain between hierarchical features and spatial aggregation. In summary, learnable hints, hierarchical features, spatial aggregation, and patch alignment loss complement each other and are organically integrated, jointly achieving the optimal anomaly detection performance of RAF-CLIP.

[0118] (3) Visual analysis of test results.

[0119] To further demonstrate the intuitive qualitative effect of the proposed method in the anomaly detection task of overhead contact line support components, the actual detection results of the model and the comparison results before and after the ablation experiment were visualized, such as... Figure 5 and Figure 6 As shown. Among them, Figure 5 This paper demonstrates the anomaly detection and localization capabilities of the proposed RAF-CLIP model for various key components, including insulators, brackets, and fasteners, under limited sample conditions. The figures clearly show that the method accurately identifies and locates typical anomaly areas such as insulator damage, bracket corrosion, and loose fasteners. The generated anomaly heatmaps exhibit clear, continuous, and complete boundaries, with minimal false positives and false negatives. This fully validates the stable and accurate anomaly detection and localization capabilities of the proposed method even under limited sample conditions.

[0120] Figure 6The anomaly localization performance of the baseline model and the complete RAF-CLIP model on the same contact network component image was further compared. The baseline model refers to a simplified version without core modules such as learnable semantic cues, multi-scale spatial aggregation, and anomaly focusing adapter. Visualization results show that the baseline model has significant missed detection issues, with relatively vague localization of small defect areas and considerable noise in the heatmap; while the complete model can accurately focus on defect areas, producing sharper, smoother, and more complete segmentation heatmaps. The comparison clearly demonstrates that the synergistic effect of the modules proposed in this application significantly improves the model's sensitivity to and localization accuracy of subtle defects, further qualitatively validating the effectiveness and superiority of the proposed method.

[0121] (4) Analysis of the impact of training sample size: To analyze the sensitivity of the proposed method to the number of training images in the overhead contact line few-shot detection task, a set of comparative experiments for different sample sizes were designed, and the specific results are shown in Table 4. The experiments first used the industrial anomaly detection dataset MVTec-AD to pre-train the RAF-CLIP model, and then fine-tuned and calibrated the model using a dataset of collected overhead contact line components. The ALL setting loads all available anomaly images from the dataset and adds an equal number of normal images, i.e., a 1:1 ratio of normal to anomaly images, serving as the reference benchmark for full training. The other 0-shot, 1-shot, 2-shot, and 4-shot settings only provide the corresponding number of normal samples in the target domain without providing any anomaly samples, simulating the extremely scarce small-sample scenario of fault samples in real-world inspections.

[0122] Based on the experimental results shown in Table 4, the following conclusions can be drawn. First, as the number of normal samples provided for each type of component increases from 0 to 4, and eventually to all samples, the model's performance on image-level metrics, including I-AUROC and I-AP, and pixel-level metrics, including P-AUROC and P-AP, shows a continuous and stable improvement. This aligns with the understanding that more normal samples help to more accurately characterize the normal distribution. Second, the method in this application shows a significant performance leap compared to 0-shot when using only 1-shot (1 normal sample per class). When using 4-shot (4 normal samples per class), the image-level I-AUROC and pixel-level P-AUROC reach 95.1% and 92.6%, respectively, approaching the 97.4% and 95.6% of full-scale training (ALL), with a difference of less than 2.5 percentage points. Third, the performance improvement from 4-shot to ALL is limited, indicating that only a very small number of normal samples are needed to significantly reduce the gap with full-scale training. The above results fully verify the efficiency and practicality of the RAF-CLIP method proposed in this application under the condition of few samples in high-speed railway catenary. It can significantly reduce the dependence on large-scale labeled fault samples and has good engineering promotion value.

[0123] Based on the same inventive concept, this application also provides a railway catenary anomaly detection device for implementing the above-described railway catenary anomaly detection method. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more embodiments of the railway catenary anomaly detection device provided below can be found in the limitations of the railway catenary anomaly detection method described above, and will not be repeated here.

[0124] In one exemplary embodiment, such as Figure 7 As shown, a railway catenary anomaly detection device is provided, comprising: The image acquisition module is used to acquire raw inspection images of the target railway catenary support components.

[0125] The image processing module is used to process the original inspection image using an image segmentation algorithm to obtain the target image.

[0126] An anomaly detection module is used to input the target image into an anomaly detection network to obtain an image-level anomaly classification score and a pixel-level anomaly location heatmap of the target railway catenary support component.

[0127] The anomaly detection network is constructed based on the RAF-CLIP model, which includes: a semantic prompting module, a CLIP text encoder, a CLIP image encoder, a multi-scale spatial aggregation module, a lightweight anomaly focusing adaptation module, and a cross-modal semantic alignment module.

[0128] The semantic prompting module is used to construct a prompting lexicon, which includes: multiple sets of bi-branch text semantic prompts; the CLIP text encoder is used to encode the prompting lexicon and output text prototype features; the CLIP image encoder is used to extract multi-level features from the original inspection image to obtain multi-level feature maps with different depths and receptive fields; the multi-scale spatial aggregation module is used to fuse neighborhood context information on each level feature map using sliding windows of different sizes to obtain multi-scale fused features; the lightweight anomaly focusing adaptation module is used to enhance the multi-scale fused features based on a learnable self-attention mechanism to obtain a visual feature set; the cross-modal semantic alignment module is used to output image-level anomaly classification scores and pixel-level anomaly localization heatmaps based on the text prototype features and the visual feature set.

[0129] The railway catenary anomaly detection method or apparatus described in the above embodiments constructs a learnable normal / abnormal semantic prompting module. It forms a dual-branch text prompt by concatenating learnable prefixes and fixed-state suffixes, and employs multiple sets of independent prefixes and orthogonality constraints to enhance semantic diversity and discriminability. A frozen CLIP text encoder extracts stable text prototype features. Furthermore, a multi-scale spatial aggregation module is designed, utilizing sliding windows of different sizes and Gaussian weighted fusion of neighborhood context to enhance feature extraction capabilities for defects at different scales. A lightweight anomaly focusing adapter is introduced, guiding global category tokens to focus on distant anomaly regions and local tile tokens to focus on subtle defect patterns through a learnable self-attention mechanism. A cross-modal semantic alignment and multi-task joint loss optimization model are employed to finally generate a pixel-level anomaly detection map. The above method or apparatus fills the gap in anomaly detection of high-speed railway catenary support devices under limited sample conditions and exhibits stronger generalization and accuracy compared to traditional manual prompting methods.

[0130] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 8As shown, the computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores image-level anomaly classification scores and pixel-level anomaly location heatmaps for target railway catenary support components. The I / O interfaces are used for information exchange between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a railway catenary anomaly detection method.

[0131] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer equipment to which the present application is applied. Specific computer equipment may include, for example, [the following is a list of possible additional structures]. Figure 8 The embodiments show more or fewer components, combinations of certain components, or different component arrangements. In one exemplary embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, which the processor executes to implement the steps in the above-described method embodiments.

[0132] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0133] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0134] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0135] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, etc., and are not limited to these.

[0136] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0137] This application uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. In summary, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for detecting anomalies in railway overhead contact lines, characterized in that, include: Obtain the original inspection images of the target railway catenary support components; The original inspection image is processed using an image segmentation algorithm to obtain the target image; The target image is input into the anomaly detection network to obtain the image-level anomaly classification score and pixel-level anomaly localization heatmap of the target railway catenary support component. The anomaly detection network is built based on the RAF-CLIP model; the RAF-CLIP model includes: a semantic prompting module, a CLIP text encoder, a CLIP image encoder, a multi-scale spatial aggregation module, a lightweight anomaly focusing adaptation module, and a cross-modal semantic alignment module; The semantic prompting module is used to construct a prompting lexicon, which includes: multiple sets of bi-branch text semantic prompts; the CLIP text encoder is used to encode the prompting lexicon and output text prototype features; the CLIP image encoder is used to extract multi-level features from the original inspection image to obtain multi-level feature maps with different depths and receptive fields; the multi-scale spatial aggregation module is used to fuse neighborhood context information on each level feature map using sliding windows of different sizes to obtain multi-scale fused features; the lightweight anomaly focusing adaptation module is used to enhance the multi-scale fused features based on a learnable self-attention mechanism to obtain a visual feature set; the cross-modal semantic alignment module is used to output image-level anomaly classification scores and pixel-level anomaly localization heatmaps based on the text prototype features and the visual feature set.

2. The railway catenary anomaly detection method according to claim 1, characterized in that, The original inspection image is processed using an image segmentation algorithm to obtain the target image, specifically including: The original inspection image is segmented using the SAM algorithm to obtain segmented regions; The segmented regions are then filtered to obtain the target image.

3. The railway catenary anomaly detection method according to claim 1, characterized in that, Building a suggestion dictionary specifically includes: Construct a learnable prefix word embedding sequence of a set length; the learnable prefix word embedding sequence is used to capture the contextual features of the surface texture, structural morphology, and defects of the contact wire. The learnable prefix word embedding sequence is concatenated with both fault-free fixed-state suffixes and faulty fixed-state suffixes to obtain a set of bi-branch text semantic prompts. A suggestion word library is constructed based on semantic suggestions from multiple sets of bi-branch texts and orthogonality constraints.

4. The railway catenary anomaly detection method according to claim 1, characterized in that, The text prototype features of the prompt word library are output through text encoding, specifically including: Text encoding is performed on each set of two-branch text semantic prompts in the prompt word library to obtain the text features of each set of two-branch text semantic prompts; The text prototype features are obtained by performing mean aggregation on multiple sets of text features; the text prototype features include: normal text prototype features and abnormal text prototype features.

5. The railway catenary anomaly detection method according to claim 1, characterized in that, Multi-level feature extraction is performed on the original inspection image to obtain multi-level feature maps with different depths and receptive fields. Specifically, this includes: dividing the original inspection image into a sequence of image blocks using a block embedding operation, encoding the image block sequence, extracting multi-level feature representations, and obtaining multi-level feature maps with different depths and receptive fields; the multi-level feature maps include: global category tokens and local tile tokens. By employing sliding windows of different sizes to fuse neighborhood context information on each level of feature map, multi-scale fusion features are obtained. Specifically, this includes: reshaping each layer's tile tokens into a two-dimensional spatial feature map; using three different sizes of sliding windows, calculating the neighborhood weights at each position in the two-dimensional spatial feature map using a Gaussian kernel function, and performing weighted summation to obtain multi-scale spatial aggregation features; and concatenating the multi-scale spatial aggregation features with the global category token to form multi-scale fusion features. The lightweight anomaly focusing adaptation module includes an adapter and a CLIP projection layer. The adapter is used to guide global category tokens to aggregate global anomaly information and guide local tile tokens to enhance defect perception capabilities based on multi-scale fusion features, thereby obtaining enhanced features. The CLIP projection layer is used to map the enhanced features to a joint embedding space to obtain a set of visual features.

6. The railway catenary anomaly detection method according to claim 1, characterized in that, Based on the text prototype features and the visual feature set, an image-level anomaly classification score and a pixel-level anomaly localization heatmap are output, specifically including: Using the text prototype features as semantic anchors, cosine similarity is calculated with the visual features in the visual feature set and accumulated to obtain the probability that each tile token belongs to the abnormal category; The probability sequence of each tile token belonging to the anomaly category is reshaped into a two-dimensional probability map, and the two-dimensional probability map is upsampled and Gaussian smoothed to obtain a pixel-level anomaly localization heatmap. The global category token probability is fused with the maximum probability value in the pixel-level anomaly localization heatmap to obtain the image-level anomaly classification score.

7. The railway catenary anomaly detection method according to claim 1, characterized in that, The multi-task joint loss optimization function used when training the RAF-CLIP model is constructed based on image-level classification loss, pixel-level segmentation loss, and patch alignment loss. The image-level classification loss is determined based on the probability corresponding to the image-level ground truth label and the global category token. The pixel-level segmentation loss is determined based on the probability that each patch token belongs to the abnormal category and the pixel-level ground truth mask. The patch alignment loss determines the normal patch token and the abnormal patch token at each level and scale.

8. A railway catenary anomaly detection device, characterized in that, include: The image acquisition module is used to acquire the original inspection images of the target railway catenary support components; The image processing module is used to process the original inspection image using an image segmentation algorithm to obtain the target image; An anomaly detection module is used to input the target image into an anomaly detection network to obtain an image-level anomaly classification score and a pixel-level anomaly location heatmap of the target railway catenary support component. The anomaly detection network is built based on the RAF-CLIP model; the RAF-CLIP model includes: a semantic prompting module, a CLIP text encoder, a CLIP image encoder, a multi-scale spatial aggregation module, a lightweight anomaly focusing adaptation module, and a cross-modal semantic alignment module; The semantic prompting module is used to construct a prompting lexicon, which includes: multiple sets of bi-branch text semantic prompts; the CLIP text encoder is used to encode the prompting lexicon and output text prototype features; the CLIP image encoder is used to extract multi-level features from the original inspection image to obtain multi-level feature maps with different depths and receptive fields; the multi-scale spatial aggregation module is used to fuse neighborhood context information on each level feature map using sliding windows of different sizes to obtain multi-scale fused features; the lightweight anomaly focusing adaptation module is used to enhance the multi-scale fused features based on a learnable self-attention mechanism to obtain a visual feature set; the cross-modal semantic alignment module is used to output image-level anomaly classification scores and pixel-level anomaly localization heatmaps based on the text prototype features and the visual feature set.

9. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that the processor executes the computer program to implement the railway catenary anomaly detection method according to any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the railway catenary anomaly detection method according to any one of claims 1-7.

Images