A pantograph image-based anomaly detection method and system

Abstract

The application relates to the technical field of image data processing, and discloses an abnormality detection method and system based on pantograph images, the abnormality detection method based on pantograph images comprising the following steps: acquiring a series of first images of pantograph regions, calculating the occlusion area and the illumination uniformity of the first images, and screening out the corresponding first images when either the occlusion area or the illumination uniformity does not satisfy a preset condition; identifying each component of the pantograph in the first images, judging whether the components are abnormal according to a preset condition, and outputting a first judgment result. Therefore, since a part of the first images in the series of first images that does not meet the requirements is screened out, and each component is identified, a more accurate and reliable pantograph abnormality detection result can be obtained.

Description

Technical Field

[0001] This invention relates to the field of image data processing technology, and in particular to an anomaly detection method and system based on pantograph images. Background Technology

[0002] With the development of image recognition technology, high-definition cameras can be installed along the train route or on the train to collect real-time image data of the pantograph. Image recognition algorithms can then be used to analyze and process these images, extracting the pantograph's feature information and comparing it with preset normal state characteristics to determine if there are any abnormalities in the pantograph.

[0003] For example, the patent with publication number CN119251755A obtains pantograph images based on target recognition algorithms, then processes the recognized images using specific algorithms, and performs data augmentation processing on the images based on GAN networks to obtain pantograph anomaly detection results.

[0004] However, when locomotives are operating outdoors, the complexity of their operation and the high complexity of their images make it difficult to detect pantograph malfunctions. Specifically, existing pantograph image detection methods merely employ different approaches to enhance image processing capabilities, which are insufficient to address the complexities of pantographs, such as the aforementioned image enhancement processing techniques. Therefore, existing pantograph image detection methods cannot obtain sufficiently accurate and reliable pantograph detection results. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide an anomaly detection method based on pantograph images. It proposes a new technical route for anomaly detection of pantograph images, which can adapt to the complex situation of locomotives running outdoors and obtain more accurate and reliable detection results.

[0006] The objective of this invention is achieved through the following technical solution:

[0007] In a first aspect, an anomaly detection method based on pantograph images is provided, which involves acquiring a series of first images of pantograph areas, calculating the occlusion area and illumination uniformity of the first images, and filtering out the corresponding first images if either the occlusion area or the illumination uniformity does not meet a preset condition; identifying the components of the pantograph in the first images, determining whether each component is abnormal according to preset conditions, and outputting a first determination result.

[0008] Its beneficial effects are as follows: by filtering out the unqualified portions of the first images from a series of first images, and then identifying each component, more accurate and reliable pantograph anomaly detection results can be obtained. For example, by calculating the uniformity of illumination, images with severely uneven illumination distribution due to strong light reflection can be accurately identified. If the uniformity of illumination of the first image does not meet the preset conditions, it can be filtered out in time. By calculating the occlusion area, the degree of occlusion of the image by rain, snow, and the gantry can be quantified. When the occlusion area of ​​the first image exceeds a preset threshold, it indicates that the occlusion by rain, snow, and / or the gantry has affected the usability of the image, and it should be filtered out at this time.

[0009] Preferably, identifying the components of the pantograph in the first image includes: carbon skateboard identification; inputting the first image into a semantic segmentation model to generate a mask image of the first image, the mask image dividing the carbon skateboard region and the non-carbon skateboard region to identify the carbon skateboard; wherein, the semantic segmentation model includes: an encoder and a decoder, the decoder adopting a nested skip connection structure and integrating a CBAM attention module.

[0010] Preferably, the first image with the mask image attached is input into the first large model, and the first large model generates a second determination result.

[0011] Preferably, determining whether each component is abnormal based on preset conditions and outputting the first determination result includes: determining whether a component is missing in the first image based on preset topological dependencies between the components; identifying the dynamic brightness threshold of the area surrounding the contact wire in the first image and using optical flow tracing technology to determine whether sparking occurs; calculating whether the proportion of the broken length of the carbon sliding plate in the first image exceeds a threshold, calculating whether the tilt angle of the carbon sliding plate in the first image exceeds a threshold, and determining that the carbon sliding plate is abnormal if the proportion of the broken length or the tilt angle exceeds the threshold; and calculating whether the relative position of the pantograph and the contact wire in the first image exceeds a threshold, and determining that it is abnormal if it exceeds the threshold.

[0012] Preferably, the topological dependency relationship is the topological dependency relationship between the base, the bow angle, and the carbon plate: if the base is not detected, it is determined that the pantograph is lost as a whole; if the base is detected but the bow angle is not detected, it is determined that the bow angle is lost; if the bow angle is detected but the carbon plate is not detected, it is determined that the carbon plate has detached.

[0013] Preferably, the first image that is determined to be a spark phenomenon in the first determination result, and the first image that does not meet the illumination uniformity, are selected and marked as disputed images; each disputed image is attached with a component bounding box obtained through target recognition, and each disputed image is also attached with a confidence level; the disputed image with the bounding box and the confidence level are input into the second large model, and the second large model outputs the classification probability as the third determination result.

[0014] Preferably, a semantic consistency score is obtained based on the detection category of the first judgment result and the semantic category of the second large model, and a classification consistency weight is assigned; a confidence weight is assigned to the confidence score and a probability judgment weight is assigned to the classification probability; the confidence score, confidence weight, classification probability, probability judgment weight, classification consistency weight, and semantic consistency score are input into the fusion decision algorithm to obtain a comprehensive score; it is determined whether the comprehensive score exceeds a threshold, and a fourth judgment result is output.

[0015] Preferably, the step of calculating the occlusion area and illumination uniformity of the first image, and filtering out the corresponding first image if either the occlusion area or the illumination uniformity does not meet a preset condition, includes: segmenting the occluder based on the Mask R-CNN model, calculating whether the proportion of the occlusion area of ​​the occluder in the first image exceeds a threshold, and filtering out the image if it does; and calculating whether the variance of the luminance component in HSV spatial analysis exceeds a threshold, and filtering out the image if it does.

[0016] Preferably, the step of calculating the occlusion area and illumination uniformity of the first image, and filtering out the corresponding first image if either the occlusion area or the illumination uniformity does not meet a preset condition, further includes: determining whether the clarity of the first image exceeds a threshold.

[0017] In a second aspect, an anomaly detection system based on pantograph images includes a storage device and a processor, the storage device storing a computer-readable medium for the above-described anomaly detection method based on pantograph images, and the processor executing the computer-readable medium. Attached Figure Description

[0018] Figure 1 This is a schematic flowchart of an anomaly detection method based on pantograph images according to an embodiment of this application;

[0019] Figure 2 This is a schematic diagram of the component loss detection process according to an embodiment of this application;

[0020] Figure 3 This is a schematic diagram illustrating the construction of a semantic segmentation model according to an embodiment of this application;

[0021] Figure 4This is a schematic diagram of the second model architecture according to an embodiment of this application;

[0022] Figure 5 This is a schematic flowchart of another anomaly detection method based on pantograph images according to an embodiment of this application;

[0023] Figure 6 This is a schematic flowchart of another anomaly detection method based on pantograph images according to an embodiment of this application. Detailed Implementation

[0024] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] When locomotives are running outdoors, pantograph images are easily affected by strong light reflection, rain and snow obstruction, and gantry shadows, resulting in a high false detection rate for traditional target detection and segmentation models. Even when image processing enhancement techniques are applied, such as data augmentation based on GAN networks, these are only means of enhancing the image and cannot solve the problem of obtaining accurate and reliable results from pantograph images.

[0026] See Figures 1-4 The present invention provides a technical solution:

[0027] An anomaly detection method based on pantograph images is proposed. The method involves acquiring images of the pantograph area in real time using a high-definition camera, which will be referred to as the first image for ease of explanation. The occlusion area and illumination uniformity of the first image are calculated. If either the occlusion area or the illumination uniformity does not meet a preset condition, the corresponding first image is filtered out.

[0028] In detail, during the acquisition and analysis of pantograph images, factors such as strong light reflection, rain and snow obstruction, and gantry shadows can seriously affect image quality, thereby interfering with the accuracy and reliability of subsequent anomaly detection.

[0029] Strong light reflection occurs when sunlight or other strong light sources directly illuminate the surface of the pantograph, producing intense reflected light that causes localized or large areas of excessive brightness in the first image captured by a high-definition camera. These overly bright areas lead to abnormal pixel values ​​in the image, resulting in the loss of detailed features of the pantograph itself. For example, the edges of critical components of the pantograph may become blurred or even completely obscured by overly bright areas due to strong light reflection.

[0030] In this embodiment, by calculating the uniformity of illumination, images with severely uneven illumination distribution caused by strong light reflection can be accurately identified. If the uniformity of illumination of the first image does not meet the preset conditions, it can be screened out in time, thus avoiding the use of such images containing a large amount of invalid information for subsequent anomaly detection, thereby preventing misjudgment caused by strong light reflection.

[0031] Rain and snow can affect image quality. In severe weather conditions, raindrops or snowflakes can fall on the pantograph surface, partially or completely obscuring critical parts of the pantograph, resulting in incomplete feature information in the first acquired image. For example, rain and snow may cover the carbon strip of the pantograph, making it impossible to accurately determine the degree of wear or the presence of cracks or other anomalies. Similarly, in the railway operating environment, the gantry may cast shadows at different times of day or directly obscure the pantograph.

[0032] In this embodiment, the degree of occlusion of the image by rain, snow, and the gantry can be quantified by calculating the occlusion area. When the occlusion area of ​​the first image exceeds a preset threshold, it indicates that the occlusion by rain, snow, and / or the gantry has affected the usability of the image. At this time, filtering it out can effectively avoid abnormal detection errors caused by rain and snow occlusion.

[0033] Furthermore, the components of the pantograph are identified in the first image, and it is determined whether each component is abnormal according to preset conditions, and a first determination result is output. In this way, by filtering out the unqualified parts of the first images from a series of first images, and then identifying each component, more accurate and reliable pantograph anomaly detection results can be obtained.

[0034] Next, refer to Figure 1 The following description will be provided in conjunction with a specific embodiment.

[0035] An anomaly detection method based on pantograph images includes the following steps:

[0036] A series of first images of the pantograph area are acquired. The occlusion area and illumination uniformity of the first images are calculated. If either the occlusion area or the illumination uniformity does not meet a preset condition, the corresponding first image is filtered out. Specifically, this includes:

[0037] S100. The obtained first image is filtered to remove images that do not meet preset conditions, wherein the preset conditions include:

[0038] S110. Sharpness assessment can be performed using the improved Brenner gradient algorithm. The formula for the algorithm is:

[0039] The threshold is set to S≥120. In the formula, S is the sharpness score, calculated using the improved Brenner gradient algorithm, which represents the sum of image gradient energies and reflects the sharpness of image edges. x These are the column coordinates (horizontal direction) of the image pixels. y These are the row coordinates (vertical direction) of the image pixels. I(x,y) For the image in coordinates (x, y) The pixel intensity value at a given location represents brightness (0-255) in a grayscale image. RGB images must be converted to grayscale first.

[0040] S120. Occlusion detection: Based on the Mask R-CNN model, segment occlusions such as gantry cranes, rain and snow, calculate the proportion of occlusion area Q1 in the total image area Q, and set the threshold to occlusion area Q1≤Q15%.

[0041] S130, Illumination uniformity: Calculate the variance S of the luminance component (V channel) through HSV spatial analysis. 2 The threshold is set to S. 2 ≤500, if S 2 If the value is greater than the threshold, it indicates that the image is overexposed.

[0042] If any of the above indicators exceed the limit, the current frame is discarded and image resampling is triggered; otherwise, the target detection module is entered. Of course, S110-S130 can be performed sequentially or simultaneously. This ensures that the gradient is calculated only for the pantograph region (ROI), eliminating background interference.

[0043] S200. The first image that meets the preset conditions is input to the target detection module one by one. The target detection module performs adaptive anchoring on each component of the pantograph, thereby identifying each component of the pantograph in the first image.

[0044] Specifically, the object detection module can employ an object detection network. For example, it can use basic structures such as CNN convolution, depthwise separable convolution, and dilated convolution combined with SPPF (spatial pyramid pooling), PAA (label smoothing assignment), and PAN (path aggregation network) modules to construct a deep learning network, thereby enhancing feature extraction capabilities and the detection effect of scale-based objects. The first image size of the input can be 640×640.

[0045] S300: Based on the target area image acquired in S200, determine whether each component has any abnormalities according to preset conditions, and output the first determination result, including:

[0046] S310. Based on the preset topological dependencies between components, determine whether any components are missing in the first image.

[0047] Specifically, key components are defined, including: the base, the fixed structure connecting the pantograph to the roof; the pantograph angles, the bending or raised structures at both ends of the pantograph's carbon contact plate; and the carbon contact plate, the conductive contact plate that directly contacts the overhead contact line.

[0048] Combination Figure 2 As shown, according to the preset topology dependency rules, the following determinations are made: if the base is not detected, the pantograph is considered lost. If the base is detected but the pantograph corner is not detected, the pantograph corner is considered lost. If the pantograph corner is detected but the carbon strip is not detected, the carbon strip is considered detached. Furthermore, the final alarm is triggered only if the same component is detected as missing in three consecutive frames (i.e., three first images) to avoid false alarms.

[0049] In this way, the judgment method based on the logical relationship between components avoids the misjudgment or omission that may occur when detecting a single component.

[0050] S320. Identify the dynamic brightness threshold of the area surrounding the contact wire in the first image (such as an area with a pixel value exceeding 220 in the image) and use optical flow tracing technology. If the high brightness area is tracked in 3 or more consecutive frames, it is determined to be a real spark and that it occurs around the contact wire, thus determining that a spark phenomenon has occurred.

[0051] S330, Determine if the carbon skateboard is abnormal, including:

[0052] Calculate whether the proportion of the fracture length of the carbon slide in the first image exceeds a threshold, and calculate whether the tilt angle of the carbon slide in the first image exceeds a threshold. If the proportion of fracture length or the tilt angle exceeds the threshold, the carbon slide is determined to be abnormal.

[0053] Specifically, fracture detection is performed to calculate the percentage of fracture length in the carbon slide mask:

[0054]

[0055] In the formula, L 断裂 L is the fracture length. 总 This represents the total length of the carbon skateboard.

[0056] The threshold is set to 5%; anything exceeding 5% is considered a break.

[0057] Tilt detection is performed by using the RANSAC algorithm to fit the straight line of the carbon skateboard edge and calculating the deviation from the reference angle. The threshold is set at 3°, and anything exceeding the threshold is considered an abnormal tilt.

[0058] Typically, fracture and tilting occur independently; therefore, if either fracture or tilting exceeds a threshold, it is considered an anomaly in the carbon slide plate. Thus, by assessing the anomaly of the carbon slide plate through two dimensions, a reliable judgment can be obtained.

[0059] S340. Calculate whether the relative position of the pantograph and the contact wire in the first image exceeds a threshold. If it exceeds the threshold, it is judged as abnormal. Specifically, the calculation formula is:

[0060]

[0061] In the formula, H The pantograph height, i.e., the vertical distance from the apex of the pantograph to the track plane, can be determined using an image. H t C represents the pantograph height in the current frame; C represents the overhead contact line height, which can be determined by a pre-defined static height value.

[0062] The above S310-S340 can be performed simultaneously or sequentially, and the execution order can be different.

[0063] Understandably, by conducting detailed and targeted assessments of various aspects such as component integrity, spark phenomena, carbon strip condition, and the relative position of the pantograph and the overhead contact line, the operating status of the pantograph can be comprehensively and accurately grasped, and various abnormalities can be detected in a timely manner. This provides accurate and reliable results for the maintenance and management of the pantograph, ensuring the safety and stability of locomotive operation.

[0064] In some embodiments, identifying the carbon strip in the pantograph in the first image includes:

[0065] The first image is input into the semantic segmentation model to generate a mask image of the first image. The mask image divides the carbon skateboard region and the non-carbon skateboard region to identify the carbon skateboard. The semantic segmentation model includes an encoder and a decoder. The decoder adopts a nested skip connection structure and integrates a CBAM attention module.

[0066] For example, an improved U-Net++ architecture is adopted to enhance the fine segmentation capability of the carbon skateboard region through multi-level feature fusion and attention mechanisms, as follows:

[0067] refer to Figure 3Understanding the concept, using ResNet-34 as the encoder backbone allows for the loading of pre-trained weights on the ImageNet dataset to accelerate convergence and enable multi-scale feature extraction. For example, ResNet-34 generates feature maps of different scales through five stages (Stage 1 to Stage 5), with sizes of 512×512 (Stage 1), 256×256 (Stage 2), 128×128 (Stage 3), 64×64 (Stage 4), and 32×32 (Stage 5), and channel numbers of 64, 128, 256, 512, and 512, respectively. The output feature map from each stage is passed to the corresponding layer of the decoder via skip connections, preserving spatial information at different granularities to output multi-scale feature maps.

[0068] The decoder employs nested skip connections, for example, using the dense nested connection design of U-Net++, fusing features from the same-level encoder with features from the deeper decoder through lateral connections. Each skip connection integrates a CBAM (Convolutional Block Attention Module), which includes dual branches: channel attention dynamically adjusts channel weights through global average pooling and fully connected layers to focus on key feature channels. Spatial attention uses convolutional layers to generate spatial weight maps, highlighting the spatial location of the carbon skateboard region. Based on upsampling strategies, such as using bilinear interpolation for 2x upsampling, combined with the refined features from skip connections, the original resolution is gradually restored. Finally, the decoder output is activated by convolution and activation functions, generating a 512×512 pixel binary mask, where white areas (pixel value = 1) represent the carbon skateboard region, and black areas (pixel value = 0) represent non-carbon skateboard regions.

[0069] In this example, during the training of the semantic segmentation model, a composite loss function is used, namely, a weighted sum of Dice Loss and Focal Loss, to balance the class imbalance between the carbon skateboard region (foreground) and the background. Data augmentation is also performed, with random rotation ±30°, random horizontal / vertical flip probability of 0.5, and fill mode of mirror reflection and brightness jitter ±20% to simulate lighting changes and rain and snow noise. For example, semi-transparent stripe noise (random direction, density 5%~15%) and Gaussian snowflakes (density 3%~8%) are superimposed.

[0070] Thus, the method demonstrates significant advantages in practical detection under complex working conditions. Addressing the texture blurring caused by edge wear, localized fractures, or oxide layers on the carbon sliding plate during high-speed pantograph operation, the nested jump connection structure, by fusing multi-scale features, can capture both the macroscopic outline of the overall carbon sliding plate and precisely locate minute cracks or gaps. The integrated CBAM attention mechanism effectively suppresses the impact of dynamic interference such as contact wire arc spots and rain / snow splashes on the segmentation results. Channel weight recalibration enhances the recognition of the metallic reflective features of the carbon sliding plate's conductive coating, while spatial attention focuses on high-risk areas such as the connection between the sliding plate and the pantograph corner. For scenarios where oil or snow may adhere to the carbon sliding plate surface, the mirror reflection filling and noise simulation mechanism introduced during model training enhances the robustness of the segmentation algorithm against non-uniform occlusion, ensuring that even if parts of the sliding plate are covered by semi-transparent foreign objects, the complete outline can still be inferred from contextual features. This high-precision pixel-level segmentation capability provides a reliable foundation for subsequent calculations of key indicators such as fracture length ratio and tilt angle. In particular, the accurate delineation of the sawtooth fracture edge of the carbon slide plate avoids the misjudgment of fracture caused by uneven illumination in the traditional threshold segmentation method, making the damage quantification assessment based on image analysis closer to the real physical state.

[0071] Based on the above examples, although targeted detection is carried out for different abnormalities of the pantograph, there may still be problems with inaccurate judgment for image samples such as subtle abnormalities (such as micro-cracks in the carbon sliding plate) and controversial samples (such as distinguishing between sparks and reflections).

[0072] To this end, two implementations were subsequently disclosed, one for identifying microcracks in carbon skateboards and the other for distinguishing between sparks and reflections.

[0073] First embodiment, reference Figure 5 As shown, after S300, S400A is performed, where the first image with the mask image is input into the first large model, and the first large model generates the second judgment result.

[0074] Specifically, the first large model is an image large model, and the first image input to the first large model is the first image that contains the carbon skateboard and is identified.

[0075] For example, the first large model selects BLIP-2 as the base model. Candidate images with masked labels are input into the visual-language large model built on the BLIP-2 architecture for secondary validation. The model achieves domain adaptation through the LoRA (Low-Rank Adaptation) lightweight fine-tuning method: a trainable rank decomposition matrix (e.g., rank dimension set to 64) is inserted between the original Q-Former and visual Transformer modules of BLIP-2, freezing 95% of the original model's parameters and updating only the weights of the newly added adaptation layer, thereby injecting prior knowledge while preserving general visual representation capabilities. The fine-tuning stage employs a multi-task joint loss function, including cross-entropy classification loss, contrastive learning loss, and image-text matching loss.

[0076] Therefore, during input, candidate images are pixel-wise overlaid with a carbon skateboard mask, high-confidence anomalous regions are cropped and enlarged to a resolution of 1024×1024. The processed image patch is then input into a large model along with a preset prompt template, such as "Does this image show a real microcrack in the carbon skateboard? Possible options: A. Yes, B. No, C. Uncertain." Fine-grained analysis results are generated through joint visual-language reasoning, and the final alarm determination includes:

[0077] If the number of first images identified by the second determination result includes the first determination result, then the second determination result is output. If the second determination result and the output result of the first determination result have an intersection or the first determination result includes the second determination result, then the second determination result is output.

[0078] Second embodiment, reference Figure 6 As shown, S400B continues after S300:

[0079] S410B: Select the first image that is determined to be a spark phenomenon in the first determination result, and the first image that does not meet the illumination uniformity (image reflection), and mark these two types of first images as disputed images.

[0080] S420B: Based on the target recognition algorithm described in the above embodiments or any target recognition method, each disputed image is accompanied by a bounding box of components obtained through target recognition. Furthermore, each disputed image is also accompanied by a confidence score. This confidence score can be further analyzed during target recognition of the disputed image (i.e., the first image that may exhibit spark phenomena or image reflection). The confidence score is calculated based on a classification model, which outputs two probability values. The larger probability value is used as the confidence score for the corresponding category (spark phenomenon or image reflection) of the disputed image. For example, if the model outputs a probability of 0.8 for a disputed image being a spark phenomenon and 0.2 for an image reflection, then 0.8 is used as the confidence score for the image being a spark phenomenon, and 0.2 is used as the confidence score for it being an image reflection. In this way, by performing the above processing on each disputed image, the confidence scores for each image being either spark phenomena or image reflection can be obtained.

[0081] S430B: Input the controversial image with bounding box and the confidence level into the second large model. The second large model outputs the classification probability as the third decision result.

[0082] For example, the second large model is also a large image model, see reference. Figure 4 Understanding its architecture, including the vision-language joint framework consisting of the ViT encoder, LoRA fine-tuning module, and OPT generation module, allows us to input controversial images (such as samples with suspected spark phenomena or image reflections) with attached component bounding boxes and primary detection confidence scores into the model. The model is then guided to perform fine-grained analysis using a preset prompt template: "Determine whether the [bounding box marked area] in the image is a [true anomaly type]. It is necessary to distinguish [normal interference factors] and refer to the primary detection confidence score {confidence score value}." Finally, the model outputs the classification probability results for the controversial category (e.g., presented in a quantitative form as "True spark probability: 0.92, reflection interference probability: 0.08").

[0083] Of course, S400A and S400B can be executed simultaneously or sequentially to output the final result and obtain a more accurate pantograph anomaly detection result.

[0084] In some embodiments, according to the above embodiments, evaluation can also be performed using a fusion algorithm to overcome the problems of insufficient model generalization performance and the difficulty of a single detection module in covering dynamically changing outdoor scenes.

[0085] Specifically, S500 follows S400B; please refer to [the relevant documentation / reference]. Figure 6 As shown: Based on the parameters in the above embodiments, to enhance the accuracy of the judgment and make a final alarm decision, the fusion algorithm is as follows:

[0086]

[0087] In the formula, scorei is the overall score, ranging from 0 to 1, with anomalies scoring above 0.9 considered correct anomalies;

[0088] si α represents the initial detection confidence, including the confidence of spark and reflection phenomena in the aforementioned examples; α represents the initial detection confidence weight, which is obtained through adjustment or training. Pi This represents the decision probability of the second largest model. β Determine probability weights for large models, based on adjustments or training; The semantic consistency score is obtained based on the judgment category of the first judgment result and the semantic category output by the second model. This represents the classification consistency weight. α, β, λ The optimal value is determined, for example, through grid search and cross-validation: α=0.5, β=0.3, λ=0.2 . Semantic consistency score is determined by comparing the categories of the first judgment result. ci Category of large semantic determination models ei Calculate the cosine similarity.

[0089] The constraints are: α+β+λ=1 .

[0090] An anomaly detection system for pantograph images according to an embodiment of this application includes a storage device and a processor. The storage device stores a computer-readable medium for implementing the anomaly detection method for pantograph images in the above embodiment. The processor is used to execute the computer-readable medium. The storage device and the processor can be located in any computer system or an integrated system of multiple computers.

[0091] The above are merely preferred embodiments of the present invention. It should be understood that the present invention is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. It can be used in various other combinations, modifications, and environments, and can be modified within the scope of the present invention through the above teachings or related technologies or knowledge. Modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention should be within the protection scope of the appended claims.

Claims

1. A pantograph image-based anomaly detection method, characterized by, The method comprises the following steps: S100, acquiring a series of first images of pantograph regions, calculating the occlusion area and the illumination uniformity of the first images, and screening out the corresponding first images if either of the occlusion area or the illumination uniformity does not meet a preset condition, comprising: S110, clarity evaluation; S120, occlusion detection, based on a Mask R-CNN model to segment the pantograph, rain and snow, etc., calculate the proportion of the occlusion area Q1 in the total image area Q, and set the threshold value as Q1≤Q15%; S130, light uniformity, calculate the variance S of the brightness component by HSV space analysis 2 , the threshold is set as S 2 ≤500, if the value of S 2 is greater than the threshold value, it indicates that the image is overexposed; If any of the above indicators is out of limit, discard the current frame and trigger image re-sampling; otherwise, enter the target detection module; S200, input the first images meeting the preset condition into the target detection module one by one, the target detection module performs adaptive anchor frame on each component of the pantograph, thereby identifying each component of the pantograph in the first image; S300, according to the target region image obtained in S200, determine whether each component is abnormal according to the preset condition, and output a first determination result, comprising: according to the preset topological dependency relationship between each component, determine whether component loss occurs in the first image; And identifying each component of the pantograph in the first image comprises: carbon slide plate identification; Input the first image into a semantic segmentation model to generate a mask image of the first image, the mask image divides the carbon slide plate region and the non-carbon slide plate region to identify the carbon slide plate; Wherein, the semantic segmentation model comprises an encoder and a decoder, the decoder adopts a nested jump connection structure, integrates a CBAM attention module in each level of jump connection, which contains a channel attention and a spatial attention double branch: the channel attention dynamically adjusts the channel weight through global average pooling and a fully connected layer; the spatial attention adopts a convolution layer to generate a spatial weight map, highlighting the spatial position of the carbon slide plate region; S400A, input the first image with the mask image into a first large model, the first large model generates a second determination result; S410B, select the first image in the first determination result which is determined as a spark phenomenon and the first image which does not meet the illumination uniformity, and mark them as controversial images; S420B, each of the controversial images is attached with a component bounding box obtained through target recognition, and each of the controversial images is also attached with a confidence, the confidence is calculated based on a classification model, which outputs two probability values, and the larger probability value is taken as the confidence of the corresponding category of the controversial image; S430B, input the controversial image with the bounding box and the confidence into a second large model, the second large model outputs a classification probability result for the controversial category as a third determination result; S500, according to the detection category of the first determination result and the semantic category of the second large model, obtain a semantic consistency score and give a classification consistency weight; give a confidence weight to the confidence and a probability judgment weight to the classification probability. inputting the confidence, the confidence weight, the classification probability, the probability judgment weight, the classification consistency weight, and the semantic consistency score into a fusion decision algorithm to obtain a comprehensive score, including: wherein scorei is the comprehensive score, ranging between 0 and 1, and an anomaly with a score higher than 0.9 is determined as a correct anomaly; si is a primary detection confidence, including spark phenomenon, reflection phenomenon confidence; a is a primary detection confidence weight; Pi is a second large model decision probability, β is a large model decision probability weight; is a semantic consistency score obtained according to the decision category of the first decision result and the semantic category output by the second large model, is a classification consistency weight; wherein, α, β, λ determined by grid search and cross-validation; The constraints are: α+β+λ=1 ; determining whether the comprehensive score exceeds a threshold value, and outputting a fourth determination result.

2. The pantograph image-based anomaly detection method according to claim 1, wherein: determining whether a component is missing in the first image according to a preset topological dependency relationship between the components; identifying a dynamic brightness threshold of a catenary peripheral region in the first image and using an optical flow tracking technology to determine whether a spark phenomenon occurs; calculating whether a fracture length proportion of the carbon slide plate in the first image exceeds a threshold value, and calculating whether an inclination angle of the carbon slide plate in the first image exceeds a threshold value, and determining that the carbon slide plate is abnormal if the fracture length proportion or the inclination angle exceeds the threshold value; calculating whether a relative position of the pantograph and the catenary in the first image exceeds a threshold value, and determining that an anomaly occurs if the threshold value is exceeded.

3. The pantograph image-based anomaly detection method according to claim 2, characterized in that, the topological dependency relationship is a topological dependency relationship between a base, a bow angle, and a carbon slide plate: if the base is not detected, it is determined that the pantograph is missing as a whole; if the base is detected but the bow angle is not detected, it is determined that the bow angle is missing; if the bow angle is detected but the carbon slide plate is not detected, it is determined that the carbon slide plate is detached. 4.The pantograph image-based anomaly detection method according to claim 1, characterized in that, the calculation of the occlusion area and the uniformity of illumination of the first image, and the exclusion of the corresponding first image if either the occlusion area or the uniformity of illumination does not meet a preset condition includes: segmenting an occlusion object based on a Mask R-CNN model, calculating whether an occlusion area proportion of the occlusion object in the first image exceeds a threshold value, and excluding if the threshold value is exceeded; and calculating whether a variance of a brightness component in an HSV space exceeds a threshold value, and excluding if the threshold value is exceeded.

5. The pantograph image-based anomaly detection method according to claim 1 or 4, characterized by, the calculation of the occlusion area and the uniformity of illumination of the first image, and the exclusion of the corresponding first image if either the occlusion area or the uniformity of illumination does not meet a preset condition further includes: determining whether the sharpness of the first image exceeds a threshold value.

6. A pantograph image-based anomaly detection system, characterized by, a storage and a processor, the storage storing a computer program for implementing the pantograph image-based anomaly detection method according to any one of claims 1-5, and the processor being configured to execute the computer program.

Images