Train pantograph detection method and system

Abstract

This application relates to the field of rail transit, specifically a method and system for detecting a train pantograph. The method includes: a data acquisition step, acquiring a video of the pantograph to be detected; an anomaly detection step, determining reference key points and structural judgment key points of the pantograph from the video; constructing a reference confidence level based on the reference key points; calculating the confidence level and its relative confidence ratio relative to the reference confidence level for each structural judgment key point; determining that the structure of the pantograph is abnormal when the structural judgment key point, its confidence level, and its relative confidence ratio meet anomaly conditions; and a judgment output step, reporting the abnormal state of the pantograph when its structure is abnormal. By using multiple judgments based on confidence level and key points, the method detects whether the pantograph is abnormal, thereby improving detection efficiency and accuracy and solving the problem of insufficient detection accuracy in existing technologies.

Description

Technical Field

[0001] This application relates to the field of rail transit, and in particular to a method and system for detecting a train pantograph. Background Technology

[0002] The pantograph is the core component for a train to obtain external electrical energy, and the stability of its working state directly determines the reliability of the train's power supply and the safety of its operation.

[0003] Train pantograph inspection typically relies on two methods: manual visual inspection and traditional single-sensor inspection. However, with the increasing speed of high-speed trains and bullet trains, and the improvement of maintenance standards, the existing inspection methods are insufficient in terms of accuracy and can no longer meet actual maintenance needs. Summary of the Invention

[0004] This application provides a method and system for detecting a train pantograph, which at least solves the problem of insufficient detection accuracy in related technologies.

[0005] In a first aspect, embodiments of this application provide a method for detecting a train pantograph, including: The data acquisition steps include acquiring video of the pantograph to be tested; The anomaly detection step involves determining the reference key points and structural judgment key points of the pantograph from the video. Based on the aforementioned benchmark key points, a benchmark confidence level is constructed; For each of the aforementioned key structural judgment points, calculate its confidence level and its relative confidence ratio relative to the benchmark confidence level; When the structural judgment key points, their confidence levels, and their relative confidence ratios meet the abnormal conditions, it is determined that the pantograph structure is abnormal. The judgment output step is to report the abnormal state of the pantograph when the structure of the pantograph is abnormal.

[0006] In some embodiments, prior to the anomaly detection step, the detection method further includes: The data filtering step converts each video frame in the video into a corresponding grayscale image; Calculate the variance of the Laplacian operator response of the grayscale image, the mean brightness of the grayscale image, and the dispersion of the brightness distribution of the grayscale image; wherein, the variance is used to characterize the sharpness of the video frame, the dispersion of the image brightness distribution is used to characterize the dynamic range of the video frame, and the mean brightness is used to characterize the exposure level of the video frame. Construct sharpness threshold ranges, brightness feature threshold ranges, and contrast threshold ranges; Determine whether the variance value is within the sharpness threshold range, whether the mean brightness value is within the brightness feature threshold range, and whether the dispersion of the image brightness distribution is within the contrast threshold range. If all of these conditions are met, the video frame is considered to have normal image quality, is retained, and used in the anomaly detection step. Otherwise, the video frame is considered to have abnormal image quality and is not retained.

[0007] In some embodiments, the data filtering step further includes: The video frames with normal image quality are input into the image validity evaluation network, and the image validity confidence score is output. Construct a threshold for image validity; Determine whether the confidence level of the image validity is greater than the image validity threshold. If so, determine the reference key point of the pantograph and the structural judgment key point from the video frame.

[0008] In some embodiments, determining that the pantograph structure is abnormal further includes: The abnormal conditions include a confidence threshold and a relative confidence threshold; Determine whether the confidence level of the key structural judgment point is less than the confidence level judgment threshold, and whether the relative confidence ratio of the key structural judgment point is less than the relative confidence level judgment threshold. If both are true, then the structure of the pantograph corresponding to the key structural judgment point is abnormal; otherwise, the structure of the pantograph corresponding to the key structural judgment point is not abnormal. The judgment output step further includes reporting that when the structure of the pantograph is abnormal, the structure of the pantograph corresponding to the key point of the structure judgment is missing and / or blocked.

[0009] In some embodiments, determining that the pantograph structure is abnormal further includes: If the structure of the pantograph corresponding to the key structural judgment point is not missing and / or obstructed, connect the key structural judgment point to construct multiple structural reference lines; Set the included angle threshold range and the length ratio threshold range; Calculate the angle deviation and length ratio between one structural reference line and the other structural reference line; Determine whether the included angle deviation is within the included angle threshold range and whether the length ratio is within the length ratio threshold range. If both are yes, the pantograph structure is normal; otherwise, the pantograph structure is abnormal. The judgment and output step further includes reporting that the pantograph structure has deflected or deformed. In some embodiments, prior to the determination and output step, the detection method further includes: The arc detection step involves obtaining the ultraviolet intensity of the area where the pantograph is located and converting the ultraviolet intensity into an analog signal. The simulated signal is detected. When the simulated signal continuously exceeds the signal threshold for a preset time, it is determined that the arcing has started. After the arcing has started, when the simulated signal continuously falls below the signal threshold for a preset time, it is determined that the arcing has ended. Record the duration from the start to the end of arcing as the arcing time, and accumulate the number of arcing times within the period; Determine whether the number of arcing times and the arcing time are abnormal. If so, output the arcing abnormality information of the pantograph.

[0010] Secondly, embodiments of this application provide a train pantograph detection system, including: The data acquisition module is configured to acquire video of the pantograph to be tested; An anomaly detection module is configured to determine the reference key points and structural judgment key points of the pantograph from the video. Based on the aforementioned benchmark key points, a benchmark confidence level is constructed; For each of the aforementioned key structural judgment points, calculate its confidence level and its relative confidence ratio relative to the benchmark confidence level; When the structural judgment key points, their confidence levels, and their relative confidence ratios meet the abnormal conditions, it is determined that the pantograph structure is abnormal. The judgment output module is configured to report the abnormal status of the pantograph when the structure of the pantograph is abnormal.

[0011] In some embodiments, it also includes: The data filtering module is configured to convert each video frame in the video into a corresponding grayscale image; Calculate the variance of the Laplacian operator response of the grayscale image, the mean brightness of the grayscale image, and the dispersion of the brightness distribution of the grayscale image; wherein, the variance is used to characterize the sharpness of the video frame, the dispersion of the image brightness distribution is used to characterize the dynamic range of the video frame, and the mean brightness is used to characterize the exposure level of the video frame. Construct sharpness threshold ranges, brightness feature threshold ranges, and contrast threshold ranges; Determine whether the variance value is within the sharpness threshold range, whether the mean brightness value is within the brightness feature threshold range, and whether the dispersion of the image brightness distribution is within the contrast threshold range. If all of these conditions are met, the video frame is considered to have normal image quality, is retained, and used in the anomaly detection step. Otherwise, the video frame is considered to have abnormal image quality and is not retained.

[0012] In some embodiments, the anomaly detection module is further configured to: The abnormal conditions include a confidence threshold and a relative confidence threshold; Determine whether the confidence level of the key structural judgment point is less than the confidence level judgment threshold, and whether the relative confidence ratio of the key structural judgment point is less than the relative confidence level judgment threshold. If both are true, then the structure of the pantograph corresponding to the key structural judgment point is abnormal; otherwise, the structure of the pantograph corresponding to the key structural judgment point is not abnormal. The judgment output module is further configured to report, when the structure of the pantograph is abnormal, that the structure of the pantograph corresponding to the key point of the structure judgment is missing and / or obstructed.

[0013] In some embodiments, it also includes: The arc detection module is configured to acquire the ultraviolet intensity of the area where the pantograph is located and convert the ultraviolet intensity into an analog signal; The simulated signal is detected. When the simulated signal continuously exceeds the signal threshold for a preset time, it is determined that the arcing has started. After the arcing has started, when the simulated signal continuously falls below the signal threshold for a preset time, it is determined that the arcing has ended. Record the duration from the start to the end of arcing as the arcing time, and accumulate the number of arcing times within the cycle; Determine whether the number of arcing times and the arcing time are abnormal. If so, output the arcing abnormality information of the pantograph.

[0014] Compared with related technologies, the train pantograph detection method and system provided in this application construct abnormal conditions for pantograph detection by setting reference key points and structural judgment key points. When the reference key points and structural judgment key points meet the abnormal conditions, the abnormal state of the pantograph can be accurately determined, solving the problem of insufficient detection accuracy in the prior art and achieving the effect of improving detection efficiency and accuracy.

[0015] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description

[0016] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a flowchart of a train pantograph detection method according to an embodiment of this application; Figure 2 This is a structural block diagram of a train pantograph detection system according to an embodiment of this application; Figure 3 This is a schematic diagram of a training method for a neural network model according to an embodiment of this application.

[0017] In the diagram: 201, Data Acquisition Module; 202, Anomaly Detection Module; 203, Judgment Output Module. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the application is described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0019] Obviously, the accompanying drawings described below are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar scenarios based on these drawings without any inventive effort. Furthermore, it is understood that although the efforts made in this development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this application, any changes to design, manufacturing, or production based on the technical content disclosed in this application are merely conventional technical means and should not be construed as insufficient disclosure of the content of this application.

[0020] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment that is mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0021] Unless otherwise defined, the technical or scientific terms used in this application shall have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms “a,” “an,” “an,” “the,” and similar words used in this application do not indicate quantity limitation and may indicate singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units not listed, or may include other steps or units inherent to these processes, methods, products, or devices. The terms “connected,” “linked,” “coupled,” and similar words used in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. “Multiple” used in this application refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following objects are in an "or" relationship. The terms "first," "second," and "third" used in this application are merely to distinguish similar objects and do not represent a specific ordering of the objects.

[0022] The pantograph is the core component for a train to obtain external electrical energy, and the stability of its working state directly determines the reliability of the train's power supply and operational safety. During train operation, the pantograph is constantly in a high-speed, high-frequency lifting state, and is also affected by various factors such as airflow impact, contact wire friction, and environmental erosion. It is prone to abnormalities such as structural damage, missing parts, obstruction, deflection, and deformation. In addition, poor contact between the pantograph and the contact wire can generate arcing. If not detected and handled in time, it may lead to contact wire burnout, pantograph damage, or even serious safety accidents such as power outages and derailments.

[0023] Currently, pantograph inspection on trains mainly relies on two methods: manual visual inspection and traditional single-sensor inspection. Manual visual inspection requires maintenance personnel to observe the train closely after it stops, which is not only inefficient and labor-intensive, but also suffers from strong subjectivity, numerous blind spots, and significant safety hazards, making it unsuitable for the large-scale, high-frequency demands of train maintenance. Traditional single-sensor inspection can only detect one type of anomaly, has a limited detection range, and is susceptible to environmental interference, resulting in high rates of false positives and false negatives, failing to achieve comprehensive, accurate, and automated detection of pantograph anomalies.

[0024] With the increasing speed of high-speed trains and bullet trains, and the raising of maintenance standards, existing detection methods can no longer meet actual maintenance needs. Meanwhile, with the rapid development of computer vision and deep learning technologies, computer-based intelligent monitoring has been gradually applied to various fields, and related supporting technologies are maturing. For the complex and ever-changing environmental interference and highly repetitive detection tasks in the monitoring of anomalies in train pantograph structures, deep learning's high robustness to image processing and its highly adaptable feature extraction capabilities are highly suitable. Therefore, the demand for applying deep learning computer vision technology to high-speed rail equipment inspection solutions is growing.

[0025] To address the aforementioned issues, this application provides a method and system for detecting train pantographs, enabling comprehensive detection of various types of pantograph anomalies, improving automation and detection accuracy, making it more adaptable to complex environments, and ensuring the safe and stable operation of trains.

[0026] like Figure 1 As shown in the figure, this application provides a method for detecting a train pantograph, including: Data acquisition step S101: Acquire video of the pantograph to be tested.

[0027] In the anomaly detection step S102, the reference key points and structural judgment key points of the pantograph are determined from the video. The reference key points are used to characterize the basic structural state of the pantograph, and the structural judgment key points are used to characterize the local structural state of the pantograph.

[0028] Specifically, the pantograph position is detected by a target detection neural network model to obtain the pantograph's position coordinates. Then, a key point detection neural network model is used to output the coordinates and confidence levels of the key points for structural judgment of the pantograph and the reference key points.

[0029] The benchmark key points include connection key points that are structurally stable and easily and reliably detectable under normal operating conditions, such as base connection points and bow support points.

[0030] Key structural assessment points include local critical points that are prone to loss, obstruction, or breakage and are sensitive to operational safety, such as the left and right bow angles and the ends of the carbon slide plate.

[0031] Based on the baseline key points, a baseline confidence level is constructed. The baseline confidence level is used to characterize the network's overall response to a normal structure under the current video conditions.

[0032] Specifically, in the output of the key point detection neural network, each key point corresponds to a confidence value. This confidence value is used to characterize the credibility of the network's localization result for the key point. It is calculated by the output layer of the neural network through a probability mapping function and is used to represent the probability of the key point's existence or the reliability of its localization in the current image.

[0033] In practical implementation, the confidence level can be calculated from the response value at the corresponding location in the keypoint heatmap, or directly represented by the probability value output by the network. Those skilled in the art can directly obtain this confidence level value based on the selected keypoint detection network structure (e.g., HRNet, Hourglass, or OpenPose), which is a conventional technique in this field.

[0034] Under normal operating conditions where the pantograph structure is intact and unobstructed, the reference key points usually have a stable and high confidence distribution. For example, the confidence of stable parts such as the base connection point and the pantograph support point is usually higher than that of the structural edge or easily obstructed parts.

[0035] Therefore, this application selects several structurally stable key points as benchmark key points and statistically analyzes the confidence scores of these benchmark key points to construct a benchmark confidence level. For example, the following methods can be used: calculate the median confidence score of all benchmark key points as the benchmark confidence level; or sort the confidence scores, remove the highest and lowest values, and then calculate the truncated mean; or perform a weighted average calculation on the confidence score set.

[0036] The above methods can reduce the impact of individual key point detection anomalies or partial occlusions on the overall judgment, thereby obtaining a baseline confidence level that reflects the overall reliability of key point detection under the current image conditions.

[0037] Overall image darkening or slight blurring can cause a simultaneous decrease in the absolute confidence of all key points. Using a fixed absolute threshold to determine structural anomalies in such cases can easily lead to false positives. This application uses a calculated baseline confidence level as a normalization reference to calculate the relative confidence ratio of subsequent structural anomaly detection key points. This eliminates the impact of overall image quality on the detection results and improves the accuracy of structural anomaly detection.

[0038] For each key point in the structural judgment, calculate its confidence level and its relative confidence ratio with respect to the baseline confidence level.

[0039] When the key points of the structure, their confidence levels, and their relative confidence ratios meet the abnormal conditions, it is determined that the pantograph structure is abnormal.

[0040] In the judgment output step S103, when the structure of the pantograph is abnormal, the abnormal state of the pantograph is reported.

[0041] This application first acquires video footage of the pantograph to be tested, accurately extracts benchmark key points and structural judgment key points from the video, constructs a unified benchmark confidence level using the benchmark key points as a reference, and then calculates the confidence level of each structural judgment key point, as well as the relative confidence ratio of that confidence level to the benchmark confidence level. By pre-setting abnormal conditions, a comprehensive judgment is made on the structural judgment key point itself, its confidence level, and its relative confidence ratio. When all three meet the abnormal conditions, an anomaly in the pantograph structure can be determined. The dual judgment logic of benchmark confidence level and relative confidence ratio, compared to a single confidence level judgment, can effectively filter out temporary fluctuations in the confidence level of key points caused by interference factors such as external light fluctuations and slight dust obstruction, thereby avoiding misjudgments and ensuring the accuracy of anomaly detection. Simultaneously, timely reporting of abnormal states allows maintenance personnel to grasp the fault situation immediately, quickly carry out repair work, thereby reducing the risk of pantograph fault escalation and ensuring the stability of train power supply and operational safety.

[0042] Since the entire detection process requires no human intervention, it achieves automated detection through video analysis and key point quantification calculation, eliminating the subjectivity, inefficiency, and safety hazards of close-range observation in manual visual inspection, effectively reducing the cost of manual inspection and significantly improving detection efficiency.

[0043] In some embodiments, prior to the anomaly detection step S102, the detection method further includes: The data filtering step converts each video frame in the video into a corresponding grayscale image.

[0044] The variance of the Laplacian operator response of the grayscale image, the mean brightness of the grayscale image, and the dispersion of the brightness distribution of the grayscale image are calculated. The variance characterizes the sharpness of the video frame, the dispersion of the image brightness distribution characterizes the dynamic range of the video frame, and the mean brightness characterizes the exposure level of the video frame.

[0045] The dispersion of image brightness distribution includes brightness standard deviation or quantile difference.

[0046] Construct sharpness threshold range, brightness feature threshold range, and contrast threshold range.

[0047] Specifically, in order to adapt the detection process to different lines, different cameras, and different lighting conditions, this application does not use a fixed threshold. Instead, it constructs a statistical threshold within a set time reference window and obtains historical video frames within the window.

[0048] The sharpness threshold range, luminance feature threshold range, and contrast threshold range are all adaptively determined based on the statistical characteristics of historical video frames. Specifically, the sharpness feature, luminance feature, and contrast feature corresponding to historical video frames are statistically analyzed, and the threshold range for each feature used in subsequent judgments is determined based on preset quantile parameters. Specifically, the lower limit of the sharpness threshold range and the contrast threshold range are respectively taken from the values ​​corresponding to the low quantile parameters of the corresponding features, representing the robust lower limit of image quality under the current environmental conditions. The lower and upper limits of the luminance feature threshold range are determined by the values ​​corresponding to the low and high quantile parameters of the luminance feature, respectively.

[0049] The quantile parameters are all adjustable statistical parameters, and their specific values ​​can be determined experimentally based on the installation location of the video shooting equipment, the shooting angle, and the ambient lighting conditions. Specifically, the low quantile parameter can be set to 0.2, and the high quantile parameter can be set to 0.9, which is used to exclude the interference of a small number of abnormal frames on the threshold determination while ensuring robustness.

[0050] The system determines whether the variance value, the mean brightness value, and the dispersion of the image brightness distribution are within the sharpness threshold range, and whether they are within the contrast threshold range. If all three are within the specified range, the video frame is considered to have normal image quality, is retained, and used in the anomaly detection step S102. Otherwise, the video frame is considered to have abnormal image quality and is not retained.

[0051] In this application, before anomaly detection, the video of the pantograph to be detected is first screened. Each video frame is converted into a grayscale image. By calculating the variance of the Laplacian operator response, the mean brightness, and the dispersion of brightness distribution of the grayscale image, the clarity, exposure level, and dynamic range of the video frame are quantified, respectively. Reasonable threshold ranges for clarity, brightness, and contrast are pre-constructed. The calculated three indicators are compared with their corresponding threshold ranges. Only when all three indicators are within their respective threshold ranges is the video frame considered to have normal image quality, retained, and used for subsequent anomaly detection; otherwise, it is considered to have abnormal image quality and discarded. Strict screening of video frame quality through quantitative indicators effectively removes poor-quality video frames such as blurry, overexposed, underexposed, and insufficient contrast. Poor-quality video frames often lead to errors in key point localization and confidence calculation deviations, resulting in misjudgments and missed detections. Therefore, the screened data significantly improves the accuracy and reliability of anomaly detection.

[0052] Meanwhile, after removing invalid and inferior video frames, there is no need to perform subsequent processing on such data. This can effectively reduce the amount of invalid data to be processed, reduce computing power consumption, improve overall detection efficiency, and ensure that qualified video frames can still be screened for detection in complex environments such as rain, night, and strong light, thus ensuring the stability of system operation.

[0053] In addition, a unified image quality screening standard can provide a consistent data foundation for subsequent key point localization, confidence calculation and anomaly detection, avoiding fluctuations in detection results due to inconsistent data quality and improving the repeatability and stability of detection results.

[0054] In some embodiments, the data filtering step further includes: Input video frames with normal image quality into the image validity evaluation network, and output the image validity confidence score.

[0055] The image validity evaluation network can use a lightweight convolutional neural network or a MobileNet-like network structure to identify image anomalies that are difficult to capture by statistical measures alone, such as raindrop occlusion, strong reflections, lens contamination, and severe shaking.

[0056] Establish a frame validity threshold. The frame validity threshold can be set according to actual needs.

[0057] Determine whether the confidence level of the image validity is greater than the image validity threshold. If so, determine the reference key points and structural judgment key points of the pantograph from the video frame.

[0058] While normal image quality ensures clear video frames and proper exposure, it doesn't guarantee the complete pantograph within the frame. Using video frames containing only a portion of the pantograph, or those completely obscured by foreign objects, for detection can lead to incomplete key point localization and judgment bias. Therefore, in this application, after screening video frames based on image quality and retaining those with normal quality, these frames are further input into an image validity evaluation network. The network analyzes and outputs the validity confidence score for each frame, which is then compared with a preset image validity threshold. Only when the image validity confidence score is greater than the preset threshold is the video frame considered to contain the complete pantograph without severe obstruction, suitable for anomaly detection. The baseline key points and structural judgment key points of the pantograph are then determined from this video frame. Otherwise, the video frame is deemed unusable for anomaly detection and discarded. This further screening through image validity evaluation effectively eliminates video frames with acceptable image quality but unsuitable for anomaly detection, further improving the validity of the detection data. With an increased proportion of valid data, interference from invalid images can be avoided in the detection results, reducing the probability of false positives and false negatives. Simultaneously, the processing of invalid images is reduced, lowering computational power consumption and improving overall operational efficiency. Furthermore, this screening process can effectively handle invalid images in complex scenarios such as obstruction by foreign objects and deviations in shooting angles, ensuring continuous and stable detection and adapting to complex train maintenance scenarios.

[0059] In some embodiments, determining that the pantograph structure is abnormal further includes: Abnormal conditions include confidence threshold and relative confidence threshold.

[0060] If the confidence level of the key point of structural judgment is less than the confidence level threshold, and the relative confidence ratio of the key point of structural judgment is less than the relative confidence ratio threshold, then the structure of the pantograph corresponding to the key point of structural judgment is abnormal. Otherwise, the structure of the pantograph corresponding to the key point of structural judgment is not abnormal.

[0061] The judgment output step S103 further includes reporting that when the structure of the pantograph is abnormal, the structure of the pantograph corresponding to the judgment key point of the structure is missing and / or blocked.

[0062] The confidence threshold and relative confidence threshold can be determined through experiments with labeled data and actual deployment scenarios.

[0063] This application clarifies the judgment conditions for pantograph structural anomalies. During the detection process, for each key structural judgment point, it simultaneously determines whether its confidence level is less than the confidence level judgment threshold and whether its relative confidence ratio is less than the relative confidence ratio judgment threshold. Only when both judgment conditions are met is it determined that the pantograph structure corresponding to the key structural judgment point is missing and / or obstructed. If one condition is not met, it is determined that the structure at that location is not missing and / or obstructed. Compared with the existing technology where single confidence level judgment is easily misjudged due to slight external interference, the relative confidence ratio of this application can reflect the difference in confidence level between the key structural judgment point and the reference key point, making up for the limitations of single confidence level judgment. Through dual threshold constraints, it can accurately distinguish between key point positioning deviation and actual structural anomalies, avoiding misjudging temporary decreases in confidence caused by external interference as structural anomalies. Meanwhile, this precise judgment logic can clearly identify the anomaly type as structural missing and / or occlusion, and can correspond to specific key points of structural judgment, providing operation and maintenance personnel with accurate fault prompts, helping them to quickly locate the fault location and formulate targeted repair plans. This not only improves the rigor of anomaly judgment and reduces the misjudgment rate, but also reduces unnecessary repair costs and improves repair efficiency.

[0064] In some embodiments, determining that the pantograph structure is abnormal further includes: If the structure of the pantograph corresponding to the key structural judgment point is not missing or / and obstructed, connect the key structural judgment points to construct multiple structural reference lines.

[0065] Specifically, based on the coordinates of key points determined by the structure, structural reference lines are constructed. These structural reference lines include, but are not limited to: base connection lines, bow support lines, and carbon skateboard end connection lines.

[0066] Set the included angle threshold range and the length ratio threshold range. The included angle threshold range and the length ratio threshold range can be determined based on historical data.

[0067] Calculate the angle deviation and length ratio between one structural reference line and another structural reference line.

[0068] Determine whether the included angle deviation is within the included angle threshold range and whether the length ratio is within the length ratio threshold range. If both are true, the pantograph structure is normal; otherwise, the pantograph structure is abnormal.

[0069] The judgment output step S103 further includes reporting that the pantograph structure has deflected or deformed.

[0070] This application, under the premise of confirming that the pantograph structure corresponding to the key structural judgment points is not missing and / or obstructed, connects the key structural judgment points to construct multiple structural reference lines that can reflect the structural shape of the pantograph. Then, it presets reasonable angle threshold ranges and length ratio threshold ranges, calculates the angle deviation between any two structural reference lines and the length ratio of different structural reference lines, and judges whether the pantograph structure is normal by determining whether the angle deviation is within the preset angle threshold range and whether the length ratio is within the preset length ratio threshold range. If any indicator exceeds the threshold range, the structure is judged to be abnormal.

[0071] Specifically, for the line connecting the carbon skateboard end point and the line connecting the base, the included angle deviation between the two is calculated to reflect the overall deflection or torsion. The length ratio between the two is calculated to reflect the dimensional changes caused by overall stretching, compression, or abnormal bending.

[0072] The same judgment method can be used for the line connecting the ends of the carbon skateboard and the support line of the bow.

[0073] Latent anomalies such as pantograph deflection and deformation do not lead to missing or decreased confidence in structural assessment points, making them unrecognizable through single-point detection. This application addresses this issue by constructing a structural reference line by connecting key points and utilizing the inherent geometric characteristics of the pantograph to quantify its spatial attitude, thereby accurately identifying these latent anomalies. This overcomes the limitations of single-point detection and improves the comprehensiveness of anomaly detection. Furthermore, through the quantitative calculation of angle deviation and length ratio, normal structural installation deviations can be accurately distinguished from abnormal structural deflections and deformations, avoiding misjudgments. The calculated angle deviation and length ratio data provide maintenance personnel with detailed anomaly parameters, helping them accurately assess the severity of faults, develop more targeted repair plans, and effectively shorten repair time.

[0074] In some embodiments, before determining the output step S103, the detection method further includes: The arc detection process involves obtaining the ultraviolet intensity in the area where the pantograph is located and converting the ultraviolet intensity into an analog signal.

[0075] The system detects a simulated signal. When the simulated signal continuously exceeds a signal threshold for a preset time, it determines that arcing has begun. After arcing begins, when the simulated signal continuously falls below the signal threshold for a preset time, it determines that arcing has ended.

[0076] Record the duration from the start to the end of arcing as the arcing time, and accumulate the number of arcing times within the period.

[0077] Determine if the number of arcing times and the arcing time are abnormal. If so, output the abnormal arcing information of the pantograph.

[0078] Before performing pantograph structural anomaly detection and outputting results, this application adds an arcing detection step. Ultraviolet (UV) intensity in the pantograph's location area is acquired using a UV sensor. This UV intensity signal is converted into an analog signal that is easy to detect and analyze. By detecting fluctuations in the analog signal, a preset time threshold is set. When the analog signal continuously exceeds the threshold for the preset time, arcing is determined to have started. After arcing begins, when the analog signal continuously falls below the threshold for the preset time, arcing is determined to have ended. The duration from the start to the end of arcing is recorded as the arcing time, and the number of arcing events is accumulated within a set period. Finally, based on preset anomaly criteria, the number of arcing events and the arcing time are judged to be abnormal. If abnormal, arcing anomaly information is output.

[0079] Arcing is a typical characteristic of poor contact between the pantograph and the overhead contact line, and arcing emits ultraviolet light of a specific wavelength. This application can accurately capture arcing signals through ultraviolet intensity detection, enabling real-time detection of arcing anomalies. This overcomes the limitations of existing structural anomaly detection methods that cannot cover poor contact faults, helping maintenance personnel quickly identify contact problems between the pantograph and the overhead contact line. This prevents the contact line from burning out, the pantograph from being damaged, or even power outages caused by the continued escalation of arcing, significantly improving train operation safety. Simultaneously, the recorded parameters such as arcing time and number of arcs provide accurate data support for maintenance personnel to assess the contact status between the pantograph and the overhead contact line, facilitating early detection of potential contact problems and enabling preventative maintenance. When combined with structural anomaly detection, it forms a comprehensive pantograph detection system, improving the practicality and completeness of the detection system and further reducing the risk of failure.

[0080] In some embodiments, it is determined whether alarm triggering conditions such as screen dirt, pantograph loss, pantograph structural abnormality, foreign object intrusion, and arcing abnormality are met, and the triggered alarm is reported to the connected client via TCP message.

[0081] After the initial structural anomaly judgment is completed based on a single frame image, in order to reduce the impact of instantaneous occlusion, occasional detection jitter or false detection in a single frame on the final alarm, this application performs time consistency verification on the results of consecutive frames.

[0082] Specifically, a time window of preset length is set, such as 15-30 frames, and the frequency of the same anomaly type is weighted and accumulated within the window. The weighting method can be set to give higher weight to the most recent frame to enhance real-time performance. When the weighted and accumulated result exceeds the preset value, the corresponding abnormal state of the pantograph structure is output, and an anomaly alarm is triggered. This alarm is then reported to the client connected to the train via a Transmission Control Protocol (TCP) message.

[0083] Furthermore, trigger and deactivation thresholds are set to form a hysteresis interval. When an abnormal alarm is triggered, the trigger frame and several frames before and after it can be saved as evidence of the abnormality. Meanwhile, during the alarm's duration, the thresholds used for anomaly detection are not updated.

[0084] In some of these embodiments, such as Figure 3 As shown, this application also includes a training method for the neural network model used in the anomaly detection step S102, comprising: acquiring pantograph videos under various conditions, selecting a portion of the videos and extracting frames to obtain training images; generating abnormal images using a controllable generation model for various abnormal conditions such as structural anomalies, foreign object intrusion, and image contamination, and combining the generated abnormal images with existing normal images to form a balanced positive and negative sample set; labeling the training samples for tasks such as image classification, object detection, keypoint detection, and image segmentation; and fine-tuning the pre-trained general neural network for tasks such as image classification, object detection, keypoint detection, and image segmentation using deep learning methods to obtain the network model.

[0085] The image classification task uses neural networks based on image classification neural network frameworks such as MobileNet, ResNet, or VGG to evaluate image quality and classify images into normal images, dirty and dark images, overexposed images, complex backgrounds, and light interference.

[0086] MobileNet is a mobile network, specifically a convolutional neural network in deep learning.

[0087] Residual Network (ResNet) is a type of convolutional neural network used in deep learning.

[0088] VGG is a CNN model for image classification in deep learning.

[0089] The target detection task uses neural networks based on target detection neural network frameworks such as Faster RCNN, YOLO, or SSD to determine whether a pantograph exists in the image and to detect the position of the pantograph.

[0090] Faster Region-based CNN (Faster RCNN), You Only Look Once (YOLO), and SingleShot MultiBox Detector (SSD) are all commonly used object detection algorithms.

[0091] The keypoint detection task employs neural networks based on keypoint detection neural network frameworks such as HRNet, Hourglass, or OpenPose to detect the location of key points on the pantograph structure and determine whether there is deformation or structural loss. The locations of key points for structural judgment include the left and right bow angles, the left and right support points of the carbon slide plate, the left and right structural points of the carbon slide plate, the left and right support points of the bow body, and the left and right connection points of the bow body support.

[0092] High-Resolution Net (HRNet), Hourglass, and OpenPose are all commonly used models in pose estimation.

[0093] The image segmentation task uses neural networks based on image segmentation neural network frameworks such as Unet, SegNet, or RefineNet to segment the pantograph area in the image and determine whether there are foreign objects obstructing it.

[0094] UNet, SegNet, and RefineNet are all classic models for image semantic segmentation.

[0095] Based on the trained network model, the model is tested using actual videos of corresponding anomalies to determine its effectiveness. For models that do not meet the expected performance, adjustments are made to the sample distribution and model structure, and the model is retrained to obtain the optimal neural network model that meets the performance requirements. Based on the optimal neural network model, the RKNN tool is used for quantization deployment, and the input and output formats are synchronized to obtain the neural network model used in the anomaly detection step S102.

[0096] like Figure 2 As shown in the figure, this application embodiment also provides a train pantograph detection system, including: The data acquisition module 201 is configured to acquire video of the pantograph to be tested.

[0097] The anomaly detection module 202 is configured to determine the reference key points and structural judgment key points of the pantograph from the video.

[0098] Based on the benchmark key points, a benchmark confidence level is constructed.

[0099] For each key point in the structural judgment, calculate its confidence level and its relative confidence ratio with respect to the baseline confidence level.

[0100] When the key points of the structure, their confidence levels, and their relative confidence ratios meet the abnormal conditions, it is determined that the pantograph structure is abnormal.

[0101] The judgment output module 203 is configured to report the abnormal status of the pantograph when the structure of the pantograph is abnormal.

[0102] A modular train pantograph detection system was constructed, comprising a data acquisition module 201, an anomaly detection module 202, and a judgment output module 203. Each module has a clear division of labor and works collaboratively. The core principle is that through modular design, the detection method is implemented as a practically applicable automated system, achieving standardization and automation of the detection process, significantly improving maintenance efficiency while reducing manual labor costs and safety hazards. The modular architecture also facilitates system maintenance, upgrades, and functional expansion. In the future, the parameters of each module can be adjusted or new functional modules added according to the detection needs of different types of pantographs, improving the system's versatility. Furthermore, the anomaly reporting function of the judgment output module 203 ensures that maintenance personnel receive fault information promptly, respond quickly, and handle issues in a timely manner, effectively guaranteeing stable train power supply and operational safety, making it suitable for large-scale, high-frequency train maintenance scenarios.

[0103] In some embodiments, it also includes: The data filtering module is configured to convert each video frame in the video into a corresponding grayscale image.

[0104] The variance of the Laplacian operator response of the grayscale image, the mean brightness of the grayscale image, and the dispersion of the brightness distribution of the grayscale image are calculated. The variance characterizes the sharpness of the video frame, the dispersion of the image brightness distribution characterizes the dynamic range of the video frame, and the mean brightness characterizes the exposure level of the video frame. Construct sharpness threshold range, brightness feature threshold range, and contrast threshold range.

[0105] The system determines whether the variance value, the mean brightness value, and the dispersion of the image brightness distribution are within the sharpness threshold range, and whether they are within the contrast threshold range. If all three are within the specified range, the video frame is considered to have normal image quality, is retained, and used in the anomaly detection step S102. Otherwise, the video frame is considered to have abnormal image quality and is not retained.

[0106] This application adds a data filtering module and clarifies its core functions to ensure the validity of the data input to the anomaly detection module 202. The data filtering module preprocesses the video data, removing blurry, overexposed, and other low-quality video frames. These low-quality frames can cause errors in key point localization and confidence calculation in the anomaly detection module 202, leading to misjudgments and missed detections. Therefore, it significantly improves the accuracy and reliability of the system's detection, ensuring the efficient operation of the anomaly detection module 202. Furthermore, by removing invalid data, the anomaly detection module 202 does not need to process low-quality video frames, reducing system computational power consumption, increasing system operating speed, and meeting the real-time detection requirements of high-speed trains. In addition, the data filtering module can adapt to complex environments such as rain, nighttime, and strong light, filtering out qualified video frames, effectively enhancing the system's environmental adaptability, ensuring stable operation in complex scenarios, and expanding the system's application scope.

[0107] In some embodiments, the anomaly detection module 202 is further configured to: Abnormal conditions include confidence threshold and relative confidence threshold.

[0108] If the confidence level of the key point of structural judgment is less than the confidence level threshold, and the relative confidence ratio of the key point of structural judgment is less than the relative confidence ratio threshold, then the structure of the pantograph corresponding to the key point of structural judgment is abnormal. Otherwise, the structure of the pantograph corresponding to the key point of structural judgment is not abnormal.

[0109] The judgment output module 203 is further configured to report that the pantograph structure is missing and / or blocked when the pantograph structure is abnormal.

[0110] The specific logic of the anomaly detection module 202 in determining pantograph structural anomalies is clarified, improving the accuracy and targeting of the module's detection. By refining the function of the anomaly detection module 202, the system can accurately identify pantograph structural defects and obstructions, avoiding misjudgments caused by external interference and improving the reliability of the system's detection results.

[0111] In some embodiments, it also includes: The arc detection module is configured to acquire the ultraviolet intensity of the area where the pantograph is located and convert the ultraviolet intensity into an analog signal.

[0112] The system detects a simulated signal. When the simulated signal continuously exceeds a signal threshold for a preset time, it determines that arcing has begun. After arcing begins, when the simulated signal continuously falls below the signal threshold for a preset time, it determines that arcing has ended.

[0113] Record the duration from the start to the end of arcing as the arcing time, and accumulate the number of arcing times within the cycle.

[0114] Determine if the number of arcing times and the arcing time are abnormal. If so, output the abnormal arcing information of the pantograph.

[0115] The newly added arc detection module overcomes the limitations of the original system, which could only detect structural anomalies. It enables real-time detection of pantograph arcing anomalies, promptly identifying poor contact between the pantograph and the overhead contact line, preventing equipment damage and power outages caused by arcing, and significantly improving train operation safety. Simultaneously, the module records parameters such as arcing time and number of arcs, providing maintenance personnel with accurate data to assess the contact status between the pantograph and the overhead contact line, facilitating early detection of potential contact problems and enabling preventative maintenance.

[0116] With the rapid development of domestically produced intelligent chips in recent years, the industry's demand for deep learning models deployed on embedded devices is increasing, compared to server-based intelligent analysis systems. Systems deployed on embedded devices can directly connect to the backend of sensors, significantly reducing communication costs and improving processing timeliness. However, existing deep learning-based detection methods, when combined with embedded devices, still suffer from technical drawbacks such as insufficient detection accuracy, low operating efficiency, and high false alarm and false negative rates in abnormal situations.

[0117] To overcome the above problems, in this application, the data acquisition module 201 of the train pantograph detection system acquires video of the pantograph to be tested through a monocular camera and stores the video through a storage hard disk. The anomaly detection module 202 and the arc judgment module of the train pantograph detection system rely on an embedded application processor chip. The embedded application processor chip is directly connected to the storage hard disk, realizing one-stop embedded device deployment and preventing equipment redundancy and additional information transmission costs.

[0118] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0119] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for detecting a train pantograph, characterized in that, include: The data acquisition steps include acquiring video of the pantograph to be tested; The anomaly detection step involves determining the reference key points and structural judgment key points of the pantograph from the video. Based on the aforementioned benchmark key points, a benchmark confidence level is constructed; For each of the aforementioned key structural judgment points, calculate its confidence level and its relative confidence ratio relative to the benchmark confidence level; When the structural judgment key points, their confidence levels, and their relative confidence ratios meet the abnormal conditions, it is determined that the pantograph structure is abnormal. The judgment output step is to report the abnormal state of the pantograph when the structure of the pantograph is abnormal.

2. The train pantograph detection method according to claim 1, characterized in that, Prior to the anomaly detection step, the detection method further includes: The data filtering step converts each video frame in the video into a corresponding grayscale image; Calculate the variance of the Laplacian operator response of the grayscale image, the mean brightness of the grayscale image, and the dispersion of the brightness distribution of the grayscale image; wherein, the variance is used to characterize the sharpness of the video frame, the dispersion of the image brightness distribution is used to characterize the dynamic range of the video frame, and the mean brightness is used to characterize the exposure level of the video frame. Construct sharpness threshold ranges, brightness feature threshold ranges, and contrast threshold ranges; Determine whether the variance value is within the sharpness threshold range, whether the mean brightness value is within the brightness feature threshold range, and whether the dispersion of the image brightness distribution is within the contrast threshold range. If all of these conditions are met, the video frame is considered to have normal image quality, is retained, and used in the anomaly detection step. Otherwise, the video frame is considered to have abnormal image quality and is not retained.

3. The train pantograph detection method according to claim 2, characterized in that, The data filtering step further includes: The video frames with normal image quality are input into the image validity evaluation network, and the image validity confidence score is output. Construct a threshold for image validity; Determine whether the confidence level of the image validity is greater than the image validity threshold. If so, determine the reference key point of the pantograph and the structural judgment key point from the video frame.

4. The train pantograph detection method according to claim 1, characterized in that, Determining that the pantograph has a structural abnormality further includes: The abnormal conditions include a confidence threshold and a relative confidence threshold; Determine whether the confidence level of the key structural judgment point is less than the confidence level judgment threshold, and whether the relative confidence ratio of the key structural judgment point is less than the relative confidence level judgment threshold. If both are true, then the structure of the pantograph corresponding to the key structural judgment point is abnormal; otherwise, the structure of the pantograph corresponding to the key structural judgment point is not abnormal. The judgment output step further includes reporting that when the structure of the pantograph is abnormal, the structure of the pantograph corresponding to the key point of the structure judgment is missing and / or blocked.

5. The train pantograph detection method according to claim 4, characterized in that, Determining that the pantograph has a structural abnormality further includes: If the structure of the pantograph corresponding to the key structural judgment point is not missing and / or obstructed, connect the key structural judgment point to construct multiple structural reference lines; Set the included angle threshold range and the length ratio threshold range; Calculate the angle deviation and length ratio between one structural reference line and the other structural reference line; Determine whether the included angle deviation is within the included angle threshold range and whether the length ratio is within the length ratio threshold range. If both are yes, the pantograph structure is normal; otherwise, the pantograph structure is abnormal. The judgment output step further includes reporting that the structure of the pantograph has deflected or deformed.

6. The method for detecting a train pantograph according to claim 1, characterized in that, Before the determination and output step, the detection method further includes: The arc detection step involves obtaining the ultraviolet intensity of the area where the pantograph is located and converting the ultraviolet intensity into an analog signal. The simulated signal is detected. When the simulated signal continuously exceeds the signal threshold for a preset time, it is determined that the arcing has started. After the arcing has started, when the simulated signal continuously falls below the signal threshold for a preset time, it is determined that the arcing has ended. Record the duration from the start to the end of arcing as the arcing time, and accumulate the number of arcing times within the period; Determine whether the number of arcing times and the arcing time are abnormal. If so, output the arcing abnormality information of the pantograph.

7. A train pantograph detection system, characterized in that, include: The data acquisition module is configured to acquire video of the pantograph to be tested; An anomaly detection module is configured to determine the reference key points and structural judgment key points of the pantograph from the video. Based on the aforementioned benchmark key points, a benchmark confidence level is constructed; For each of the aforementioned key structural judgment points, calculate its confidence level and its relative confidence ratio relative to the benchmark confidence level; When the structural judgment key points, their confidence levels, and their relative confidence ratios meet the abnormal conditions, it is determined that the pantograph structure is abnormal. The judgment output module is configured to report the abnormal status of the pantograph when the structure of the pantograph is abnormal.

8. The train pantograph detection system according to claim 7, characterized in that, Also includes: The data filtering module is configured to convert each video frame in the video into a corresponding grayscale image; Calculate the variance of the Laplacian operator response of the grayscale image, the mean brightness of the grayscale image, and the dispersion of the brightness distribution of the grayscale image; wherein, the variance is used to characterize the sharpness of the video frame, the dispersion of the image brightness distribution is used to characterize the dynamic range of the video frame, and the mean brightness is used to characterize the exposure level of the video frame. Construct sharpness threshold ranges, brightness feature threshold ranges, and contrast threshold ranges; Determine whether the variance value is within the sharpness threshold range, whether the mean brightness value is within the brightness feature threshold range, and whether the dispersion of the image brightness distribution is within the contrast threshold range. If all of these conditions are met, the video frame is considered to have normal image quality, is retained, and used in the anomaly detection step. Otherwise, the video frame is considered to have abnormal image quality and is not retained.

9. The train pantograph detection system according to claim 7, characterized in that, The anomaly detection module is further configured to: The abnormal conditions include a confidence threshold and a relative confidence threshold; Determine whether the confidence level of the key structural judgment point is less than the confidence level judgment threshold, and whether the relative confidence ratio of the key structural judgment point is less than the relative confidence level judgment threshold. If both are true, then the structure of the pantograph corresponding to the key structural judgment point is abnormal; otherwise, the structure of the pantograph corresponding to the key structural judgment point is not abnormal. The judgment output module is further configured to report, when the structure of the pantograph is abnormal, that the structure of the pantograph corresponding to the key point of the structure judgment is missing and / or obstructed.

10. The train pantograph detection system according to claim 7, characterized in that, Also includes: The arc detection module is configured to acquire the ultraviolet intensity of the area where the pantograph is located and convert the ultraviolet intensity into an analog signal; The simulated signal is detected. When the simulated signal continuously exceeds the signal threshold for a preset time, it is determined that the arcing has started. After the arcing has started, when the simulated signal continuously falls below the signal threshold for a preset time, it is determined that the arcing has ended. Record the duration from the start to the end of arcing as the arcing time, and accumulate the number of arcing times within the cycle; Determine whether the number of arcing times and the arcing time are abnormal. If so, output the arcing abnormality information of the pantograph.

Images