Pantograph structure abnormity real-time detection method and system

Abstract

The invention discloses a pantograph structure abnormity real-time detection method and system, and the method comprises the steps: obtaining a pantograph image in a video stream of a bullet train in real time, and inputting a pantograph target detection model; performing target detection on the pantograph by adopting a target detection algorithm, and if an abnormal state occurs, triggering an alarm; if the abnormal state does not occur, performing picture cutting on the pantograph region bounding box to obtain a pantograph body region image; inputting the pantograph body area image into a pantograph key point detection model; and detecting the structure of the pantograph by adopting a key point detection algorithm, and if the structure is abnormal, triggering an alarm. According to the method and the system, target detection is carried out on the pantograph body and the abnormal state, and a rapid and accurate result is provided for detection and attitude matching of the pantograph key point.

Description

Technical Field

[0001] The present invention relates to the field of rail transit technology, and in particular to a real-time detection method and system for pantograph structural abnormality. Background Art

[0002] The pantograph is a key component for electric locomotives and EMUs to obtain electrical energy through the contact network. Its normal operation is directly related to the safe operation of the train.

[0003] Pantograph structural abnormality refers to changes in the rigid structure of the pantograph, which affects the normal operation of the pantograph. Common pantograph structural abnormalities include flipping and deformation of the carbon slide plate, tilting of the pantograph, broken but not falling off of the left and right pantograph corners, and falling off of the left and right pantograph corners. Since there are many cases of pantograph structural abnormalities, and the scene switches frequently during the operation of the EMU, and the background interference is large, it is difficult to detect the structural abnormalities of the pantograph body in a timely and effective manner.

[0004] Existing methods for detecting pantograph structural anomalies include regular parking inspection and replacement, manual video monitoring, online detection using computer vision methods such as traditional image processing, and detection of pantograph structural anomalies using deep learning methods. The first two methods require manual inspection, which is inefficient, has high false alarms, and high labor costs. The use of computer vision methods can alleviate the disadvantages of manual inspection to a certain extent, but it requires more computing resources, requires the installation of specific computing servers, and is time-consuming, with low detection rates and high false alarms. Once proposed, deep learning methods have achieved great success in various industries, and there have been studies on their use in detecting pantograph structural anomalies, but they are generally ordinary applications that use a method of learning first and then matching, or simply use deep learning to locate the position of the pantograph, and then use computer vision methods to match the fault type, without fully utilizing the huge potential of deep learning.

[0005] Therefore, it is of great practical significance to construct a high-quality real-time detection method for pantograph structural anomalies. Summary of the invention

[0006] In order to solve the above technical problems, the present invention proposes a real-time detection method and system for pantograph structural abnormality by means of the idea of ​​human body key point recognition.

[0007] In a first aspect, an embodiment of the present invention provides a method for real-time detection of pantograph structural abnormality, comprising: Obtain pantograph images in real time from the video stream of the train and input them into the pantograph target detection model; Use target detection algorithm to detect the pantograph, and trigger an alarm if an abnormal state occurs; If no abnormal state occurs, the pantograph area boundary frame is cropped to obtain the pantograph body area image; Inputting the pantograph body area image into the pantograph key point detection model; The key point detection algorithm is used to detect the structure of the pantograph, and if any structural abnormality is found, an alarm is triggered.

[0008] Furthermore, constructing a pantograph target detection model includes: Obtain pantograph images from the video stream of the train and create a benchmark dataset of pantographs in train operation; Perform data augmentation on images in the pantograph benchmark dataset; The target detection algorithm is used to train the pantograph benchmark dataset after data enhancement to obtain the pantograph target detection model.

[0009] Furthermore, the minimax game performed by the discriminator network and the generator network in the data enhancement process is: ; in, represents the expected value, represents the real EMU pantograph image distribution, Represents the Gaussian noise distribution of the generator network input.

[0010] Furthermore, the target detection algorithm is used to calculate the overall loss function of the pantograph image for: ; in, is the weight parameter used to balance the positioning loss, represents the center coordinate loss, represents the size loss, represents the confidence loss, represents the classification loss.

[0011] Furthermore, the key point detection algorithm is used to detect the pantograph structure including: Use key point detection algorithm to detect the key points of the pantograph; The pantograph attitude change and bow angle change are calculated based on the key points.

[0012] Furthermore, constructing a pantograph key point detection model includes: Marking multiple key points on each pantograph body of the pantograph benchmark dataset to create a pantograph key point dataset; The key point recognition algorithm is used to train the pantograph key point dataset to obtain the pantograph key point detection model.

[0013] Furthermore, key point detection also uses posture non-maximum suppression to eliminate redundant key point detection.

[0014] In a second aspect, an embodiment of the present invention provides a real-time detection system for pantograph structural abnormalities, comprising: an image acquisition module, a pantograph target detection module, a pantograph key point detection module, a pantograph body cropping module and an alarm module, wherein the image acquisition module is used to acquire a pantograph image in real time in a video stream of a motor vehicle, the pantograph target detection module is used to perform target detection on the pantograph, the pantograph key point detection module is used to detect the structure of the pantograph, the pantograph body cropping module is used to crop an image of a pantograph area boundary box to obtain an image of the pantograph body area, and the alarm module is used to trigger an alarm when abnormalities occur in target detection and pantograph structure detection.

[0015] Furthermore, the pantograph target detection module performs target detection on the pantograph, including: The pantograph image acquired by the image acquisition module is input into the pantograph target detection model; Use target detection algorithm to detect the target of pantograph; If an abnormal state occurs, the alarm module is notified; If no abnormal state occurs, the pantograph image is sent to the pantograph body cropping module.

[0016] Furthermore, the pantograph key point detection module detects the structure of the pantograph including: Inputting the pantograph body area image into the pantograph key point detection model; Use key point detection algorithm to detect the key points of the pantograph; Calculate the pantograph attitude change and bow angle change based on key points; If the pantograph has structural abnormalities, the alarm module will be notified.

[0017] Beneficial effects: (1) The real-time detection method and system for pantograph structural abnormality provided by the embodiment of the present invention can detect the pantograph body and abnormal state during the operation of the EMU, and provide fast and accurate results for real-time detection and posture matching of key points of the pantograph. The present invention adopts automated detection of pantographs, completely replacing the traditional manual detection process, reducing the risk of human error, greatly saving human resources, and significantly improving the intelligence and efficiency of train pantograph abnormality detection, further ensuring the safety and reliability of train operation.

[0018] (2) The pantograph structural anomaly real-time detection method and system provided in the embodiment of the present invention constructs a train running pantograph benchmark data set, which includes normal pantograph images, abnormal pantograph images, and images of external environment anomalies. The pantograph benchmark data set mainly comes from the pantograph video monitoring system in actual operation, covering a variety of weather and working conditions. All images are in high-definition format to ensure that the details are clearly visible, so as to be used for the training of the pantograph target detection model. The data enhancement method based on the GAN network is adopted to effectively improve the detection accuracy and robustness of the target detection model.

[0019] (3) The real-time detection method and system for pantograph structure anomalies provided in the embodiments of the present invention use a key point detection model to accurately detect abnormal conditions of the pantograph structure in various environments. In complex and harsh environments such as tunnels and lighting, the system can still maintain excellent detection accuracy, thereby ensuring the reliability and practicality of pantograph anomaly detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is a flow chart of a method provided by an embodiment of the present invention; Figure 2 It is a schematic diagram of the principle of enhancing a benchmark data set based on a GAN network according to an embodiment of the present invention; Figure 3 This is an example target detection image under sunlight conditions of an embodiment of the present invention; Figure 4 This is an example target detection diagram in a scenario where an electric spark is generated according to an embodiment of the present invention; Figure 5 This is an example target detection diagram of an abnormal pantograph structure according to an embodiment of the present invention; Figure 6 is an example target detection diagram of foreign body intrusion according to an embodiment of the present invention; Figure 7 is a schematic diagram of the key point positions of the pantograph in an embodiment of the present invention; Figure 8 This is a key point detection flow chart in an embodiment provided by the present invention; Fig. 9 A structural block diagram of a real-time detection system for pantograph structural abnormality provided by an embodiment of the present invention; Fig.10 A system flow chart provided for an embodiment of the present invention. DETAILED DESCRIPTION

[0021] The present solution is described below in conjunction with the accompanying drawings and specific implementation methods.

[0022] The real-time detection method of pantograph structure abnormality in an embodiment of the present invention follows a two-step framework: first, target detection is performed on the pantograph body and abnormal state; second, key point detection is performed on the pantograph area boundary box; finally, the main purpose of the present invention is to provide fast and accurate results for pantograph key point detection and posture matching through the operation of the basic model.

[0023] Figure 1 A flow chart of a method for real-time detection of pantograph structural abnormality provided by an embodiment of the present invention includes: S1, obtain the pantograph image in real time from the video stream of the EMU and input it into the pantograph target detection model; S2, using the target detection algorithm to detect the pantograph, if an abnormal state occurs, an alarm is triggered; S3, if no abnormal state occurs, the image after target detection is input into the pantograph key point detection model; S4, uses key point detection algorithm to detect the structure of the pantograph, and triggers an alarm if any structural abnormality occurs.

[0024] Building a pantograph target detection model includes: Obtain pantograph images from the video stream of the train and create a benchmark dataset of pantographs in train operation; Perform data augmentation on images in the pantograph benchmark dataset; The target detection algorithm is used to train the pantograph benchmark dataset after data enhancement to obtain the pantograph target detection model.

[0025] The pantograph benchmark dataset includes images of pantographs in normal and abnormal states.

[0026] During operation, the EMU will encounter various working conditions and environments such as entering and leaving the station, rain, snow, thunder and lightning, and dark fog. At the same time, considering that some types of faults occur less frequently under actual operating conditions, insufficient data preparation may occur, and the imbalance between category data will have a greater impact on the effect of target recognition. In order to alleviate the uncertain effects of different working conditions and environments on the pantograph target detection model, the embodiment of the present invention adopts a data enhancement method based on the GAN network (GAN, Gererative Advesarial Networks, Generative Adversarial Network) for different category data. By generating image data with abnormal pantograph structures similar to the original data, the performance of the pantograph target detection model is improved, and its robustness is enhanced, so that the pantograph target detection algorithm in subsequent steps can be applicable to various complex weather environments and working condition changes, and can ensure recognition accuracy under complex backgrounds, and enhance the generalization ability of the target detection model in different scenarios.

[0027] Data augmentation processing for images in the pantograph benchmark dataset includes: Constructing the generator network , the generator network Receive a random style image , generate an image through the preset environment output, record it as ; Through the generator network Training the Discriminator Network , the discriminator network Determine whether the generated image is "real".

[0028] Discriminator network The input is an image , output represent The probability of being a real image. If it is 1, it means 100% is a real image. If it is 0, it means it is not a real image.

[0029] Discriminator network Described as a function that maps image data to the probability that the image came from the true data rather than the generator distribution: .

[0030] The schematic diagram of the benchmark dataset enhancement principle based on the GAN network in the embodiment of the present invention is as follows: Figure 2 As shown, the training goal of the GAN network is to make the generator network The generated image data is as close to the real data as possible, making the discriminator network Unable to distinguish real data (i.e. real images) and generated data (i.e. generate images). Discriminator network and the generator network The minimax game described by the following formula is performed, specifically: ; in, represents the expected value, represents the real EMU pantograph image distribution, Represents the Gaussian noise distribution of the generator network input.

[0031] Discriminator network Maximize the objective function , which can maximize the distinction between real data and generate data , specifically: ; Generator Network Try to minimize the objective function , so that the generated data Look as real as possible to fool the discriminator network , specifically: ; In addition, the embodiment of the present invention also uses data enhancement methods such as horizontal and vertical flipping, brightness and contrast adjustment, color space conversion, and adding random Gaussian noise to expand the samples of the pantograph benchmark data set.

[0032] The target detection algorithm is used to train the pantograph benchmark dataset after data enhancement to obtain the pantograph target detection model.

[0033] The target detection algorithm is used to detect the target of the pantograph. If an abnormal state is detected in the image, an alarm is triggered. The abnormal state characteristics are as follows: 1) The video screen is dirty for 180 seconds; 2) The contact network produces large sparks for 1 second; 3) Foreign matter intrusion lasts for 3 seconds; 4) Carbon slide plate abnormality lasts for 3 seconds; 5) Left bow angle abnormality lasts for 3 seconds; 6) Right bow angle abnormality lasts for 3 seconds.

[0034] If the target detection algorithm does not detect an abnormal state in the image, the pantograph area boundary box is cropped to obtain the pantograph body area image.

[0035] The target detection algorithm in the embodiment of the present invention is not limited to the YOLO full series of algorithms such as YOLOv3, YOLOv4, YOLOv5, YOLOv6, YOLOv7, YOLOv8, YOLOv9, and YOLOP. Here, only the lightweight and high-speed YOLOv3 algorithm is used as an example to illustrate the training process.

[0036] The YOLOv3 algorithm uses Darknet-53 as its backbone network, which includes convolutional layers, residual blocks, and pooling layers. Compared with other networks, the design of Darknet-53 is more lightweight and efficient. At the same time, it realizes cross-layer information transmission through residual connections, and enhances the stability of gradient back propagation. When the YOLOv3 algorithm is trained on the pantograph benchmark dataset, the images in the benchmark dataset are uniformly scaled to 416*416 size, and then the image is feature extracted through a convolutional neural network. Feature grid maps with three cell sizes of 13*13, 26*26, and 52*52 are generated on three scales. Each scale is associated with a different detection layer, which is used to predict bounding boxes of different sizes. This is very effective for the detection of small targets and is suitable for detection targets of different sizes such as pantograph bodies, electric sparks, foreign object intrusion, and video screen contamination. In the output layer, the output of the detection layer at each scale includes the coordinates of the bounding box (center point coordinates, width, and height), the confidence of the bounding box (reflecting the possibility that the bounding box contains the target), and the probability of each category. For each bounding box, the class probability is the logistic function prediction for all possible classes.

[0037] Calculate the overall loss function of the pantograph image using the target detection algorithm As described in the following formula, it consists of localization loss, confidence loss and classification loss: ; in, is the weight parameter used to balance the positioning loss, represents the center coordinate loss, represents the size loss, represents the confidence loss, represents the classification loss.

[0038] During training, the center coordinate loss It is used to measure the distance between the center of the image prediction bounding box and the center of the real bounding box, expressed as: ; in, Indicates the number of cells occupied by the image, Represents the number of bounding boxes predicted for each cell. represents the predicted value of the center coordinate of the i-th cell bounding box, Represents the true value of the center coordinate of the i-th cell bounding box, Represents the first indicator variable, which has a value of 1 when the cell and bounding box contain the target, and 0 otherwise.

[0039] Size loss It is used to measure the difference between the width and height of the predicted bounding box and the real bounding box, expressed as: ; in, represents the predicted width and height of the i-th cell bounding box, Represents the true width and height of the i-th cell bounding box.

[0040] Confidence loss of the target object (i.e. image) It is used to measure whether the predicted bounding box contains an object, as well as the difference between the predicted confidence and the actual situation, including the confidence loss of the target and the confidence loss of the target. The specific expression is: ; in, The weight parameter representing the non-existence of the target part in the confidence loss, represents the confidence prediction value of the i-th cell, represents the true value of the confidence of the i-th cell, Represents the second indicator variable, which is 0 when the cell and bounding box contain the target and 1 otherwise.

[0041] Classification Loss It is used to measure the difference between the predicted value and the actual value of the category probability, expressed as: ; in, represents the class probability prediction value of the i-th cell, Represents the actual value of the category probability of the i-th cell.

[0042] In the process of training the pantograph target detection model, the back propagation algorithm and Adam optimizer are used to update the weights of the model, with the goal of minimizing the overall loss function After a large number of iterative optimizations, the overall loss function of the pantograph target detection model is The error of the prediction of each cell is gradually reduced. Finally, by calculating and decoding the cell with the smallest error, the target in the image can be accurately identified and its specific location can be determined. This process ensures that the pantograph target detection model can perform target detection efficiently and accurately in a variety of practical environments and working conditions.

[0043] Figure 3 , Figure 4 The following figure shows the target detection examples under sunlight conditions and spark generation scenarios. Figure 5 , Figure 6The example target detection diagrams of pantograph structural abnormality and foreign object intrusion are respectively displayed. After actual verification, the detection result of the pantograph target detection model of the embodiment of the present invention is accurate, can clearly distinguish different targets, and has good detection accuracy and robustness.

[0044] The image of the pantograph body area is input into the pantograph key point detection model, and the key point detection algorithm is used to detect the structure of the pantograph. If any structural abnormality is found, an alarm is triggered.

[0045] The key point detection algorithm is used to detect the pantograph structure including: Use key point detection algorithm to detect the key points of the pantograph; The pantograph attitude change and bow angle change are calculated based on the key points.

[0046] A reference posture is required before calculating the posture change. The system of the embodiment of the present invention records the average value of the pantograph posture detected from the first 100 frames of images taken after power-on as the initial posture of the pantograph, and then detects the key points of the pantograph frame by frame, converts them into postures and compares them with the reference frame images, calculates the rotation of the pantograph in the up and down and left and right directions, and simultaneously calculates the angle of the bow corner to determine whether the bow corner is broken or detached.

[0047] Calculate whether the pantograph structure is abnormal during the subsequent operation of the train. If abnormal, trigger an alarm. The structural abnormality of the pantograph is as follows: 1) Determine the change in the posture of the pantograph. When the posture of the pantograph exceeds the first threshold, it is determined that the pantograph is tilted; 2) Calculate the angle of the pantograph relative to the pantograph body through the key points at the pantograph angle. When the angle exceeds the second threshold, it is determined that the pantograph angle is broken but not detached.

[0048] Building a pantograph key point detection model includes: Marking multiple key points on each pantograph body of the pantograph benchmark dataset to create a pantograph key point dataset; The key point recognition algorithm is used to train the pantograph key point dataset to obtain the pantograph key point detection model.

[0049] In the detection of key points in the embodiment of the present invention, nine key points are defined on the pantograph body, such as Figure 7 Nine key points are manually annotated on each pantograph body of the pantograph benchmark dataset to construct a pantograph key point dataset.

[0050] There are two reasons for using the pantograph benchmark dataset instead of creating a new dataset. First, the benchmark dataset contains enough pantograph images in various postures. Second, after being enhanced, the benchmark dataset includes various environments and working conditions, which will further improve the detection accuracy and environmental adaptability of key points.

[0051] Selection of key point detection model. The embodiment of the present invention draws on the ideas and models of human key point detection. The models that can be used include but are not limited to various human key point detection models such as openPose, AlphaPose, PoseC3D, and MobilePose. The embodiment of the present invention takes the AlphaPose model as an example to illustrate the detection process of the key point detection model. Figure 8 shown.

[0052] First, the pantograph body region frame is obtained through the target detection algorithm. Even if this step is not used, it is feasible to directly detect the pantograph key points on the original image, but adding this step can effectively improve the detection accuracy of the key points. Then the pantograph body region frame is input into the SSTN (Symmetric Space Transformer Network) module composed of the STN module and the SDTN module to automatically detect the pantograph posture. During the training process, the parallel SPPE module acts as an additional regularizer in the training stage to avoid local optimality and further improve the effect of the SSTN module. At the same time, the posture-guided region frame generator structure (PGPG) is used to expand the data samples so that the pantograph key point detection algorithm can adapt to the imperfect pantograph target detection frame image. Finally, the posture non-maximum suppression algorithm is used to eliminate redundant posture estimation and further improve the performance of the system.

[0053] The parallel SPPE module is used to help the SSTN module extract a better pantograph area position and avoid falling into the local minimum, so a parallel SPPE branch module is added in the key point detection model training phase. The SPPE branch module shares the same STN module as the SPPE module, but omits the SDTN module, and then processes it in parallel with the SPPE module. The pantograph posture label extracted by the SPPE branch module is designated as the center, and the output of the SPPE module is directly compared with the true value of the pantograph posture label. All layers of the Parallel SPPE module are turned off during the training process. The weights of the SPPE branch module are fixed, and the purpose is to back-propagate the error generated after posture positioning to the STN module. If the posture extracted by the STN module is not the center position, the ParallelSPPE module will return a large error. In this way, it can help the STN module focus on the correct center position and extract high-quality area positions. Parallel SPPE will not be used in the testing phase, and Parallel SPPE will only work in the training phase.

[0054] The pantograph positioning will inevitably generate redundant detection frames and redundant posture detection. The AlphaPose model uses Pose Non-Maximum Suppression (Pose NMS) to eliminate redundant key point detection. First, select the standard posture For reference, if the pose distance metric Less than , judged as posture is redundant and will Eliminate. Repeat on all poses until all redundant poses are eliminated. The pose distance metric consists of pose distance and spatial distance, as described below: ; in, For function The parameter set, is the attitude distance function, is the spatial distance function, is a hyperparameter that balances the pose distance and the spatial distance. is a hyperparameter that controls the attitude distance attenuation, is a hyperparameter that controls spatial distance decay.

[0055] Pose distance function It is expressed as: ; in, represents the nth position of the key point, represents the confidence score of the key point, For a Centered box.

[0056] Spatial distance function It is expressed as: ; Fig. 9 The structural block diagram of the real-time detection system of pantograph structural abnormality provided in an embodiment of the present invention comprises: an image acquisition module, a pantograph target detection module, a pantograph key point detection module, a pantograph body cropping module and an alarm module, wherein the image acquisition module is used to acquire the pantograph image in real time in the video stream of the EMU, the pantograph target detection module is used to perform target detection on the pantograph, the pantograph key point detection module is used to detect the structure of the pantograph, the pantograph body cropping module is used to crop the image of the pantograph area boundary box to obtain the pantograph body area image, and the alarm module is used to trigger an alarm when abnormalities occur in the target detection and pantograph structure detection.

[0057] The pantograph target detection module performs target detection on the pantograph, including: inputting the pantograph image acquired by the image acquisition module into the pantograph target detection model; using the target detection algorithm to perform target detection on the pantograph; if an abnormal state occurs, notifying the alarm module; if no abnormal state occurs, sending the pantograph image to the pantograph body cropping module.

[0058] Building a pantograph target detection model includes: obtaining pantograph images in the video stream of the EMU and making a train running pantograph benchmark data set; performing data enhancement processing on the images in the pantograph benchmark data set; using a target detection algorithm to train the pantograph benchmark data set after data enhancement processing to obtain a pantograph target detection model.

[0059] The pantograph key point detection module detects the structure of the pantograph by: inputting the pantograph body area image into the pantograph key point detection model; using the key point detection algorithm to detect the key points of the pantograph; calculating the posture change and bow angle change of the pantograph based on the key points; if the pantograph has a structural abnormality, the alarm module is notified.

[0060] Building a pantograph key point detection model includes: Marking multiple key points on each pantograph body of the pantograph benchmark dataset to create a pantograph key point dataset; The key point recognition algorithm is used to train the pantograph key point dataset to obtain the pantograph key point detection model.

[0061] Key point detection also uses posture non-maximum suppression to eliminate redundant key point detection.

[0062] Fig.10 The system flow chart provided in the embodiment of the present invention obtains the pantograph image in real time in the video stream of the EMU, inputs the pantograph target detection model, performs target detection on the pantograph, and triggers an alarm if an abnormal state occurs. If no abnormal state occurs, the pantograph area boundary box is cropped to obtain the pantograph body area image, and the pantograph body area image is input into the pantograph key point detection model, and the reference frame image is taken as a comparison to detect the structure of the pantograph, calculate the inclination degree of the pantograph body and the local angle of the pantograph, and trigger an alarm if an abnormal state occurs.

[0063] Compared with the prior art, the method and system of the embodiment of the present invention can detect the abnormality of the pantograph structure in real time, and is not affected by complex indoor and outdoor factors such as noise from the station roof and light, and has strong anti-interference ability. When detecting the abnormality of the pantograph structure in real time, it does not need to rely on manual auxiliary identification, and can realize automatic detection, while the real-time and accuracy of the detection are improved.

[0064] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

Claims

1. A real-time detection method for pantograph structural abnormality, characterized in that: include: Obtain pantograph images in real time from the video stream of the train and input them into the pantograph target detection model; Use target detection algorithm to detect the pantograph, and trigger an alarm if an abnormal state occurs; If no abnormal state occurs, the pantograph area boundary frame is cropped to obtain the pantograph body area image; Inputting the pantograph body area image into the pantograph key point detection model; The key point detection algorithm is used to detect the structure of the pantograph, and if any structural abnormality is found, an alarm is triggered.

2. The method for real-time detection of pantograph structural abnormality according to claim 1, characterized in that: Building a pantograph target detection model includes: Obtain pantograph images from the video stream of the train and create a benchmark dataset of pantographs in train operation; Perform data augmentation on images in the pantograph benchmark dataset; The target detection algorithm is used to train the pantograph benchmark dataset after data enhancement to obtain the pantograph target detection model.

3. The method for real-time detection of pantograph structural abnormality according to claim 2, characterized in that: The minimax game performed by the discriminator network and the generator network in the data enhancement process is: ； in, represents the expected value, represents the real pantograph image distribution of the train, Represents the Gaussian noise distribution of the generator network input.

4. The method for real-time detection of pantograph structural abnormality according to claim 2, characterized in that: Calculate the overall loss function of the pantograph image using the target detection algorithm for: ； in, is the weight parameter used to balance the positioning loss, represents the center coordinate loss, represents the size loss, represents the confidence loss, represents the classification loss.

5. The method for real-time detection of pantograph structural abnormality according to claim 1, characterized in that: The key point detection algorithm is used to detect the pantograph structure including: Use key point detection algorithm to detect the key points of the pantograph; The pantograph attitude change and bow angle change are calculated based on the key points.

6. The method for real-time detection of pantograph structural abnormality according to claim 5, characterized in that: Building a pantograph key point detection model includes: Marking multiple key points on each pantograph body of the pantograph benchmark dataset to create a pantograph key point dataset; The key point recognition algorithm is used to train the pantograph key point dataset to obtain the pantograph key point detection model.

7. The method for real-time detection of pantograph structural abnormality according to claim 5, characterized in that: Key point detection also uses posture non-maximum suppression to eliminate redundant key point detection.

8. A real-time detection system for pantograph structural abnormality, characterized in that: According to the method described in any one of claims 1 to 7, the system includes: an image acquisition module, a pantograph target detection module, a pantograph key point detection module, a pantograph body cropping module and an alarm module, the image acquisition module is used to acquire the pantograph image in real time in the video stream of the EMU, the pantograph target detection module is used to perform target detection on the pantograph, the pantograph key point detection module is used to detect the structure of the pantograph, the pantograph body cropping module is used to crop the image of the pantograph area boundary box to obtain the pantograph body area image, and the alarm module is used to trigger an alarm when abnormalities occur in target detection and pantograph structure detection.

9. The pantograph structural abnormality real-time detection system according to claim 8, characterized in that: The pantograph target detection module performs target detection on the pantograph including: The pantograph image acquired by the image acquisition module is input into the pantograph target detection model; Use target detection algorithm to detect the target of pantograph; If an abnormal state occurs, the alarm module is notified; If no abnormal state occurs, the pantograph image is sent to the pantograph body cropping module.

10. The pantograph structural abnormality real-time detection system according to claim 8, characterized in that: The pantograph key point detection module detects the pantograph structure including: Inputting the pantograph body area image into the pantograph key point detection model; Use key point detection algorithm to detect the key points of the pantograph; Calculate the pantograph attitude change and bow angle change based on key points; If the pantograph has structural abnormalities, the alarm module will be notified.

Images