Method and system for detecting pantograph cleat state

Abstract

The invention discloses a method and a system for detecting a pantograph cleat state. The method comprises the following steps: acquiring a cleat state image in a train running state in real time; identifying a pantograph region image in the cavel state image by using the overall pantograph detection model; performing semantic segmentation on the pantograph region image by using the target cavel detection model to form a cavel image; and performing fine segmentation on the cavel image to form a cavel feature image, and determining whether each cavel in the cavel feature image is in a contour structure abnormal state or not by using an abnormal state detection model. According to the method, the state of the pantograph cleat can be quickly and accurately judged in real time in a complex rail transit environment.

Description

Technical Field

[0001] The present invention relates to the technical field of pantograph detection in an electrified railway system, and in particular to a method and system for detecting the state of a pantograph horn. Background Art

[0002] The pantograph is one of the key components in the electrified railway system. It conducts electrical energy to the train's electrical system through sliding contact with the contact wire, thereby providing power for the train. With the rapid development of high-speed railway networks and urban rail networks, the good condition of the pantograph is becoming increasingly important to ensure the safety of railway operations. As an important component of the pantograph, the horn is responsible for ensuring the uniform distribution of pressure between the pantograph and the contact wire, and ensuring the stability of the pantograph under different operating conditions of the train. Therefore, the detection technology of the abnormal state of the pantograph horn has become one of the research focuses in the field of rail transit safety.

[0003] On the one hand, the existing pantograph horn abnormal state detection technology uses manual detection. After the train enters the depot and stops, the train will manually climb onto the roof and rely on visual or professional tools such as calipers and measuring tapes to accurately measure the size and angle of the pantograph horn to determine whether the pantograph horn is missing, deformed, or in other abnormal conditions. This type of method often requires a lot of manpower and time, has low detection efficiency, and has certain safety risks because the inspectors need to directly contact the equipment. At the same time, since manual detection requires the train to stop and enter the depot before it can be carried out, it is impossible to achieve real-time monitoring of the pantograph horn. If a problem occurs between two inspections, it may lead to serious consequences.

[0004] In addition, in the existing patent document (patent number: CN116152188A), a pantograph detection algorithm based on deep learning is provided. This scheme installs a camera on the car to take pictures of the pantograph at regular intervals, and after pre-processing such as scaling and cropping the pictures, the two horns on the left and the two horns on the right of the pantograph are detected based on the deep learning algorithm. A computer program is used to calculate the intersection and union ratio of the coverage areas of the two horn identification frames on the same side to determine whether there is an abnormality in the horn structure on one side.

[0005] In addition, in the existing patent document (patent number: CN114877814A), a pantograph state detection device and method based on image processing is provided. This solution collects the pantograph image of the running train through the field trigger module and the image acquisition module, sets the upper and lower thresholds according to experience to realize the pantograph positioning, and then performs Gaussian filtering, Canny edge detection and Hough transformation in sequence, detects the edge straight line of the horn according to the edge angle and position information of the horn, and realizes the detection of the missing horn by analyzing whether the edge straight line of the horn exists.

[0006] In summary, the existing pantograph horn abnormal state detection technology provides a technical solution that uses image recognition detection methods and relies on the on-board contact network operation state detection device installed on the roof of the EMU to achieve high-definition image acquisition of the pantograph overall structure, and realizes the positioning of the pantograph horn structure and abnormal state identification based on the image processing algorithm. However, due to the relatively drastic changes in the image background when the train is running, the detection accuracy and robustness of the traditional algorithm based on image recognition technology are poor, and missed detection and false detection are prone to occur.

[0007] Therefore, the prior art needs to provide a real-time detection solution for the abnormal state of the pantograph horns that can simultaneously meet the requirements of high detection accuracy, real-time performance and robustness.

[0008] At present, the status detection of pantograph horns mainly relies on manual visual inspection and fixed-point image detection. These methods are not only costly, but also have low detection accuracy and poor real-time performance, and cannot realize the real-time detection and alarm function under the running state of the train. In addition, abnormal state detection based on traditional image processing methods is difficult to meet the needs of high-speed train running status, especially when the background environment changes drastically, there are obvious deficiencies in detection accuracy and robustness. This method realizes real-time detection of abnormal state of pantograph horns under the running state of the train through image processing technology based on deep learning.

[0009] Therefore, the technical problem to be solved by the present invention can be summarized as follows: the present invention uses computer vision technology in combination with an image recognition algorithm to achieve real-time detection of abnormal pantograph horns during train operation. Summary of the invention

[0010] The purpose of the present invention is to provide a real-time detection solution for abnormal state of pantograph horns that can simultaneously meet the requirements of high detection accuracy, real-time performance and robustness.

[0011] In order to solve the above technical problems, an embodiment of the present invention provides a method for detecting the state of pantograph horns, including: acquiring a horn state image under a train running state in real time; using an overall pantograph detection model to identify a pantograph area image in the horn state image; using a target horn detection model to perform semantic segmentation on the pantograph area image to form a horn image; performing fine segmentation on the horn image to form a horn feature image, and using an abnormal state detection model to determine whether each horn in the horn feature image is in an abnormal contour structure state.

[0012] Preferably, the overall pantograph detection model is constructed using the following steps: obtaining the pantograph state images collected during the historical operation of the train; selecting the overall pantograph area in the historical pantograph state images to form a historical pantograph area image; forming a first training set and a first verification set based on the historical pantograph area image; using the first training set and the first verification set to train and optimize the preset YOLOv5 algorithm model to form the overall pantograph detection model.

[0013] Preferably, the target horn detection model is constructed using the following steps: annotating the horn contours of the historical pantograph area image to form a historical horn image and obtaining a second training set; based on the second training set, training a preset U-net semantic segmentation model to form the target horn detection model.

[0014] Preferably, the step of obtaining a second training set based on the historical sheep horn images includes: converting the historical sheep horn images into a file format to form the second training set, including: converting a json file with the sheep horn outlines annotated into a mask png file.

[0015] Preferably, the abnormal state detection model is constructed using the following steps: dividing the historical horn images into a normal sample set and a horn structure abnormal sample set; performing grayscale conversion and fine segmentation processing on the horn images in the normal sample set in turn, and using the horn feature images obtained after the processing to establish a first type of data set to be trained, and performing grayscale conversion and fine segmentation processing on the horn images in the horn structure abnormal sample set in turn, and using the horn feature images obtained after the processing to establish a second type of data set to be trained; using the first type of data set to be trained and the second type of data set to be trained to train a preset support vector machine model to obtain the abnormal state detection model.

[0016] Preferably, the step of performing fine segmentation processing on the sheep horn image includes: using a first position range area frame to crop the left side of the sheep horn image in the sheep horn image, and using a second position range area frame to crop the right side of the sheep horn image in the sheep horn image; splicing the left side of the sheep horn image and the right side of the sheep horn image to form the sheep horn feature image.

[0017] Preferably, the step of obtaining the image of the state of the horns of the train in real time under the running state includes: using the on-board contact network operation detection device installed on the roof of the train to obtain video data of the complete structure of the pantograph in the field of view; performing video frame extraction processing on the video data according to a preset frame extraction frequency to obtain a dynamic image of the state of the horns of the train.

[0018] Preferably, the method further comprises: after respectively obtaining the ram's horn state image or the pantograph area image or the ram's horn image, saving the corresponding type of image under the corresponding target path for easy calling.

[0019] In addition, an embodiment of the present invention also provides a system for detecting the state of pantograph horns, including: a real-time image acquisition module, which is configured to acquire a horn state image under a train running state in real time; a pantograph recognition module, which is configured to use an overall pantograph detection model to identify a pantograph area image in the horn state image; a horn recognition module, which is configured to use a target horn detection model to perform semantic segmentation on the pantograph area image to form a horn image; a horn state detection module, which is configured to perform fine segmentation on the horn image to form a horn feature image, and use an abnormal state detection model to determine whether each horn in the horn feature image is in an abnormal contour structure state.

[0020] In addition, an embodiment of the present invention further provides a computer-readable storage medium storing computer-executable instructions, which can implement the steps of the method described above when executed by a processor.

[0021] Compared with the prior art, one or more embodiments of the above scheme may have the following advantages or beneficial effects:

[0022] The present invention proposes a method and system for detecting the state of the pantograph horns. The method and system obtain the high-resolution image information of the pantograph through the on-board contact network operation state detection device, and use the YOLOv5 algorithm to realize the pantograph position positioning after frame extraction processing of the real-time video, and on this basis, use the U-net network to realize the fine segmentation of the left and right horn images of the pantograph, and then use the support vector machine to classify the horn images to distinguish the normal and abnormal states of a single pantograph image. The present invention can realize the real-time, fast and accurate judgment of the pantograph horn state in a complex rail transit environment through computer vision technology, in conjunction with the image recognition algorithm based on deep learning, thereby improving the efficiency and accuracy of equipment maintenance, reducing the workload of manual inspection, and improving the efficiency of rail transit operation and maintenance; at the same time, the present invention has strong robustness without occupying a large amount of computing power.

[0023] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0025] Figure 1 A schematic diagram of the steps of a method for detecting the state of a pantograph horn according to an embodiment of the present application.

[0026] Figure 2 It is a schematic diagram of a specific process of the method for detecting the state of the pantograph horn according to an embodiment of the present application.

[0027] Figure 3 This is an example diagram of a pantograph horn status image in a method for detecting a pantograph horn status in an embodiment of the present application.

[0028] Figure 4 This is an example diagram of a pantograph area image in a method for detecting a pantograph horn status according to an embodiment of the present application.

[0029] Figure 5 This is an example diagram of a horn image in a method for detecting the state of a pantograph horn in an embodiment of the present application.

[0030] Figure 6 It is a schematic diagram of the construction process of the overall pantograph detection model in the method for detecting the pantograph horn status according to an embodiment of the present application.

[0031] Figure 7 It is a schematic diagram of the construction process of the target horn detection model in the method for detecting the horn status of the pantograph according to an embodiment of the present application.

[0032] Figure 8 It is a schematic diagram of the construction process of the abnormal state detection model in the method for detecting the pantograph horn state according to an embodiment of the present application.

[0033] Fig. 9 It is a structural schematic diagram of a system for detecting the state of a pantograph horn according to an embodiment of the present application. DETAILED DESCRIPTION

[0034] The following will describe the implementation methods of the present invention in detail with reference to the accompanying drawings and embodiments, so that the implementation process of how the present invention applies technical means to solve technical problems and achieve technical effects can be fully understood and implemented accordingly. It should be noted that as long as there is no conflict, the various embodiments of the present invention and the various features in the embodiments can be combined with each other, and the technical solutions formed are all within the protection scope of the present invention.

[0035] In addition, the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions. Also, although a logical sequence is shown in the flowchart, in some cases, the steps shown or described can be performed in a sequence different from that here.

[0036] The terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments. Unless the context clearly indicates otherwise, the singular forms "one", "one" and "item" used herein are also intended to include plural numbers. It should also be understood that the terms "include" and / or "comprise" used herein specify the existence of stated features, integers, steps, operations, units and / or components, without excluding the existence or addition of one or more other features, integers, steps, operations, units, components and / or combinations thereof.

[0037] At present, the state detection of the pantograph horns mainly relies on manual visual inspection and fixed-point image detection. These methods are not only costly, but also have low detection accuracy and poor real-time performance, and cannot realize the real-time detection alarm function under the train running state. In addition, it is difficult to adapt to the needs of the high-speed train running state based on the traditional image processing method for abnormal state detection, especially when the background environment changes drastically, there are obvious deficiencies in detection accuracy and robustness. The present application embodiment provides a method and system for detecting the state of the pantograph horns. The method and system will obtain the pantograph high-resolution image information through the on-board contact network running state detection camera, and after the acquired real-time video is frame-processed, the efficient network structure of the YOLOv5 algorithm is used to quickly identify the position of the pantograph overall structure to provide an accurate bounding box, and then the U-net network is used to further process the detection result output of the YOLOv5, and through its unique encoder-decoder structure, the fine segmentation of the left and right horns of the pantograph is realized; after the segmentation is completed, the left and right horns of the image are used as features to train the SVM model, and the SVM is used for classification to effectively distinguish the normal and abnormal states of the pantograph horns.

[0038] Therefore, the present invention can realize rapid and accurate judgment of the pantograph horn status in a complex rail transit environment, improve the efficiency and accuracy of equipment maintenance, reduce the workload of manual inspection, and has important practical value.

[0039] Embodiment 1

[0040] Figure 1 A schematic diagram of the steps of a method for detecting the state of a pantograph horn according to an embodiment of the present application. Figure 2 This is a schematic diagram of a specific process of the method for detecting the pantograph horn state according to an embodiment of the present application. Figure 1 and Figure 2, the specific steps of the method for detecting the pantograph horn state (also called "pantograph horn state detection method") described in an embodiment of the present invention are described.

[0041] Step S110, obtaining in real time the state image of the ram's horns when the train is in operation.

[0042] In the embodiment of the present invention, each horn state image is an image containing the real structural state characteristics of all horns in the pantograph. In addition, the train described in the present invention is an EMU train running on a high-speed railway network and an urban rail network.

[0043] Specifically, in step S110, Figure 2 As described, first, an on-board overhead contact network operation detection device installed on the roof of a train is used to obtain video data of the complete structure of the pantograph in the field of view, and then the video data is processed by video frame extraction at a preset frame extraction frequency to obtain a dynamic sheep horn state image, that is, a frame of the sheep horn state is obtained at a preset time interval.

[0044] After obtaining multiple frames of continuously acquired horn state images, this type of image is saved in the horn state image target path for easy calling.

[0045] After relying on the on-board contact network operation status detection device installed on the roof of the EMU to realize real-time high-definition image acquisition of the pantograph's overall structure, and performing frame extraction processing on the acquired video data, multiple frames of continuously acquired horn status images are obtained, and the multiple frames of horn status images are saved in the horn status image target path to enter step S120.

[0046] In one embodiment, after obtaining a single-frame horn state image after video frame extraction processing, the current horn state image is saved in the horn state image target path, and after completing the detection of step S120, the horn state image saved in this frame is deleted.

[0047] In the embodiment of the present invention, the frame extraction frequency is set to 1 second / frame.

[0048] Step S120, using the overall pantograph detection model, identifying the pantograph area image in the horn state image obtained in step S110. Figure 3 A specific example figure showing the obtained pantograph area image is shown.

[0049] In the embodiment of the present invention, the overall pantograph detection model is a model for selecting the pantograph area in the horn state image.

[0050] In this way, the embodiment of the present invention can directly use the overall pantograph detection model to convert each horn state image into a corresponding pantograph area image. Figure 4A specific example diagram showing the pantograph area image.

[0051] In addition, after obtaining multiple frames of pantograph area images, step S120 will save the type of image under the pantograph area image target path for easy calling.

[0052] Specifically, the horn state image is called from the horn state image target path, and then the overall pantograph detection model is used to detect the overall pantograph structure in the horn state image as the target image, the target detection data is output and the image in the prediction box is cropped to form a pantograph area image, so as to save the pantograph area image under the pantograph area image target path, and then enter step S130.

[0053] Step S130, using the target horn detection model, semantically segment the pantograph area image obtained in step S120 to form a horn image. In the embodiment of the present invention, the horn image includes an image of the semantic extraction results of all horns.

[0054] In this way, the embodiment of the present invention can directly use the target horn detection model to convert each pantograph area image into a corresponding horn image. Figure 5 A specific example image of a sheep horn image is shown.

[0055] In addition, after obtaining multiple frames of sheep horn images, step S130 will save the type of image in the sheep horn image target path for easy calling.

[0056] Specifically, the pantograph area image is called from the pantograph area image target path, and then the target horn detection model is used to determine the area where the target horns exist in the image, so as to semantically segment the horn area from the non-horn area to form a horn image, and then the horn image is saved under the horn image target path, and then enters step S140.

[0057] Step S140, finely segmenting the goat horn image obtained in step S130 to form a goat horn feature image, and using an abnormal state detection model to determine whether each goat horn in the goat horn feature image is in an abnormal state of contour structure.

[0058] In step S140, first, the goat horn image obtained in step S130 is grayed and then finely segmented to form a goat horn feature image.

[0059] Specifically, after calling the horns image from the horns image target path, the horns image is first grayscaled, and then the first position range area box is used to crop the left horns image in the horns image, and the second position range area box is used to crop the right horns image in the horns image, and then, the left horns image and the right horns image are spliced ​​to form a horns feature image, thereby highlighting the fine features of the horns part.

[0060] In an embodiment of the present invention, the abnormal state detection model is a model for detecting the abnormal state of the horn contour structure in the horn feature image, and directly outputs a detection result indicating the abnormal location and type of the abnormality. In addition, when the abnormal state detection model does not detect an abnormality in the horn contour structure, a detection result indicating that the state is normal is directly output.

[0061] In the embodiment of the present invention, the abnormal types include but are not limited to: deformation and defect (missing).

[0062] Specifically, the horn image is called in the horn image target path and grayscaled first. Then, the image with the upper left coordinates (0,0) and the lower right coordinates (65,127) is cropped as the left horn image, and the image with the upper left coordinates (447,0) and the lower right coordinates (511,127) is cropped as the right horn image. The cropped left and right horn images are spliced ​​into a whole image to form a horn feature image. After that, the spliced ​​horn feature image is imported as a feature into the trained abnormal state detection model to realize the pantograph horn abnormal state detection function.

[0063] Embodiment 2

[0064] Based on the horn state detection method described in Example 1, the construction processes of the overall pantograph detection model, the target horn detection model and the abnormal state detection model are described in turn.

[0065] First, the construction process of the overall pantograph detection model is explained.

[0066] Specifically, the horn status image collected in real time during the historical operation of the train is obtained, and then the entire pantograph area is framed in the historical horn status image to form a historical pantograph area image. After that, a first training set and a first verification set are formed according to the historical pantograph area image, and then the current first training set and the first verification set are used to train and optimize the preset YOLOv5 algorithm model to form an overall pantograph detection model.

[0067] like Figure 6As shown, by sampling the historically collected video data under the complete pantograph field of view during the historical operation of the train, the video data is converted into a multi-frame pantograph state image, and these historical multi-frame pantograph state images are divided into a first training set and a first verification set according to a preset ratio (for example, 85:15). After that, each pantograph state image in the first training set and the first verification set is labeled respectively (the labeling operation refers to selecting the entire pantograph area in the pantograph state image), and based on the labeled training set and verification set data files, the pre-built YOLOv5 algorithm model is trained and optimized to form an overall pantograph detection model.

[0068] In an embodiment of the present invention, the confidence threshold in the overall pantograph detection model can be set to 0.85, and the IoU threshold can be set to 0.65. That is, when the confidence of the input horn state image is greater than 0.85, the current sample is determined to be a positive sample, the horn area image can be selected, and the anchor frame with IoU exceeding 0.65 is considered to have identified the target area object.

[0069] Next, the construction process of the target horn detection model is explained.

[0070] In the process of constructing the target horn detection model, the horn contours of the above-mentioned historical pantograph area image are first annotated to form a historical horn image, and a second training set is obtained. Then, based on the second training set, the preset U-net semantic segmentation model is trained to form a target horn detection model.

[0071] In a specific embodiment of obtaining the second training set based on historical images of goat horns, each historical image of goat horns is converted into a file format to form the second training set. For example, a json file with the goat horn outline annotated is converted into a mask png file.

[0072] like Figure 7 As shown, the cropped historical pantograph area images are obtained, and normalization operations such as scaling are performed on these historical pantograph area images, and then the contours of the horns in the normalized images are annotated. After that, the annotated json image file is converted into a mask png image file to generate a second training set of masks. Finally, the current second training set is used to train the U-net semantic segmentation model, so as to use the target horn detection model to realize the image semantic segmentation processing of at least four horns on the left and right sides of the pantograph.

[0073] In an embodiment of the present invention, the image threshold parameter in the target horn detection model can be set to 0.6, that is, when the confidence of the input pantograph area image is greater than 0.6, it is determined that the current image has target horns, and a horn image after semantic segmentation can be formed.

[0074] The following describes the construction process of the abnormal state detection model.

[0075] Specifically, a plurality of historical horn images are divided into a normal sample set and a horn structure abnormal sample set, and then each horn image in the normal sample set is successively subjected to grayscale conversion and fine segmentation processing, and the horn feature images obtained after the processing are used to establish a first type of data set to be trained, and each horn image in the horn structure abnormal sample set is successively subjected to grayscale conversion and fine segmentation processing, and the horn feature images obtained after the processing are used to establish a second type of data set to be trained, and finally, the first type of data set to be trained and the second type of data set to be trained are used to train a preset support vector machine model to obtain an abnormal state detection model.

[0076] like Figure 8 As shown, the historical sheep horn images after semantic segmentation under the target path are read, and these historical sheep horn images are divided into a normal (image) sample set and a sheep horn structure abnormality (image) sample set.

[0077] Each historical horn image in the two sample sets is first converted into a grayscale image, and then the left horn image in the grayscale historical horn image is cropped using the first position range area frame, such as: the image with the upper left point coordinates (0,0) and the lower right point coordinates (65,127) of the cropped image is used as the left horn image; and the right horn image in the grayscale historical horn image is cropped using the second position range area frame, such as: the image with the upper left point coordinates (447,0) and the lower right point coordinates (511,127) of the cropped image is used as the right horn image; the cropped left and right horn images belonging to the same horn image are spliced ​​into a whole image to obtain multiple horn feature images. Finally, the first type of training data set is constructed according to the horn feature image corresponding to each horn image in the normal sample set, and the second type of training data set is constructed according to the horn feature image corresponding to each horn image in the horn structure abnormality sample set.

[0078] Continue to refer Figure 8 The first and second training data sets after image stitching are imported into the pre-built SVM support vector machine model as data sets containing fine features of sheep horns for training. The SVC algorithm is used as the estimator, and the random parameter sampling combination method is used to randomly extract the parameter subspace and search in the subspace to quickly select the best parameters in the support vector machine model, thereby obtaining a support vector machine model with the function of distinguishing sheep horn structure abnormalities, which is recorded as the abnormal state detection model.

[0079] In this way, after completing the construction of the overall pantograph detection model, the target horn detection model and the abnormal state detection model, the pantograph horn state detection can be realized by applying these models during the actual operation of the train.

[0080] (1) The on-board overhead contact network operation status detection device installed on the roof of the EMU is used to acquire real-time high-definition images of the pantograph structure as a whole, and the acquired video data is processed by frame extraction and saved in the target path.

[0081] (2) Import the image data under the target path into the Yolov5 detection model trained in , set the confidence threshold to 0.85 in the model, and set the IoU threshold to 0.65, that is, when the confidence of the image to be detected is greater than 0.85, the sample is judged as a positive sample, and the anchor box with IoU exceeding 0.65 is considered to be the same object. Based on this, the overall pantograph structure in the target image is detected, the target detection data is output, and the image in the prediction box is cropped and saved in the target path.

[0082] (3) The pantograph image cropped in (2) is uniformly scaled to 512*128 pixels and imported into the trained U-net model. The image threshold parameter is set to 0.6, that is, when the threshold is greater than 0.6, it is determined that the target horn image exists. Then, the segmented horn image is saved in the target path.

[0083] (4) Call the ram horn image saved in the target path in (3), crop the image with the upper left point coordinates (0, 0) and the lower right point coordinates (65, 127) as the left ram horn image, crop the image with the upper left point coordinates (447, 0) and the lower right point coordinates (511, 127) as the right ram horn image, and splice the cropped left and right ram horn images into the whole image. Import the spliced ​​image as a feature into the trained support vector machine model to realize the abnormal state detection function of the pantograph ram horn.

[0084] Embodiment 3

[0085] Based on the horn state detection method of the above embodiment 1 and embodiment 2, the embodiment of the present invention further provides a system for detecting the horn state of a pantograph (also called "horn state detection system"). The horn state detection system is used to implement the above horn state detection method.

[0086] Fig. 9 Schematic diagram of the structure of the system for detecting the pantograph horn state according to the embodiment of the present application. Fig. 9 As shown, the horn status detection system described in the embodiment of the present invention includes: a real-time image acquisition module 91, a pantograph recognition module 92, a horn recognition module 93 and a horn status detection module 94.

[0087] The real-time image acquisition module 91 is implemented according to the method described in the above step S110, and is configured to acquire the horn status image under the running state of the train in real time; the pantograph recognition module 92 is implemented according to the method described in the above step S120, and is configured to use the overall pantograph detection model to identify the pantograph area image in the horn status image; the horn recognition module 93 is implemented according to the method described in the above step S130, and is configured to use the target horn detection model to perform semantic segmentation on the pantograph area image to form a horn image; the horn status detection module 94 is implemented according to the method described in the above step S140, and is configured to perform fine segmentation on the horn image to form a horn feature image, and use the abnormal state detection model to determine whether each horn in the horn feature image is in an abnormal contour structure state.

[0088] Embodiment 4

[0089] Based on the above-mentioned methods for detecting the state of pantograph horns of the first and second embodiments, the present invention provides a computer-readable storage medium in an embodiment. The storage medium stores a computer program, and the computer program is executed to run a method for detecting the state of pantograph horns. The computer program can run computer instructions, and the computer instructions include computer program codes, and the computer program codes can be in source code form, object code form, executable file or some intermediate form, etc.

[0090] Computer-readable storage media may include: any entity or device that can carry computer program code, recording media, USB flash drives, mobile hard disks, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0091] It should be noted that the content contained in computer-readable storage media can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, the content contained in computer-readable storage media does not include electrical carrier signals and telecommunication signals.

[0092] The present invention discloses a method and system for detecting the state of pantograph horns. The method and system obtain high-resolution image information of the pantograph through an on-board contact network operation state detection device, and use the YOLOv5 algorithm to realize the pantograph position positioning after frame extraction processing of the real-time video, and on this basis, use the U-net network to realize the fine segmentation of the left and right horn images of the pantograph, and then use the support vector machine to classify the horn images to distinguish the normal and abnormal states of a single pantograph image. The present invention can realize the real-time, fast and accurate judgment of the horn state of the pantograph in a complex rail transit environment through computer vision technology, in conjunction with an image recognition algorithm based on deep learning, thereby improving the efficiency and accuracy of equipment maintenance, reducing the workload of manual inspection, and improving the efficiency of rail transit operation and maintenance; at the same time, the present invention has strong robustness without occupying a large amount of computing power.

[0093] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by any person familiar with the technology within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

[0094] In the description of the present invention, unless otherwise specified, "plurality" means two or more than two; the orientations or positional relationships indicated by the terms "upper", "lower", "left", "right", "inner", "outer", "front end", "rear end", "head", "tail", etc. are based on the orientations or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", "third", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0095] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "connected" and "connection" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0096] It should be understood that the embodiments disclosed in the present invention are not limited to the specific structures, processing steps or materials disclosed herein, but should be extended to equivalent substitutions of these features understood by ordinary technicians in the relevant field. It should also be understood that the terms used herein are only used for the purpose of describing specific embodiments and are not meant to be limiting.

[0097] The "one embodiment" or "embodiment" mentioned in the specification means that the specific features, structures or characteristics described in conjunction with the embodiment are included in at least one embodiment of the present invention. Therefore, the phrases "one embodiment" or "embodiment" appearing in various places throughout the specification do not necessarily refer to the same embodiment.

[0098] Although the embodiments disclosed in the present invention are as above, the contents described are only embodiments adopted for facilitating the understanding of the present invention and are not intended to limit the present invention. Any technician in the technical field to which the present invention belongs can make any modifications and changes in the form and details of the implementation without departing from the spirit and scope disclosed in the present invention, but the patent protection scope of the present invention shall still be subject to the scope defined in the attached claims.

Claims

1. A method for detecting the state of a pantograph horn, characterized in that: include: Real-time acquisition of the state image of the horns of the train in operation; Using the overall pantograph detection model, identifying the pantograph area image in the horn state image; Using a target horn detection model, semantic segmentation is performed on the pantograph area image to form a horn image; The horn image is finely segmented to form a horn feature image, and an abnormal state detection model is used to determine whether each horn in the horn feature image is in an abnormal state of contour structure.

2. The method according to claim 1, characterized in that The overall pantograph detection model is constructed using the following steps: Obtain the images of the state of the horns collected during the historical operation of the train; Select the entire pantograph area in the historical pantograph status image to form a historical pantograph area image; According to the historical pantograph area image, forming a first training set and a first verification set; The first training set and the first validation set are used to train and optimize the preset YOLOv5 algorithm model to form the overall pantograph detection model.

3. The method according to claim 2, characterized in that The target horn detection model is constructed using the following steps: Annotating the historical pantograph area image with a ram's horn outline to form a historical ram's horn image and obtain a second training set; According to the second training set, the preset U-net semantic segmentation model is trained to form the target sheep horn detection model.

4. The method according to claim 3, characterized in that The step of obtaining a second training set according to the historical sheep horn images includes: The historical sheep horn images are converted into file formats to form the second training set, which includes: Convert the json file with the sheep horn outline marked into a mask png file.

5. The method according to claim 3 or 4, characterized in that: The abnormal state detection model is constructed using the following steps: Dividing the historical sheep horn images into a normal sample set and a sheep horn structure abnormal sample set; The sheep horn images in the normal sample set are sequentially subjected to grayscale conversion and fine segmentation processing, and the sheep horn characteristic images obtained after the processing are used to establish a first type of data set to be trained, and the sheep horn images in the sheep horn structure abnormality sample set are sequentially subjected to grayscale conversion and fine segmentation processing, and the sheep horn characteristic images obtained after the processing are used to establish a second type of data set to be trained; The first type of data set to be trained and the second type of data set to be trained are used to train a preset support vector machine model to obtain the abnormal state detection model.

6. The method according to any one of claims 1 to 5, characterized in that: The steps of finely segmenting the sheep horn image include: Using the first position range area frame to crop the left side of the goat horn image, and using the second position range area frame to crop the right side of the goat horn image; The left-side sheep horn image and the right-side sheep horn image are spliced ​​to form the sheep horn characteristic image.

7. The method according to any one of claims 1 to 6, characterized in that: The step of obtaining the horn status image of the train in real time under the running state includes: The on-board overhead contact network operation detection device installed on the roof of the train is used to obtain video data of the complete structure of the pantograph within the field of view; The video data is subjected to video frame extraction processing according to a preset frame extraction frequency to obtain a dynamic sheep horn state image.

8. The method according to any one of claims 1 to 7, characterized in that: The method further comprises: After respectively obtaining the ram's horn state image or the pantograph area image or the ram's horn image, the corresponding type of image is saved in the corresponding target path for easy calling.

9. A system for detecting the state of pantograph horns, characterized in that: include: A real-time image acquisition module, configured to acquire a state image of the horns of a train in real time when the train is in operation; A pantograph recognition module configured to recognize a pantograph area image in the ram's horn state image using an overall pantograph detection model; A horn recognition module is configured to use a target horn detection model to perform semantic segmentation on the pantograph area image to form a horn image; The horn state detection module is configured to perform fine segmentation on the horn image to form a horn feature image, and use an abnormal state detection model to determine whether each horn in the horn feature image is in an abnormal state of contour structure.

10. A computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement the steps of the method according to any one of claims 1 to 8.

Images