Contact line hard bend state detection method and device based on deep learning and medium

Abstract

The application discloses a contact line hard bending state detection method and device based on deep learning and a medium. The method comprises the following steps: acquiring a detection image containing a pantograph and a contact line which is shot in real time during train operation; locating a pantograph region from the detection image, and determining a contact line extraction region according to the pantograph region; extracting the contact line from the contact line extraction region to obtain a contact line image; inputting the extracted contact line image into a pre-trained hard bending state detection model to obtain a detection result output of whether the contact line exists hard bending, wherein the hard bending state detection model is obtained by training a deep learning model using an image data set, and each image data in the image data set contains the pantograph, the contact line and the labeling information of a hard bending region of the contact line with hard bending. The application has the advantages of simple implementation method, low cost, high detection precision and efficiency, strong flexibility and the like.

Description

Technical Field

[0001] This invention relates to the field of contact wire detection technology for rail transit systems, and in particular to a method, device, and medium for detecting the hard bend state of contact wires based on deep learning. Background Technology

[0002] A hard bend in the contact wire refers to a small, localized bend in the contact wire caused by factors such as the material of the contact wire or a sudden, significant change in contact wire tension during installation. At high speeds, a hard bend in the contact wire can lead to arcing, causing irreversible damage to the pantograph and contact wire. Therefore, it is necessary to monitor the hard bend status of the contact wire for safety. Currently, the hard bend status of the contact wire is usually checked manually, which is inefficient, inflexible, and prone to misjudgment. Intelligent detection of the hard bend status of the contact wire could greatly improve detection efficiency and flexibility.

[0003] Some practitioners have proposed using the line width parameter to extract the contact line, then performing segmented straight-line fitting on the contact line, and calculating whether the distance from each segment point to the fitted straight line exceeds a specified threshold to determine whether the contact line has experienced a hard bend. Although this method can detect the hard bend of the contact line, its actual detection efficiency and accuracy are not high. Because the bending amplitude of a hard bend fault is small, it appears small in the image, making it difficult to accurately determine whether the contact line has experienced a hard bend by directly using the straight-line distance, which easily leads to false detections. Summary of the Invention

[0004] The technical problem to be solved by this invention is: in view of the technical problems existing in the prior art, this invention provides a method, device, equipment and medium for detecting the hard bending state of contact wires based on deep learning, which is simple to implement, low in cost, efficient and accurate, and highly flexible.

[0005] To solve the above-mentioned technical problems, the technical solution proposed by this invention is as follows:

[0006] A deep learning-based method for detecting the hard bend state of contact wires, comprising the following steps:

[0007] Acquire real-time detection images of the train, including the pantograph and contact wire, during train operation;

[0008] The pantograph region is located from the detected image, and the contact wire extraction region is determined based on the pantograph region; the contact wire is extracted from the contact wire extraction region to obtain a contact wire image;

[0009] The extracted contact wire image is input into a pre-trained hard bend state detection model to obtain the detection results of whether the contact wire has a hard bend and the hard bend area when a hard bend exists. The hard bend state detection model is obtained by training a deep learning model using an image dataset in advance. Each image data in the image dataset contains annotation information of the pantograph, contact wire, and the hard bend area of ​​the contact wire with a hard bend.

[0010] Furthermore, the step of locating the pantograph region from the detection image and determining the contact wire extraction region based on the pantograph region includes: matching a standard template containing pantographs of different vehicle models with the detection image to locate the pantograph region, and using a specified area above the pantograph region as the contact wire extraction region.

[0011] Furthermore, the step of extracting the contact line image from the contact line extraction area includes: extracting all lines from the contact line extraction area, merging lines with similar slopes and close distances, filtering the merged lines according to their characteristics to obtain filtered lines, and finally extracting the contact line above the pantograph from the intersection state of each line with the pantograph in the filtered lines.

[0012] Furthermore, the line features include any one or more of the following: line length, line angle, line curvature, line brightness, line width, and line grayscale.

[0013] Furthermore, after removing lines whose intersection with the upper edge of the pantograph is located in the horn area on both sides of the pantograph from the filtered lines, the line with the widest line width and the brightest line gray level is selected as the extracted contact line.

[0014] Furthermore, after extracting the contact line image from the contact line extraction area, the process further includes processing the background of the extracted contact line by opening a rotating rectangular window in the neighborhood of the contact line with the direction of the contact line, and setting the pixels outside the rotating rectangular window to a specified value.

[0015] Furthermore, after obtaining the detection results of whether the contact line has a hard bend and the hard bend area when a hard bend exists, the method further includes opening a processing area of ​​a specified size in the upper left corner and the lower right corner of the detection frame, and statistically analyzing the gray values ​​of the processing areas, and verifying the current detection result based on the gray values ​​in the upper left corner and the lower right corner.

[0016] Furthermore, after obtaining the detection result of whether the contact line has a hard bend, the method also includes extracting the skeleton points in the contact line area, calculating the distance from the skeleton points to the center point of the detection result, and verifying the current detection result based on the calculated distance.

[0017] A deep learning-based contact wire hard bend state detection device, comprising:

[0018] The image acquisition module is used to acquire real-time detection images of the train, including the pantograph and contact wire, during train operation;

[0019] The positioning module is used to locate the pantograph area from the detected image and determine the contact wire extraction area based on the pantograph area;

[0020] The extraction module is used to extract the contact line from the contact line extraction area to obtain a contact line image;

[0021] The hard bend recognition module is used to input the extracted contact wire image into a pre-trained hard bend state detection model to obtain whether there is a hard bend in the contact wire and the detection result of the hard bend area when a hard bend exists. The hard bend state detection model is obtained by pre-training a deep learning model using an image dataset. Each image data in the image dataset contains annotation information of the pantograph, the contact wire, and the hard bend area of ​​the contact wire where a hard bend exists.

[0022] A deep learning-based contact wire hard bend detection device includes a processor and a memory, wherein the memory is used to store a computer program and the processor is used to execute the computer program to perform the method described above.

[0023] A contact wire hard bend detection system, comprising:

[0024] Image acquisition equipment is installed on the top of the train to acquire images of the pantograph and contact wire above the train during operation to obtain detection images;

[0025] The aforementioned hard bend detection device is used to receive detection images acquired by an image acquisition device and detect the hard bend state of the contact wire.

[0026] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described above.

[0027] Compared with existing technologies, the advantages of this invention are as follows: This invention pre-trains a hard bend state detection model based on a deep learning model. After acquiring the detection images of the pantograph and contact wire in real time, the pantograph area is first located, and then the contact wire extraction area is determined using the pantograph area location. Subsequently, the real-time contact wire image is extracted and input into the trained hard bend state detection model. The hard bend state detection model can then output whether a hard bend exists and its location. This invention can efficiently and accurately achieve intelligent detection of the contact wire's hard bend state using deep learning target detection methods, greatly improving the efficiency, accuracy, and flexibility of detection. Attached Figure Description

[0028] Figure 1 This is a schematic diagram illustrating the implementation process of the contact wire hard bend state detection method based on deep learning in this embodiment.

[0029] Figure 2 This is a schematic diagram of the system structure for detecting the hard bend state of the contact wire constructed in a specific application embodiment of the present invention.

[0030] Figure 3 This is a schematic diagram of the implementation process of the offline training phase in a specific application embodiment of the present invention.

[0031] Figure 4 This is a schematic diagram of the implementation process of the online phase in a specific application embodiment of the present invention. Detailed Implementation

[0032] The present invention will be further described below with reference to the accompanying drawings and specific preferred embodiments, but this does not limit the scope of protection of the present invention.

[0033] like Figure 1 As shown, the steps of the contact wire hard bend state detection method based on deep learning in this embodiment include:

[0034] Step S01. Acquire detection images containing pantograph and contact wire captured in real time during train operation; Step S02. Locate the pantograph area from the detection images, and determine the contact wire extraction area based on the pantograph area; Step S03. Extract the contact wire from the contact wire extraction area to obtain a contact wire image;

[0035] Step S04. Input the extracted contact wire image into the pre-trained hard bend state detection model to obtain the detection results of whether the contact wire has a hard bend and when a hard bend exists. The hard bend state detection model is obtained by pre-training a deep learning model using an image dataset. Each image in the image dataset contains annotation information of the pantograph, contact wire, and the hard bend area of ​​the contact wire with a hard bend.

[0036] This embodiment pre-trains a hard bend detection model based on a deep learning model. After acquiring real-time detection images of the pantograph and contact wire, the pantograph area is first located, and then the contact wire extraction area is determined using the pantograph area location. The real-time contact wire image is then extracted and input into the trained hard bend detection model. The hard bend detection model can then output the existence of a hard bend and the coordinate information of the area where the hard bend is located. This method can efficiently and accurately detect the hard bend state of the contact wire using deep learning target detection, greatly improving the efficiency, accuracy, and flexibility of the detection.

[0037] This embodiment divides the entire detection process into two parts: an offline training stage and an online detection stage. In the offline training stage, an image dataset is first constructed. This dataset includes two types: normal images without hard bends in the contact wire and faulty images containing hard bends. The contact wire is extracted from each image in the dataset. If a hard bend exists on the contact wire, the hard bend area is labeled to form hard bend labeling information. The image dataset with hard bend labeling information is used as the training set and input into a deep learning model for training. After training, a hard bend detection model capable of identifying whether the contact wire has a hard bend is obtained. In the online detection stage, for the real-time acquired detection images, the pantograph area is first located. Based on the pantograph area, the contact wire extraction area is determined, which narrows the extraction range, reduces data processing volume, and improves extraction accuracy. The contact wire is then extracted from the extraction area and input into the trained hard bend detection model for hard bend detection. This utilizes deep learning object detection to detect hard bends in the contact wire, improving the hard bend detection rate and accuracy.

[0038] In specific application embodiments, such as Figure 2 As shown, a visible light camera can be installed on the top of the train to image the pantograph and contact wire during train operation. The image data is then transmitted to a server via a network. On the server, the contact wire's hard bend condition is detected using the method described above, allowing for real-time detection of any hard bend faults in the contact wire. It is understood that other types of image acquisition equipment can also be used for image acquisition, and the arrangement of these devices can be configured according to actual needs.

[0039] In this embodiment, locating the pantograph region from the detection image and determining the contact wire extraction region based on the pantograph region includes: matching a standard template containing pantographs of different vehicle models with the detection image to locate the pantograph region; and using a designated area above the pantograph region as the contact wire extraction region. The contact wire is located above the pantograph, meaning the positional relationship between the contact wire and the pantograph is fixed. The pantograph is relatively easy to identify in the image. This embodiment adjusts the image acquisition range to simultaneously acquire images of both the pantograph and the contact wire. The pantograph region is first located using image matching, and then the approximate location of the contact wire is determined from the pantograph region. This allows for narrowing the contact wire detection range by utilizing the pantograph's location, improving detection efficiency and accuracy. For example, a designated area above the pantograph region can be used as the contact wire extraction region to reduce the extraction area. The specific range of the contact wire extraction region can also be determined using the possible line width and contrast range of the contact wire, allowing for more precise determination of the extraction region while ensuring extraction accuracy and minimizing data processing.

[0040] In this embodiment, extracting the contact line from the contact line extraction area to obtain the contact line image includes: extracting all lines from the contact line extraction area, merging lines with similar slopes and close distances, filtering the merged lines according to their characteristics to obtain filtered lines, and finally extracting the contact line above the pantograph from the intersection state of each line with the pantograph in the filtered lines. After extracting all lines from the contact wire extraction area, multiple irrelevant interference lines may exist. If the slopes are similar and the distances are close, it indicates a high probability of detection error during the detection of the same line. Therefore, lines with similar slopes and close distances can be merged. Since the contact wire has specific line characteristics, these characteristics can be used to filter out interference lines that do not conform to the contact wire's characteristics. Furthermore, contact wire faults with a sharp bend at the center of the pantograph require special attention. In the image, this is represented by the contact wire intersecting the pantograph at its center. This characteristic can be used to further evaluate the filtered lines and ultimately select the desired contact wire. This embodiment, following the above method, fully utilizes the characteristics of the contact wire and its positional relationship with the pantograph to achieve accurate contact wire detection, further ensuring the accuracy of contact wire detection.

[0041] The complex nature of the pantograph-catenary system makes contact wire extraction prone to errors, such as misidentifying catenary cables as contact wires. This embodiment utilizes line features to filter extracted lines. These features include line length, line angle, line curvature, line brightness, line width, and line grayscale, comprehensively leveraging multiple line features for contact wire extraction and filtering. This ensures more accurate contact wire extraction, improves extraction accuracy, and avoids the problem of misidentifying catenary cables as contact wires.

[0042] In a specific application embodiment, when performing contact wire detection, lines with similar slopes and close distances are merged. Then, all lines are further divided into line elements and circle elements. Lines with excessive curvature (exceeding a preset range) are discarded. The merged lines are then filtered based on line length, removing those that do not meet the length requirements. After this operation, several lines may still meet the requirements. These lines are then filtered using the coordinates of their intersection points with the upper edge of the pantograph. If the intersection point is located within the horn area on either side of the pantograph, that line is discarded. Simultaneously, the tilt angle information is used to discard relatively horizontal lines with small tilt angles. Further, the remaining lines are filtered again. By using a weighted index of line width and line grayscale, the line with the widest width and brightest grayscale is selected as the contact wire above the pantograph. The judgment steps for each of the above line features can be flexibly adjusted according to actual needs. There is no limitation on the order of the judgment steps. The key is to quickly screen out the contact wire by combining multiple line features and the positional relationship between the contact wire and the pantograph.

[0043] Since hard bends are relatively minor faults, similar textures can appear in other areas outside the contact line, potentially causing difficulties in subsequent training and detection. To reduce the difficulty of subsequent training and hard bend detection, this embodiment, after extracting the contact line image from the contact line extraction region, also includes processing the background of the extracted contact line. This is achieved by opening a rotating rectangular window in the neighborhood of the contact line, with the direction aligned with the contact line direction. Pixels outside the rotating rectangular window are set to specified values, while pixel values ​​inside the rotating rectangular window remain unchanged. Since other areas outside the contact line exhibit similar characteristics to hard bends, these areas can easily interfere with hard bend detection. This embodiment addresses this by setting a rotating rectangular window in the neighborhood of the contact line, reconfiguring pixels outside the rotating rectangular window, retaining the grayscale values ​​within the neighborhood of the contact line, and resetting the grayscale values ​​of other areas to specified grayscale constants. This achieves contact line background filtering, allowing preprocessing of backgrounds that can easily cause false detections of hard bends. This reduces interference from non-contact line areas with the detection results, accelerates training convergence, improves the real-time detection detection rate, and reduces the false detection rate of hard bends in the contact line.

[0044] In this embodiment, after obtaining the detection result of whether the contact line has a hard bend, a processing area of ​​a specified size is opened in the upper left and lower right corners of the detection frame, and the gray values ​​of the processing areas are statistically analyzed. The current detection result is verified based on the gray values ​​in the upper left and lower right corners. If there are uneven lighting or overexposure during image acquisition, the contact line may form a false bend, resulting in false detection. In this embodiment, after obtaining the detection result of the hard bend state, the gray values ​​of the detection result are further statistically analyzed. The statistical results are used to determine whether there is overexposure or uneven lighting. For example, if the statistical value is greater than a specified threshold (the gray value is too large), it is considered that there is overexposure or other factors causing false detection. This can avoid false alarms caused by uneven lighting or overexposure, thereby reducing the interference of lighting factors on hard bend detection.

[0045] In this embodiment, after obtaining the detection results of whether the contact wire has a hard bend and the hard bend area when it does, the method further includes extracting the skeleton points within the contact wire area and calculating the distance from the skeleton points to the center point of the detection result. The current detection result is then verified based on the calculated distance. Sudden overlap of the dropper clamp or contact wire with other wires in a certain area can easily lead to false alarms. This embodiment uses the distance between the skeleton points and the center coordinates of the hard bend result to filter interfering results, thereby reducing the interference of dropper clamps, contact wires, and other wires overlapping on hard bend detection.

[0046] Specifically, this embodiment detects the hard bend condition of the contact wire by using a visible light camera installed on the top of the train to capture images of the pantograph and contact wire above the train. The contact wire is extracted from the area above the pantograph by identifying the pantograph. During the contact wire extraction stage, features such as wire length, angle, curvature, brightness, width, and the coordinates of the intersection point between the wire and the pantograph are comprehensively utilized for extraction and filtering. After extraction, a rectangular window with the same rotation direction as the contact wire is opened in the neighborhood of the contact wire in the image. The grayscale values ​​of pixels outside the window are set to a specified constant. The detected contact wire image is then fed into an offline, pre-trained hard bend detection model for hard bend detection, yielding preliminary hard bend detection results. Further filtering of false detections in the preliminary results is performed using grayscale value statistics and constraints such as the distance between skeleton points, ultimately resulting in accurate hard bend detection results. This method filters out the vast majority of false detections, thus detecting genuine hard bend faults.

[0047] In this embodiment, the method for extracting the contact line from the contact line hard bend image dataset during the offline training phase can be the same as that used in the online phase. That is, the contact line is extracted in the area above the pantograph by identifying the pantograph. During the contact line extraction phase, features such as line length, line angle, line curvature, line brightness, line width, and the coordinates of the intersection point between the line and the pantograph are comprehensively used to extract and filter the contact line. After the contact line extraction is completed, a rectangular window with the same rotation direction as the contact line is opened in the neighborhood of the contact line in the image, and the gray value of the pixels outside the window is set to a specified constant.

[0048] The following describes the invention further using the implementation of contact wire hard bend detection in a specific application embodiment as an example. The detailed process is as follows:

[0049] I. Offline Training Phase

[0050] like Figure 3 As shown, the detailed execution steps of the offline training phase are as follows:

[0051] Step 1. Offline collection of contact wire hard bend data samples. In order to improve the detection rate of contact wire hard bend detection and reduce the false detection rate, a large number of contact wire hard bend data samples are collected in advance to form a hard bend image dataset.

[0052] Step 2. Offline template creation for the pantograph in the image. To locate the pantograph using template matching, a template for the pantograph of each vehicle model is created in advance and saved offline.

[0053] Step 3. Pantograph localization for offline samples. After completing the offline template creation, for each hard bend data sample in the offline hard bend image dataset, the pantograph template is used for matching to complete the pantograph localization.

[0054] Step 4. Contact Line Detection for Offline Samples. After completing the pantograph positioning for offline sample data, the area above the pantograph is obtained. Using the possible line width and contrast range of the contact line, all lines within this area that meet the conditions are extracted. Further, using the possible line width and contrast range of the contact line, all lines within this area are extracted. All lines are then segmented into line elements and circle elements. Lines with excessive curvature are removed, and lines with similar slopes and close distances are merged. The merged lines are then filtered using line length, removing lines that do not meet the length requirement. Additionally, the lines are filtered using the coordinates of their intersection points with the upper edge of the pantograph, removing lines whose intersection points are located in the horn-like areas on either side of the pantograph. Simultaneously, relatively horizontal lines are removed using the line's tilt angle information. Finally, by using a weighted index of line width and line grayscale, the widest and brightest line is selected as the contact line above the pantograph.

[0055] Step 5. Contact line background processing for offline samples. After the contact line is extracted, the background of the contact line is processed. A rotating rectangular window with the same direction as the contact line is opened in the neighborhood of the contact line. Pixels inside the window retain their original pixel values, while pixels outside the window have their pixel values ​​set to a fixed constant to reduce the interference of pixels outside the window on subsequent training and detection, resulting in a contact line image with a clean background.

[0056] Step 6. Offline sample hard bend data annotation. Data annotation software is used to annotate the hard bend areas of the contact line in the image and generate corresponding label files.

[0057] Step 7. Training offline samples. Using the clean contact line images and label information generated above, a deep learning model is trained. After reaching the specified training epochs or accuracy, the model's weight file is generated, thus obtaining the required hard bend state detection model.

[0058] At this point, the offline phase of the contact wire hard bend detection has been completed.

[0059] II. Online Real-Time Stage

[0060] like Figure 4 As shown, the detailed execution steps of the online detection phase are as follows:

[0061] Step 1. Load the offline pantograph template and the weights of the offline-trained contact wire hard bend detection model in real time. Online detection requires the weights of the offline pantograph template and hard bend detection model, so these two pieces of information need to be loaded in advance.

[0062] Step 2. Acquire real-time images. Acquire real-time visible light images of the pantograph catenary taken at the pantograph raising end, or visible light images from fault data packets transmitted in real-time from the pantograph raising end line.

[0063] Step 3. Online pantograph localization. Using an offline pantograph template, the pantograph in the real-time image is matched to the template to achieve pantograph localization.

[0064] Step 4. Contact line detection in real-time images. After locating the pantograph in the real-time image, the area above the pantograph is obtained. Using the possible line width and contrast range of the contact line, all lines within this area that meet the conditions are extracted. All lines are then segmented into line elements and circle elements. Lines with excessive curvature are removed, and lines with similar slopes and close distances are merged. The merged lines are then filtered using line length, removing lines that do not meet the length requirement. Additionally, the intersection coordinates of the lines with the upper edge of the pantograph are used to filter lines, removing those whose intersections are located on either side of the pantograph. Simultaneously, the tilt angle information is used to remove relatively horizontal lines. Finally, by using a weighted index of line width and line grayscale, the widest and brightest line is selected as the contact line above the pantograph. At this point, the contact line in the real-time image has been successfully obtained.

[0065] Step 5. Contact line background processing in real-time images. Since similar features to hard bends exist in other areas outside the contact line in real-time images, these areas can cause false positives in hard bend detection. To improve the detection rate of hard bends, reduce the false positive rate, and maintain consistency with the processing method of the sample images used during training, this solution requires processing the contact line background to improve the algorithm's detection performance. After completing the contact line extraction in Step 4, a rotating rectangular window with the same direction as the contact line is opened in the neighborhood of the contact line. Pixels within the window retain their original pixel values ​​from the image, while pixels outside the window have their pixel values ​​set to a fixed constant to reduce interference from pixels outside the window on subsequent contact line hard bend detection.

[0066] Step 6. Real-time image detection of contact wire hard bend. After completing Step 5, input the image generated into the trained hard bend state detection model to obtain preliminary results of contact wire hard bend detection.

[0067] Step 7. Post-processing and filtering of real-time results. A small area is created in the upper left and lower right corners of the hard bend detection result frame, and the grayscale values ​​within these areas are statistically analyzed. If the statistical values ​​in the upper left and lower right corners exceed a specified threshold, it is considered that overexposure or other factors are causing false detections, thus avoiding false detections due to uneven lighting or overexposure. Further, the skeleton points within the contact line area are extracted, and the distance from the skeleton points to the center point of the detection result is calculated. If the distance is less than a specified threshold, the current detection result is considered to be a false detection caused by the overlap of the dropper clamp or contact line with other lines in a certain area, thus avoiding false detections caused by the overlap of dropper clamps or contact lines with other lines in a certain area. This completes the contact line hard bend detection.

[0068] It is understandable that in other embodiments, the offline training phase may employ data processing methods that are not entirely the same as those in the online phase. The focus of the offline training phase is on building the training dataset and training the hard bend detection model, while the online phase utilizes the trained hard bend detection model for real-time detection. For example, to reduce data processing complexity, the pantograph can be located without performing pantograph localization in the offline training phase; instead, the contact wire can be extracted directly, or other types of prior contact wire information (position, features, etc.) can be used to assist in contact wire extraction. The specific extraction method can be configured according to actual needs.

[0069] This embodiment also provides a contact wire hard bend state detection device based on deep learning, including:

[0070] The image acquisition module is used to acquire real-time detection images of the train, including the pantograph and contact wire, during train operation;

[0071] The positioning module is used to locate the pantograph area from the detection image and determine the contact wire extraction area based on the pantograph area;

[0072] The extraction module is used to extract the contact wire from the contact wire extraction area to obtain a contact wire image;

[0073] The hard bend recognition module is used to input the extracted contact wire image into the pre-trained hard bend state detection model to obtain the detection results of whether the contact wire has a hard bend and when a hard bend exists. The hard bend state detection model is obtained by pre-training a deep learning model using an image dataset. Each image in the image dataset contains annotation information of the pantograph, contact wire, and the hard bend area of ​​the contact wire with a hard bend.

[0074] The contact wire hard bend state detection device based on deep learning in this embodiment corresponds one-to-one with the contact wire hard bend state detection method based on deep learning described above, and will not be described in detail here.

[0075] In another embodiment, the contact wire hard bend state detection device based on deep learning of the present invention may further include a processor and a memory, wherein the memory is used to store a computer program and the processor is used to execute the computer program to perform the method as described above.

[0076] It is understood that the method described in this embodiment can be executed by a single device, such as a computer or server, or it can be applied to a distributed scenario where multiple devices cooperate to complete the task. In a distributed scenario, one of the multiple devices may execute only one or more steps of the method described in this embodiment, and the multiple devices interact to complete the method. The processor can be implemented using a general-purpose CPU, microprocessor, application-specific integrated circuit, or one or more integrated circuits, and is used to execute relevant programs to implement the method described in this embodiment. The memory can be implemented using read-only memory (ROM), random access memory (RAM), static storage devices, and dynamic storage devices. The memory can store the operating system and other applications. When the method described in this embodiment is implemented through software or firmware, the relevant program code is stored in the memory and called and executed by the processor.

[0077] This embodiment also provides a contact wire hard bend detection system, including:

[0078] Image acquisition equipment is installed on the top of the train to acquire images of the pantograph and contact wire above the train during operation to obtain detection images;

[0079] The aforementioned hard bend detection device is used to receive detection images acquired by an image acquisition device and detect the hard bend state of the contact wire.

[0080] In specific application embodiments, such as Figure 2 As shown, the image acquisition device can be a visible light camera or similar device, and the hard bend detection device can be a software module with the hard bend detection function and loaded into the server. The server receives the detection image acquired by the image acquisition device and detects the hard bend state of the contact wire according to the above method.

[0081] This embodiment further provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described above.

[0082] Those skilled in the art will understand that the above embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create an implementation for the process. Figure 1 One or more processes and / or boxes Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process. Figure 1 One or more processes and / or boxes Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0083] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the invention. Therefore, any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention should fall within the protection scope of the present invention.

Claims

1. A method for detecting the hard bend state of contact wires based on deep learning, characterized in that the steps include... include: Acquire real-time detection images of the train, including the pantograph and contact wire, during train operation; The pantograph region is located from the detected image, and the contact wire extraction region is determined based on the pantograph region; A contact line image is obtained by extracting the contact line from the contact line extraction area; The extracted contact wire image is input into a pre-trained hard bend state detection model to obtain the detection results of whether the contact wire has a hard bend and when a hard bend exists. The hard bend state detection model is obtained by training a deep learning model using an image dataset in advance. Each image data in the image dataset contains annotation information of the pantograph, the contact wire, and the hard bend area of ​​the contact wire with a hard bend. After obtaining the detection result of whether the contact line has a hard bend, the method further includes opening a processing area of ​​a specified size in the upper left corner and the lower right corner of the detection frame, and statistically analyzing the gray values ​​of the processing areas. The current detection result is verified based on the gray values ​​in the upper left corner and the lower right corner. After obtaining the detection result of whether the contact line has a hard bend and the detection result of the hard bend area when a hard bend exists, the method further includes extracting the skeleton points in the contact line area, calculating the distance from the skeleton points to the center point of the detection result, and verifying the current detection result based on the calculated distance.

2. The method for detecting the hard bend state of contact wire based on deep learning according to claim 1, characterized in that, The step of locating the pantograph region from the detection image and determining the contact wire extraction region based on the pantograph region includes: matching a standard template containing pantographs of different vehicle models with the detection image to locate the pantograph region, and using a specified area above the pantograph region as the contact wire extraction region.

3. The method for detecting the hard bend state of contact wire based on deep learning according to claim 1, characterized in that, The step of extracting the contact line image from the contact line extraction area includes: extracting all lines from the contact line extraction area, merging lines with similar slopes and close distances, filtering the merged lines according to their characteristics to obtain filtered lines, and finally extracting the contact line above the pantograph from the intersection state of each line with the pantograph in the filtered lines.

4. The deep learning-based contact wire hard bend detection method according to claim 3, characterized in that, The line features include any one or more of the following: line length, line angle, line curvature, line brightness, line width, and line grayscale.

5. The deep learning-based contact wire hard bend detection method according to claim 3, characterized in that, After removing lines from the filtered lines whose intersection with the upper edge of the pantograph is located in the horn area on both sides of the pantograph, the line with the widest line width and the brightest line gray level is selected as the extracted contact line.

6. The deep learning-based contact wire hard bend detection method according to any one of claims 1 to 5, characterized in that, After extracting the contact line from the contact line extraction area to obtain the contact line image, the process further includes processing the background of the extracted contact line by opening a rotating rectangular window in the neighborhood area of ​​the contact line with the direction of the contact line, and setting the pixels outside the rotating rectangular window to a specified value.

7. A contact wire hard bend state detection device based on deep learning, characterized in that, include: The image acquisition module is used to acquire real-time detection images of the train, including the pantograph and contact wire, during train operation; The positioning module is used to locate the pantograph area from the detected image and determine the contact wire extraction area based on the pantograph area; The extraction module is used to extract the contact line from the contact line extraction area to obtain a contact line image; The hard bend recognition module is used to input the extracted contact wire image into a pre-trained hard bend state detection model to obtain whether there is a hard bend in the contact wire and the detection result of the hard bend area when there is a hard bend. The hard bend state detection model is obtained by training a deep learning model in advance using an image dataset. Each image data in the image dataset contains annotation information of the pantograph, the contact wire, and the hard bend area of ​​the contact wire where there is a hard bend. After obtaining the detection result of whether the contact line has a hard bend, the hard bend recognition module further opens a processing area of ​​a specified size in the upper left and lower right corners of the detection frame, and performs statistical analysis on the grayscale values ​​of the processing areas. The current detection result is verified based on the grayscale values ​​in the upper left and lower right corners. After obtaining the detection result of whether the contact line has a hard bend and the hard bend area when a hard bend exists, the module further extracts the skeleton points in the contact line area, calculates the distance from the skeleton points to the center point of the detection result, and verifies the current detection result based on the calculated distance.

8. A contact wire hard bend state detection device based on deep learning, comprising a processor and a memory, wherein the memory is used to store a computer program, characterized in that, The processor is used to execute the computer program to perform the method as described in any one of claims 1 to 6.

9. A contact wire hard bend detection system, characterized in that, include: Image acquisition equipment is installed on the top of the train to acquire images of the pantograph and contact wire above the train during operation to obtain detection images; The hard bend state detection device as described in claim 7 or 8 is used to receive detection images acquired by an image acquisition device and detect the hard bend state of the contact wire.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 6.

Images