Overhead wire fitting abnormality detection device and overhead wire fitting abnormality detection method

The device improves the accuracy and cost-effectiveness of detecting abnormalities in overhead line fittings by employing a multi-stage detection method with separate equipment and part models, addressing the limitations of existing technologies in handling small targets and reducing computational and annotation costs.

WO2024262595A1PCT designated stage expired Publication Date: 2024-12-26MEIDENSHA CORP

Patent Information

Application Number
PCT/JP2024/022484
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-06-23
Filing Date
2024-06-21
Publication Date
2024-12-26

AI Technical Summary

Technical Problem

Existing methods for detecting abnormalities in overhead line fittings, such as those used in electrical railways, face challenges with low accuracy when the detection target is small, and are costly due to high annotation and learning requirements, with issues related to semantic segmentation and information loss during resizing.

Method used

A device and method that utilize a learning processing unit to detect equipment and parts using separate detection models, with an abnormality detection model determining abnormalities based on restoration error, and displaying results for operator review, allowing for improved accuracy without high computational or annotation costs.

Benefits of technology

Enhances detection accuracy of abnormalities in overhead line fittings, particularly for small targets, while reducing costs and avoiding issues related to semantic segmentation and information loss, by using a multi-stage detection approach with deep learning models.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure JP2024022484_26122024_PF_FP_ABST
    Figure JP2024022484_26122024_PF_FP_ABST
Patent Text Reader

Abstract

Provided is an overhead wire fitting abnormality detection device and an overhead wire fitting abnormality detection method capable of improving accuracy in detecting abnormalities in an overhead wire fitting at low cost even when a component to be detected is small. The overhead wire fitting abnormality detection device comprises a learning processing unit 17 that performs learning using image data generated by imaging an overhead wire fitting, and a data processing unit 15 that performs abnormality determination using the image data. The learning processing unit 17 is provided with a facility learning unit 19 that learns detection for a facility, and a component learning unit 21 that learns detection for components of the facility. In the data processing unit 15, the facility is detected according to a facility detection model learned by the facility learning unit 19, the components are detected according to a component detection model learned by the component learning unit 21 from the detected facility, and abnormalities in the facility and / or the components are determined according to an abnormality detection model trained in advance.
Need to check novelty before this filing date? Find Prior Art

Description

Overhead wire fitting abnormality detection device and overhead wire fitting abnormality detection method

[0001] The present invention relates to an apparatus and method for detecting an abnormality in a metal fitting of an overhead contact wire.

[0002] Traditionally, maintenance and inspection work on overhead line fittings, a type of electric railway equipment, has been carried out by inspectors who visually check for abnormalities in the overhead line fittings. Overhead line fittings are fittings that hold the overhead contact wires (trolley wires, suspension wires, auxiliary suspension wires; hereafter referred to as "overhead wires") that supply power to trains in electric railway equipment in a predetermined position, or electrically connect or separate multiple overhead lines. However, there are limitations to visual inspection, and there is a risk of overlooking abnormalities, so there has been a demand for automation of this inspection.

[0003] Therefore, Patent Document 1 discloses a technology in which an object is photographed with a camera mounted on a moving body, and the photographed image is used to generate an image using a generative model such as a neural network, and the similarity between the generated image and the photographed image is calculated, and if this similarity is small, it is determined that an abnormality has occurred.

[0004] Patent Literature 2 discloses an overhead line fitting anomaly detection device that performs learning for each component of the overhead line fitting to detect anomalies in order to improve the accuracy of anomaly detection. This device first acquires an aerial image of the railway vehicle using a line sensor camera mounted on the roof of the railway vehicle. Next, trained overhead line fittings are detected from the acquired image using machine learning object detection, and semantic segmentation is applied to the target equipment image to create a one-hot representation. This simplifies the image and simplifies the anomaly detection for shape anomalies. Semantic segmentation makes it possible to extract only the regions of each part of the fitting. This technology is used to generate a mask image of only the part, learn normal data for each part, and use the restoration error of the masked portion to detect anomalies.

[0005] JP 2019-133306 A JP 2022-159010 A

[0006] However, the technology disclosed in Patent Document 1 does not perform detection covering a small area, so if the part to be detected is small, it may be difficult to improve detection accuracy.

[0007] The technology disclosed in Patent Document 2 requires semantic segmentation, which poses the problem of high annotation and learning costs. Another problem is that the accuracy (error) of the segmentation affects subsequent anomaly detection. Furthermore, when an anomaly is detected, the target equipment detected must be resized so that it can be handled by the anomaly detection model. However, if the target part is small compared to the target equipment, information is lost due to resizing, making anomaly detection difficult.

[0008] Therefore, an object of the present invention is to provide an overhead line fitting abnormality detection device and an overhead line fitting abnormality detection method that can improve the accuracy of detecting abnormalities in overhead line fittings at low cost, even when the parts to be detected are small.

[0009] In order to solve the above-mentioned problems, the overhead line fitting abnormality detection device of the present invention comprises a learning processing unit that performs learning using image data generated by photographing overhead line fittings, and a data processing unit that performs abnormality judgment using the image data, wherein the learning processing unit comprises an equipment learning unit that learns to detect equipment, and a part learning unit that learns to detect parts of the equipment, and the data processing unit detects the equipment using an equipment detection model learned by the equipment learning unit, detects the parts from the detected equipment using a part detection model learned by the part learning unit, and judges abnormalities in the equipment and / or parts using a previously trained abnormality detection model.

[0010] In one aspect of the present invention, the data processing unit detects the facility based on the facility detection model, vehicle traveling position information, and facility information.

[0011] In one aspect of the present invention, the data processing unit determines an abnormality in equipment and / or components based on a restoration error of the anomaly detection model.

[0012] In one aspect of the present invention, the system further includes an input / output unit, and the detection results of the equipment and / or parts detected by the data processing unit are displayed on the input / output unit, and an operator can check and edit the detection results from the input / output unit.

[0013] The overhead line fitting abnormality detection method of the present invention includes learning using image data generated by photographing the overhead line fittings, and making an abnormality judgment using the image data, wherein the learning includes learning to detect equipment and learning to detect parts of the equipment, and making the abnormality judgment includes detecting the equipment using an equipment detection model learned by learning to detect the equipment, detecting the parts from the detected equipment using a part detection model learned by learning to detect the parts of the equipment, and judging an abnormality in the equipment and / or parts using a previously trained anomaly detection model.

[0014] In one aspect of the present invention, performing the abnormality determination includes detecting the facility based on the facility detection model, vehicle traveling position information, and facility information.

[0015] In one aspect of the present invention, performing the abnormality determination includes determining an abnormality in equipment and / or components based on a recovery error of the abnormality detection model.

[0016] One aspect of the present invention includes displaying the detection results of the equipment and / or the parts detected by performing the abnormality judgment on an input / output unit, and an operator checking and editing the detection results from the input / output unit.

[0017] An overhead line fitting abnormality detection device according to one aspect of the present invention comprises a learning processing unit that performs learning using image data generated by photographing overhead line fittings, and a data processing unit that performs abnormality judgment using the image data. The learning processing unit comprises an equipment learning unit that learns to detect equipment, and a component learning unit that learns to detect parts of the equipment. The data processing unit detects the equipment using an equipment detection model learned by the equipment learning unit, detects the parts from the detected equipment using a component detection model learned by the component learning unit, corrects the image data in which the equipment or component is detected, and judges abnormalities in the equipment and / or components using a previously trained anomaly detection model.

[0018] According to another aspect of the present invention, an overhead line fitting abnormality detection device comprises a learning processing unit that performs learning using image data generated by photographing overhead line fittings, and a data processing unit that performs abnormality judgment using the image data. The learning processing unit comprises an equipment learning unit that learns to detect equipment, and a part learning unit that learns to detect parts of the equipment. The data processing unit corrects the image data, detects the equipment using an equipment detection model learned by the equipment learning unit, detects the parts from the detected equipment using a part detection model learned by the part learning unit, and judges abnormalities in the equipment and / or parts using a previously trained anomaly detection model.

[0019] According to yet another aspect of the present invention, an overhead line fitting abnormality detection device comprises a learning processing unit that performs learning using image data generated by photographing overhead line fittings, and a data processing unit that performs abnormality judgment using the image data. The learning processing unit comprises an equipment learning unit that learns to detect equipment, and a part learning unit that learns to detect parts of the equipment. The data processing unit detects the equipment using an equipment detection model learned by the equipment learning unit, detects the parts from the detected equipment using a part detection model learned by the part learning unit, corrects the image data detecting the equipment or parts based on the time of photographing and vehicle traveling position information, and judges abnormalities in the equipment and / or parts using a previously trained anomaly detection model.

[0020] According to yet another aspect of the present invention, an overhead line fitting abnormality detection device comprises a learning processing unit that performs learning using image data generated by photographing overhead line fittings, and a data processing unit that performs abnormality judgment using the image data. The learning processing unit comprises an equipment learning unit that learns to detect equipment, and a component learning unit that learns to detect parts of the equipment. The data processing unit detects the equipment using an equipment detection model learned by the equipment learning unit, detects the parts from the detected equipment using a component detection model learned by the component learning unit, corrects the image data based on brightness information or identification AI of the image data in which the equipment or parts are detected, and judges abnormalities in the equipment and / or components using a previously trained anomaly detection model.

[0021] In one aspect of the present invention, the data processing unit performs an inversion process on the image data based on the position of the detected object in the image data before determining whether there is an abnormality in the equipment and / or parts using the pre-trained anomaly detection model.

[0022] A method for detecting abnormalities in overhead line fittings according to one aspect of the present invention includes learning using image data generated by photographing overhead line fittings and making an abnormality judgment using the image data, wherein the learning includes learning to detect equipment and learning to detect parts of the equipment, and the making an abnormality judgment includes detecting the equipment using an equipment detection model learned by learning to detect the equipment, detecting the parts from the detected equipment using a part detection model learned by learning to detect the parts of the equipment, correcting the image data in which the equipment or the parts are detected, and judging an abnormality in the equipment and / or parts using a previously trained anomaly detection model.

[0023] A method for detecting abnormalities in overhead line fittings according to another aspect of the present invention includes performing learning using image data generated by photographing overhead line fittings and performing abnormality judgment using the image data, wherein performing the learning includes learning to detect equipment and learning to detect parts of the equipment, and performing the abnormality judgment includes correcting the image data, detecting the equipment using an equipment detection model trained by learning to detect the equipment, detecting the parts from the detected equipment using a part detection model trained by learning to detect the parts of the equipment, and judging abnormalities in the equipment and / or parts using a previously trained anomaly detection model.

[0024] A method for detecting abnormalities in overhead line fittings according to yet another aspect of the present invention includes performing learning using image data generated by photographing overhead line fittings, and performing abnormality judgment using the image data, wherein the learning includes learning to detect equipment and learning to detect parts of the equipment, and performing abnormality judgment includes detecting the equipment using an equipment detection model trained by learning to detect the equipment, detecting the parts from the detected equipment using a part detection model trained by learning to detect the parts of the equipment, correcting the image data in which the equipment or the parts have been detected based on the time of photographing and vehicle traveling position information, and judging whether there is an abnormality in the equipment and / or the parts using a previously trained anomaly detection model.

[0025] A method for detecting abnormalities in overhead line fittings according to yet another aspect of the present invention includes performing learning using image data generated by photographing overhead line fittings, and performing abnormality judgment using the image data, wherein performing the learning includes learning to detect equipment and learning to detect parts of the equipment, and performing the abnormality judgment includes detecting the equipment using an equipment detection model trained by learning to detect the equipment, detecting the parts from the detected equipment using a part detection model trained by learning to detect the parts of the equipment, correcting the image data based on brightness information or identification AI of the image data in which the equipment or the parts are detected, and judging abnormalities in the equipment and / or parts using a pre-trained anomaly detection model.

[0026] In one aspect of the present invention, performing the abnormality determination includes performing an inversion process on the image data based on the position of the detected object in the image data before determining an abnormality in the equipment and / or parts using the pre-trained abnormality detection model.

[0027] According to the present invention, it is possible to provide an overhead wire fitting abnormality detection device and an overhead wire fitting abnormality detection method that can improve the accuracy of detecting abnormalities in overhead wire fittings at low cost, even when the parts to be detected are small.

[0028] FIG. 1 is a schematic diagram showing an installation example of an overhead wire fitting abnormality detection device according to an embodiment of the present invention. FIG. 2 is a block diagram showing the configuration of an overhead wire fitting detection device according to an embodiment of the present invention. FIG. 3 is a flowchart illustrating the operation of an overhead wire fitting abnormality detection device according to an embodiment of the present invention. FIG. 4 is a flowchart illustrating the operation of equipment detection learning according to an embodiment of the present invention. FIG. 5 is a schematic diagram illustrating equipment detection results according to an embodiment of the present invention. FIG. 6 is a flowchart illustrating the operation of component detection learning according to an embodiment of the present invention. FIG. 7 is a schematic diagram illustrating component detection results according to an embodiment of the present invention. FIG. 8 is a schematic diagram illustrating image cropping and resizing results according to an embodiment of the present invention. FIG. 9 is a flowchart illustrating the operation of an abnormality detection learning according to an embodiment of the present invention. FIG. 10 is a schematic diagram illustrating abnormality detection results according to an embodiment of the present invention. FIG. 11 is a flowchart illustrating the operation of an overhead wire fitting abnormality detection device according to an embodiment of the present invention. FIG. 12 is a flowchart illustrating the operation of an overhead wire fitting abnormality detection device according to an embodiment of the present invention. Fig. 1 is a flowchart illustrating the operation of the overhead wire fitting abnormality detection device according to an embodiment of the present invention. Fig. 2 is a schematic diagram illustrating the positional relationship between a camera and the overhead wire fittings, for illustrating the image inversion processing of the overhead wire fitting abnormality detection device according to an embodiment of the present invention. Fig. 3 is a schematic diagram illustrating the image inversion processing of the overhead wire fitting abnormality detection device according to an embodiment of the present invention. Fig. 4 is a schematic diagram illustrating the image inversion processing of the overhead wire fitting abnormality detection device according to an embodiment of the present invention. Fig. 5 is a schematic diagram illustrating the image inversion processing of the overhead wire fitting abnormality detection device according to an embodiment of the present invention. Fig. 6 is a schematic diagram illustrating the relationship between the railway track and the overhead wire, for illustrating the image inversion processing of the overhead wire fitting abnormality detection device according to an embodiment of the present invention.1 is a flowchart illustrating the operation of the overhead wire fitting abnormality detection device according to the embodiment of the present invention.

[0029] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. A. First Embodiment Fig. 1 is a schematic diagram showing an example of installation of an overhead line fitting abnormality detection device 13 according to a first embodiment of the present invention. Fig. 1 shows a vehicle 1, which is a railway vehicle, a railway track 2 on which the vehicle 1 runs, and an overhead line fitting 3, which is the target of abnormality detection. The vehicle 1 shown in Fig. 1 includes a camera 11, a storage device 12, an overhead line fitting abnormality detection device 13, and a user interface (input / output unit) 14.

[0030] The camera 11 is installed on the roof of the vehicle 1, photographs the overhead line fittings 3, and outputs image data. Examples of the camera 11 include an area sensor camera and a line sensor camera. Note that a lighting device (not shown) may be provided as needed to project light onto the overhead line fittings 3 for photographing with the camera 11.

[0031] The storage device 12 stores the image data output from the camera 11. The user interface 14 includes an input unit including at least one of a pointing device and a keyboard, and an output unit including a display. Here, the storage device 12 may be implemented by a storage device such as a semiconductor memory or a magnetic disk.

[0032] 2 is a block diagram showing the configuration of the overhead line fitting abnormality detection device 13 according to this embodiment. The overhead line fitting abnormality detection device 13 includes a data processing unit 15 having object detection and abnormality determination functions and a learning processing unit 17 that learns the object detection and abnormality determination model. The device detects abnormalities in the overhead line fittings 3 based on image data stored in the storage device 12. The learning processing unit 17 includes an equipment learning unit 19 that learns to detect equipment and a component learning unit 21 that learns to detect equipment components. Based on the image data stored in the storage device 12, the equipment learning unit 19 generates an equipment detection model and the component learning unit 21 generates a component detection model through deep learning. The data processing unit detects equipment using the equipment detection model learned by the equipment learning unit, detects components from the detected equipment using the component detection model learned by the component learning unit, and determines abnormalities in the equipment and / or components using the previously trained anomaly detection model. The overhead line fitting abnormality detection device 13 may be configured as an information processing device such as a personal computer.

[0033] 3 is a flowchart illustrating the operation of the overhead wire fitting abnormality detection device 13 according to this embodiment. The operation of the overhead wire fitting abnormality detection device 13 will be described using this flowchart.

[0034] <Image Input> The camera 11 mounted on the roof of the vehicle 1 takes an image of the overhead line fitting 3 (FIG. 3, S001). <Saving in Storage Device> The image data captured by the camera 11 is stored in the storage device 12 (FIG. 3, S002).

[0035] <Equipment Detection> Target equipment is detected using an equipment detection model from the equipment detection model database M01 ( FIG. 3 , S003). The equipment detection model has undergone deep learning in advance. FIG. 4 is a flowchart showing the equipment detection model learning operation executed by the equipment learning unit 19. FIG. 5 is a schematic diagram showing the results of target equipment detection using the equipment detection model MS. As shown in FIG. 4 , the equipment learning unit 19 first reads image data from the traveling image dataset D03 ( FIG. 4 , S101). Next, equipment detection learning is performed using data from the equipment annotation database D04, including both position information (coordinates of the four vertices of the rectangular area) surrounding the target equipment in the image data and a correct label ( FIG. 4 , S102). The deep-learned equipment detection model is stored in the equipment detection model database M01 ( FIG. 4 , S103).

[0036] 5 illustrates an example of equipment detection using an equipment detection model MS that has been trained in advance by deep learning. The equipment detection model MS receives an input image G1 and outputs an equipment detection result G2. The equipment detection result includes "rectangular coordinate information of the equipment" as "rectangular coordinate information in image data" indicated by dashed line A in the equipment detection result G2, and an "equipment label" corresponding to the type of equipment as "label information." <Equipment Detection Result Determination> Next, if equipment is detected, the process proceeds to a part presence determination (FIG. 3, S004, YES). If equipment is not detected, the process proceeds to a shooting end condition determination (FIG. 3, S004, NO).

[0037] <Determining whether a part exists> When equipment is detected during equipment detection (FIG. 3, S003), a determination is made as to whether a target part exists in the detected target equipment based on the "equipment label" and the inspection item database D01 (FIG. 3, S005). Here, the inspection item database D01 lists whether a target part exists in each equipment label. If a target part exists, the process proceeds to part detection processing (FIG. 3, S005, YES); if not, the process proceeds to image cropping and resizing processing (FIG. 3, S005, NO).

[0038] <Part Detection> Part detection involves object detection limited to the rectangular area detected by equipment detection ( FIG. 3 , S003). At this time, part detection is performed using a part detection model from the part detection model database M02 for each piece of target equipment where a target part exists ( FIG. 3 , S006). The part detection model has undergone deep learning in advance. FIG. 6 is a flowchart illustrating the part detection model learning operation executed by the part learning unit 21. FIG. 7 is a schematic diagram illustrating the results of target part detection using the part detection model MB. As shown in FIG. 6 , the part learning unit 21 first reads image data from the equipment image dataset D05 ( FIG. 6 , S201). Next, part detection learning is performed using data from the part annotation database D06, including both position information (coordinates of the four vertices of the rectangular area) surrounding the target part in the image data and a correct label ( FIG. 6 , S202). The deep-learned part detection model is stored in the part detection model database M02 ( FIG. 6 , S203). Next, if learning of all equipment parts has not been completed (FIG. 6, S204, NO), the same flow is repeated again from image input S201. If learning of all equipment parts has been completed (FIG. 6, S204, YES), the learning operation ends.

[0039] FIG. 7 illustrates an example of equipment detection using a part detection model MB that has been trained in advance through deep learning. The part detection model MB receives an input image G3 and outputs a part detection result G4. The part detection result includes multiple pieces of "rectangular coordinate information of parts" indicated by dashed lines B in the detection result G2 as "rectangular coordinate information in image data," and "part labels" corresponding to the type of part as "label information." The detection results retain both the equipment and part detection results, including the "equipment rectangular coordinate information" and "equipment labels" carried over from equipment detection, and the "part rectangular coordinate information" and "part labels" detected in part detection.

[0040] <Image cropping and resizing> An image is cropped from the image data based on the "rectangular coordinate information of the equipment" indicated by dashed line A in Fig. 5 or the "rectangular coordinate information of the component" indicated by dashed line B in Fig. 7, and resized to an image size that can be handled by the anomaly detection model (Fig. 3, S007). Fig. 8 is a schematic diagram showing an example of the results of the image cropping and resizing process. Fig. 8 illustrates an example of image cropping and resizing for the rectangular coordinate area indicated by dashed line A for the equipment detection result. The cropped image is resized to a square for object detection. The "rectangular coordinate information in the image data," "label information," and "resized cropped image" are sent to subsequent processing.

[0041] <Calculation of Abnormality Level> The anomaly detection model for each piece of equipment or part that has been previously trained is read from the anomaly detection model database M03 and applied to the resized cropped image to calculate the anomaly level ( FIG. 3 , S008). The anomaly detection model has undergone deep learning in advance. FIG. 9 is a flowchart showing the learning operation of the anomaly detection model executed by the equipment learning unit 19 and the part learning unit 21. FIG. 10 is a schematic diagram showing the results of anomaly level calculation using the anomaly detection model MK. As shown in FIG. 9 , the equipment learning unit 19 and the part learning unit 21 first read image data from their respective anomaly detection image datasets D07 ( FIG. 9 , S301). Anomaly detection learning is performed from the read images ( FIG. 9 , S302). The deep-learned anomaly detection model is saved in the anomaly detection model database M03 ( FIG. 9 , S303). Next, if anomaly detection learning has not been completed for all equipment and parts ( FIG. 9 , S304, NO), the same flow is repeated again from image input S301. When abnormality detection learning for all equipment and components is completed (FIG. 9, S304, YES), the learning operation ends.

[0042] FIG. 10 illustrates an example of an anomaly detection operation using an anomaly detection model MK that has been trained in advance through deep learning. The anomaly detection model MK receives input images G5 and G6 and outputs a part detection result G7. Here, G5 is an example of normal data, and G6 is an example of abnormal data. In this embodiment, a distance-based anomaly detection model using multidimensional features is illustrated as the anomaly detection model. The anomaly detection model used here may be supervised or unsupervised as training data, as long as it outputs multidimensional features. However, in the case of supervised training data, it is necessary to annotate the data as normal or abnormal in advance. The output multidimensional features are plotted in a latent variable space, and the degree of anomaly is calculated based on the distance from the normal center. Abnormal data is plotted at a position relatively far from the normal center.

[0043] <Abnormality Determination> The abnormality degree calculated as a distance in the latent variable space is used to read the abnormality determination threshold set for each target equipment and target component from the anomaly detection threshold database D02 created in advance, and an abnormality determination is performed using the threshold ( FIG. 3 , S009). As described above, when the output is a multidimensional feature, the abnormality degree is calculated based on the distance from the normal center. If the abnormality degree is equal to or greater than the threshold ( FIG. 3 , S009, YES), an abnormality flag is assigned ( FIG. 3 , S010). If the abnormality degree is not equal to or greater than the threshold ( FIG. 3 , S009, NO), a normality flag is assigned ( FIG. 3 , S011). After each flag is assigned, the abnormality detection result is saved ( FIG. 3 , S012). The information saved here is "rectangle coordinate information," "label information," and "abnormality determination result." <Image Capture Completion Determination> Finally, it is determined whether processing has been completed for all images ( FIG. 3 , S013). If the photographing is not completed (FIG. 3, S013, NO), the series of steps from image input S001 are repeated. If the photographing is completed (FIG. 3, S013, YES), the process ends.

[0044] In this processing flow, examples of deep learning techniques using detection models for equipment detection (FIG. 3, S004) and part detection (FIG. 3, S006) include, but are not limited to, SSD (Single Shot Multibox Detector).

[0045] As described above, in this embodiment, equipment detection is performed using an equipment detection model, and then component detection is performed for the detected equipment using a component detection model for each piece of equipment. The degree of anomaly is calculated for each detected piece of equipment and component using its own anomaly detection model. Therefore, anomaly detection can be performed not only for equipment in relatively large areas but also for components in small areas, improving the accuracy of anomaly detection. By performing object detection in stages, first for equipment detection and then for component detection, it is possible to detect relatively small areas without using semantic segmentation. Furthermore, by employing object detection with low computational and annotation costs for target component detection, it is possible to reduce computational costs and improve detection accuracy. Thus, it is possible to provide an overhead line fitting anomaly detection device and an overhead line fitting anomaly detection method that can improve the accuracy of detecting anomalies in overhead line fittings at low cost, even when the target components are small.

[0046] B. Second Embodiment Figure 11 shows a flowchart of the operation of the overhead line fitting abnormality detection device 13 of this embodiment. This embodiment differs from the first embodiment in that equipment installation location information (equipment information) and vehicle running location information F2 from an equipment database F1 are added to equipment detection, and equipment installation location information and vehicle running location information F4 from an equipment database F3 are added to part detection. Other than this, the operations and functions are the same, so the same reference numerals are used in the drawings and their descriptions are omitted. Adding vehicle running information and equipment installation information to equipment detection and part detection means, for example, that it is possible to determine whether equipment is installed based on vehicle location information in a captured image. This not only achieves the effects of the first embodiment, but also improves detection accuracy by suppressing overdetection, which is the determination that equipment or parts are present when they are not, and by ensuring detection at running locations where equipment or parts are installed.

[0047] C. Third Embodiment FIG. 12 shows a flowchart of the operation of the overhead line fitting anomaly detection device 13 of this embodiment. FIG. 13 shows a schematic diagram illustrating anomaly detection operation using the restoration error-based anomaly detection model MA of this embodiment. This embodiment differs from the first embodiment in that the restoration error calculated from an image restored by an autoencoder or the like and an input image is used to determine the degree of anomaly (S409 in FIG. 12). Since the remaining operations and functions are the same, the same reference numerals are used in the drawings and their descriptions are omitted. As shown in FIG. 13, the anomaly detection model MA of this embodiment is composed of an encoder and a decoder, inputs images G8 and G9, and outputs restored images G10 and G11. It then compares the differences between the input images G8 and G9 and the restored images G10 and G11 and outputs the results. For example, the difference comparison is performed by calculating the difference in RGB values ​​for each pixel, and the sum of these values ​​is used as an index of the degree of anomaly. Therefore, in addition to the effects of the first embodiment, the anomaly degree calculation method of this embodiment can be applied based on the target to be detected, thereby improving detection accuracy.

[0048] D. Fourth Embodiment Figure 14 shows a flowchart of the operation of the overhead line fitting abnormality detection device 13 of this embodiment. This embodiment differs from the third embodiment in that equipment installation location information (equipment information) and vehicle running position information F2 from an equipment database F1 are added to equipment detection, and equipment installation location information and vehicle running position information F4 from an equipment database F3 are added to part detection. Other than this, the operations and functions are the same, so the same reference numerals are used in the drawings and their descriptions are omitted. Adding vehicle running information and equipment installation information to equipment detection and part detection means, for example, that it is possible to determine whether equipment is installed based on vehicle position information in a captured image. This not only achieves the effects of the third embodiment, but also improves detection accuracy by suppressing overdetection, which is the determination that equipment or parts are present when they are not, and by ensuring detection at running positions where equipment or parts are installed.

[0049] E. Fifth Embodiment FIG. 15 shows a flowchart of the operation of the overhead line fitting abnormality detection device 13 of this embodiment. This embodiment differs from the first embodiment in that, after equipment detection (FIG. 15, S003) or component detection (FIG. 15, S006), the detection results are displayed and edited by the operator. Other than this, the operations and functions are the same, so the same reference numerals are used in the drawings and their description is omitted. In the operation of this embodiment, for example, the detection results are displayed on the display of the user interface (input / output unit) 14. The operator checks the detection results displayed on the display, and if an erroneous detection is found, manually corrects the rectangle position and / or label using the pointing device and / or keyboard of the user interface (input / output unit) 14 (FIG. 15, S501). This not only achieves the effects of the first embodiment, but also corrects the image input for abnormality determination, enabling highly accurate abnormality determination. The configuration of this embodiment, in which the operator manually edits the detection results, can also be applied to the second, third, and fourth embodiments, as shown in Figures 16, 17, and 18, respectively. In addition to the same effects as those of the second, third, and fourth embodiments, the images input for abnormality determination are corrected and made correct, enabling highly accurate abnormality determination to be performed.

[0050] F. Sixth Embodiment FIG. 19 shows a flowchart of the operation of the overhead line fitting abnormality detection device 13 of this embodiment. The device configuration is the same as that shown in FIG. 2 for the first embodiment. This embodiment differs from the first embodiment in that image correction processing (FIG. 19, S601) and image inversion processing (FIG. 19, S602, S603) are performed after image cropping (FIG. 19, S007a) and image resizing (FIG. 19, S007b). Note that the step represented as image cropping and resizing (FIG. 3, S007) in the flowchart of the first embodiment in FIG. 19 is represented as two steps, image cropping (FIG. 19, S007a) and image resizing (FIG. 19, S007b), but the operation is the same as in the first embodiment. Other than these, the operations and functions are the same as in the first embodiment, so the same reference numerals are used in the flowchart of FIG. 19 and their description is omitted.

[0051] The following describes the new functions added to this embodiment. <Image Correction Processing> In the image correction processing, an image is cut out based on the "equipment rectangular coordinate information" indicated by dashed line A in FIG. 5 or the "component rectangular coordinate information" indicated by dashed line B in FIG. 7 (FIG. 19, S0007a), and the image data is resized to an image size that can be handled by the anomaly detection model (FIG. 19, S007b). The image data is then corrected (FIG. 19, S601). The image correction processing includes, for example, brightness correction processing and edge enhancement processing. Any method may be used for brightness correction and edge enhancement processing. Examples of the brightness correction method include gamma correction and contrast correction, and edge enhancement methods include bilateral filtering. The corrected "corrected image" is then sent to the next image inversion processing together with the "rectangular coordinate information" and "label information."

[0052] <Image Inversion Processing> In the image inversion processing, horizontal inversion target area information is read from the horizontal inversion target area information database (FIG. 19, D20), and it is checked whether the position of the detected object in the corrected image data received from the previous image correction processing (FIG. 19, S601) belongs to the horizontal inversion target area (FIG. 19, S602). If the position of the detected object belongs to the horizontal inversion target area (FIG. 19, S601, YES), image inversion processing is performed (FIG. 19, S603), and the image-inverted image data is sent to the abnormality level calculation processing (FIG. 19, S008). If the detected position of the detected object does not belong to the horizontal inversion target area (FIG. 19, S601, NO), the image-corrected image data is sent directly to the abnormality level calculation processing (FIG. 19, S008).

[0053] Here, the image inversion process will be described in more detail. FIG. 20 is a schematic diagram illustrating the situation in which an overhead contact line fitting 3, which is an object to be inspected, is photographed in this embodiment. As shown in FIG. 20 , a railway vehicle 1 is traveling on a railway track 2. A right camera 11a and a left camera 11b are installed on the roof of the vehicle 1. The right camera 11a and the left camera 11b are installed with their optical axes facing upward so as to photograph the overhead contact line fitting 3 located above the vehicle 1. In FIG. 20 , the overhead contact line fitting 3 is illustrated as including a suspension wire 3a, a hanger 3b, and a trolley wire 3c. FIG. 21 is a plan view of the configuration including the vehicle 1, the right camera 11a, and the left camera 11b shown in FIG. 20 . Rectangular images of equipment extracted from the images photographed by the right camera 11a and the left camera 11b are shown as images R and L. The white arrow S indicates the scanning direction of the right camera 11a and the left camera 11b, which face upward. Here, the explanation focuses on the left camera, but the same situation occurs with the right camera.

[0054] Figure 22 shows image L0 captured by the left camera 11b in the positional relationship shown in Figure 21 before image cropping. Image L is cropped from image L0 by equipment detection processing. The area divided left and right by line C defines the left-right reversal target area described above. This area information is recorded in the left-right reversal target area information database (Figure 19, D20). In Figure 22, the overhead line fitting 3 is captured on the right side of line C. As shown in Figures 20 and 21, the overhead line fitting 3 is located in the upper right corner of Figure 21 when viewed from the left camera 11b. Therefore, when the camera scans in the direction of arrow S, the captured image captures the contact wire 3c, hanger 3b, and suspension wire 3a in this order from the right side of the page, as shown in Figure 22.

[0055] Fig. 23 is a plan view similar to Fig. 21, but shows that the overhead line fitting 3 is located in the upper left corner of the paper as viewed from the left camera 11b. Fig. 24 shows image L1 before cropping, captured by the left camera 11b in the positional relationship of Fig. 23. Image La is cropped from image L1 by equipment detection processing. Here, the overhead line fitting 3 is captured on the left side of the wire C. As shown in Fig. 23, the overhead line fitting 3 is located in the upper left corner of the paper as viewed from the left camera 11b. Therefore, when the camera scans in the direction of arrow S, the captured image captures the suspension wire 3a, hanger 3b, and trolley wire 3c in this order from the right of the paper, as shown in Fig. 24.

[0056] From the above considerations, the symmetry of the detection object changes depending on whether the object is photographed in the right or left region of the line C. When learning an anomaly detection model and when detecting an anomaly using the model, it is better to align the configuration of the detection object because aligning the symmetry during learning and detection can improve detection accuracy. For example, as shown in FIG. 22 , when images are aligned in the order of the trolley wire 3c, hanger 3b, and suspension wire 3a from the right side of the page, image processing is performed so that if the detection object is located in the right region of the line C, the left-right image is not inverted, but if the detection object is located in the left region of the line C, the left-right image is inverted. In this case, for the right camera 11a, the area where the detection object is located is determined and the left-right image is inverted so that the image is similarly arranged from the right side of the image as the trolley wire 3c, hanger 3b, and suspension wire 3a.

[0057] FIG. 25 is a diagram showing the positional relationship of the overhead wire 20 with respect to the railway track 2. FIG. 25 shows two railway tracks 2 and overhead wires 20, one on the left and one on the right. In the images captured by the right camera 11a and the other on the left camera 11b, the overhead wire fittings 3, which are the detection target, move left and right across the image area. This is because, as shown in FIG. 25, the overhead wire 20 is installed so as to be pulled left and right relative to the railway track 2 by the curve drawing device 23. The overhead wire 20 is arranged in this manner to prevent local wear of the pantograph, which serves as the power receiving device. That is, the overhead wire 20 is arranged in a so-called zigzag pattern to ensure that the contact frequency between the overhead wire 20 and the pantograph is uniform across the contact surface of the pantograph. Therefore, even on a running line, the overhead wire 20 may cross the central axis of the camera. Furthermore, if there are parallel railway tracks 2 (such as up and down tracks), the overhead wire equipment of the parallel railway track 2 may be detected, and these may also be captured in the reverse area. In either case, the left and right images are inverted based on whether the object to be detected is in an image inversion area, and the image inversion process is performed so that the images provided for calculating the degree of abnormality have a common configuration (the arrangement of the suspension wire, hanger, and trolley wire is constant).

[0058] The processing following the image inversion processing in this embodiment is the same as that in the first embodiment. However, the calculation of the degree of abnormality is applied to the cropped image that has undergone image correction and inversion processing, and the degree of abnormality is calculated. At this time, the pre-training model is trained using data that has undergone image correction processing and image inversion processing.

[0059] As described above, in this embodiment, image correction processing and image inversion processing are performed on the cropped and resized image data, thereby improving the accuracy of anomaly detection in addition to the effects and advantages of the first embodiment. Specifically, brightness correction processing and edge enhancement processing are performed on all data as image correction processing for the cropped and resized images after equipment detection. This makes it possible to achieve highly accurate anomaly detection even in cases where the target equipment is not clearly captured, such as in environments where low brightness is likely to occur, such as at night or inside a tunnel. Furthermore, in this embodiment, image inversion processing can align the order of detection targets in the image data used to calculate the degree of anomaly, thereby improving the accuracy of anomaly detection.

[0060] G. Seventh Embodiment Figure 26 shows a flowchart of the operation of the overhead line fitting abnormality detection device 13 of this embodiment. The device configuration is the same as that shown in Figure 2 for the first embodiment. This embodiment differs from the sixth embodiment in that image correction processing (Figure 26, S701) is performed on all captured images. Other than this, the operations and functions are the same as those of the sixth embodiment, so the same reference numerals are used in the flowchart of Figure 26 and their descriptions are omitted. In this embodiment, in addition to the functions and effects of the sixth embodiment, brightness correction processing and edge enhancement processing are added as image correction processing before equipment detection. This enables highly accurate equipment detection and abnormality detection even in cases where the target equipment is not clearly captured, such as in environments where images are likely to be captured at night or in tunnels due to low brightness.

[0061] H. Eighth Embodiment FIG. 27 shows a flowchart of the operation of the overhead wire fitting abnormality detection device 13 of this embodiment. The device configuration is the same as that shown in FIG. 2 for the first embodiment. This embodiment differs from the sixth embodiment in that, instead of the image correction process of the sixth embodiment (FIG. 19, S601), correction is performed on the cropped and resized image based on the shooting time and vehicle running position information (FIG. 27, S801, S802). Other operations and functions are the same as those of the sixth embodiment, so the same reference numerals are used in the flowchart of FIG. 27 and their descriptions are omitted. Specifically, in the sixth embodiment, image correction processing is performed on all cropped and resized image data. However, in this embodiment, based on the target shooting time and running information (position information of the current running point obtained from the vehicle), a running section that may be blurred, such as at night or inside a tunnel, is determined (determined by comparing previously acquired mileage information, such as inside a tunnel, with the current running position information), and the image data to be subjected to image correction is limited.

[0062] The functions added in this embodiment will be described below. <Determination of Blurred Section> When a cropped and resized image is supplied, the data processing unit 15 refers to the shooting time and driving position information, and if it determines based on that information that the driving section is likely to be blurred, such as at night or inside a tunnel, it sets a flag to execute image correction processing (FIG. 27, S801). If it is determined that the section is not blurred (FIG. 27, S801, NO), the process proceeds to the next left-right image flip processing (FIG. 27, S602). If it is determined that the section is blurred (FIG. 27, S801, YES), the image correction processing is performed (FIG. 27, S802). The image correction processing here is the same as S601 in FIG. 19 of the sixth embodiment. After image correction is completed, the process proceeds to the next left-right image flip processing determination (FIG. 27, S602).

[0063] In this embodiment, in addition to the effects and advantages of the sixth embodiment, brightness correction and edge enhancement are performed as image correction processes on the cropped and resized image after facility detection, based on the shooting time and driving information. Therefore, when the target facility is not clearly captured, such as in an environment where low brightness is likely to occur, such as at night or inside a tunnel, image correction is performed by accurately determining the situation, thereby realizing highly accurate anomaly detection.

[0064] I. Ninth Embodiment The flowchart of the operation of the overhead line fitting abnormality detection device 13 of this embodiment is the same as that shown in FIG. 27 in the eighth embodiment. The device configuration is also the same as that shown in FIG. 2 in the first embodiment. This embodiment differs from the eighth embodiment in the condition determination for determining the unclear section. Other than this, the operation and functions are the same as those of the eighth embodiment, so a description thereof will be omitted.

[0065] In this embodiment, the conditions for determining a blurred section are based on brightness information of the image data, instead of the shooting time and vehicle traveling position information of the eighth embodiment. For example, an automatic determination process is used to determine a traveling section that may become blurred, such as at night or inside a tunnel. In the automatic determination process, the determination may be made using an algorithm that uses brightness information such as the average brightness of the captured image (determining each line or each image obtained by combining multiple lines based on "average brightness < threshold"). Alternatively, the determination may be made using a classification AI (AlexNet, VGG, GoogleNet, ResNet, etc.). Furthermore, when an algorithm based on brightness information is used, the average brightness and variance may be used. When statistical information of brightness is used, preprocessing such as noise removal may be performed as necessary before calculating the statistical information.

[0066] In this embodiment, in addition to the effects and advantages of the sixth embodiment, brightness correction and edge enhancement are performed as image correction processes on the cropped and resized images after equipment detection, based on the image brightness information and the discrimination AI. Therefore, when the target equipment is not clearly photographed, such as in an environment where photography is likely to be performed under low brightness, such as at night or inside a tunnel, image correction is performed by accurately determining the situation, thereby realizing highly accurate anomaly detection.

[0067] In addition to this, it is possible to select and discard the configurations given in the above embodiments, or to change them to other configurations as appropriate, without departing from the spirit of the present invention.

[0068] REFERENCE SIGNS LIST 1 Vehicle 2 Railway track 3 Overhead wire fitting 11 Camera 12 Storage device 13 Overhead wire fitting abnormality detection device 14 User interface (input / output unit) 15 Data processing unit 17 Learning processing unit 19 Equipment learning unit 21 Part learning unit

Claims

1. An overhead line fitting anomaly detection device comprising: a learning processing unit that performs learning using image data generated by photographing overhead line fittings; and a data processing unit that performs abnormality judgment using the image data, wherein the learning processing unit comprises an equipment learning unit that learns to detect equipment, and a part learning unit that learns to detect parts of the equipment, and wherein the data processing unit detects the equipment using an equipment detection model learned by the equipment learning unit, detects the parts from the detected equipment using a part detection model learned by the part learning unit, and judges abnormalities in the equipment and / or parts using a previously trained anomaly detection model.

2. The overhead line fitting abnormality detection device according to claim 1, wherein the data processing unit detects the equipment based on the equipment detection model, vehicle running position information, and equipment information.

3. The overhead line fitting abnormality detection device according to claim 1 or 2, wherein the data processing unit judges abnormalities in equipment and / or components based on a restoration error of the abnormality detection model.

4. A catenary fitting abnormality detection device as described in claim 1 or 2, further comprising an input / output unit, the detection results of the equipment and / or the parts detected by the data processing unit being displayed on the input / output unit, and an operator being able to check and edit the detection results from the input / output unit.

5. A method for detecting an abnormality in overhead line fittings, comprising: performing learning using image data generated by photographing overhead line fittings; and performing an abnormality judgment using the image data, wherein the performing learning includes learning to detect equipment and learning to detect parts of the equipment, and performing an abnormality judgment includes detecting the equipment using an equipment detection model learned by learning to detect the equipment, detecting the parts from the detected equipment using a part detection model learned by learning to detect the parts of the equipment, and judging an abnormality in the equipment and / or the parts using a previously trained anomaly detection model.

6. A method for detecting abnormalities in overhead line fittings as described in claim 5, wherein performing the abnormality judgment includes detecting the equipment based on the equipment detection model, vehicle running position information, and equipment information.

7. A method for detecting an abnormality in overhead line fittings as described in claim 5 or 6, wherein performing the abnormality judgment includes judging an abnormality in equipment and / or components based on a restoration error of the anomaly detection model.

8. A method for detecting abnormalities in overhead line fittings as described in claim 5 or 6, comprising displaying the detection results of the equipment and / or components detected by performing the abnormality judgment on an input / output unit, and an operator checking and editing the detection results from the input / output unit.

9. An overhead line fitting anomaly detection device comprising: a learning processing unit which performs learning using image data generated by photographing overhead line fittings; and a data processing unit which performs abnormality judgment using the image data, wherein the learning processing unit comprises an equipment learning unit which learns to detect equipment, and a part learning unit which learns to detect parts of the equipment, and wherein the data processing unit detects the equipment using an equipment detection model learned by the equipment learning unit, detects the parts from the detected equipment using a part detection model learned by the part learning unit, corrects the image data in which the equipment or parts have been detected, and judges abnormalities in the equipment and / or parts using a previously trained anomaly detection model.

10. An overhead line fitting anomaly detection device comprising: a learning processing unit which performs learning using image data generated by photographing overhead line fittings; and a data processing unit which performs abnormality judgment using the image data, wherein the learning processing unit comprises an equipment learning unit which learns to detect equipment, and a part learning unit which learns to detect parts of the equipment, and wherein the data processing unit corrects the image data, detects the equipment using an equipment detection model learned by the equipment learning unit, detects the parts from the detected equipment using a part detection model learned by the part learning unit, and judges abnormalities in the equipment and / or parts using a previously trained anomaly detection model.

11. An overhead line fitting anomaly detection device comprising: a learning processing unit that performs learning using image data generated by photographing overhead line fittings; and a data processing unit that performs abnormality judgment using the image data, wherein the learning processing unit comprises an equipment learning unit that learns to detect equipment, and a part learning unit that learns to detect parts of the equipment, and wherein the data processing unit detects the equipment using an equipment detection model learned by the equipment learning unit, detects the parts from the detected equipment using a part detection model learned by the part learning unit, corrects the image data in which the equipment or parts have been detected based on the photographing time and vehicle traveling position information, and judges abnormalities in the equipment and / or parts using a previously trained anomaly detection model.

12. An overhead line fitting anomaly detection device comprising: a learning processing unit that performs learning using image data generated by photographing overhead line fittings; and a data processing unit that performs abnormality judgment using the image data, wherein the learning processing unit comprises an equipment learning unit that learns to detect equipment, and a part learning unit that learns to detect parts of the equipment, and wherein the data processing unit detects the equipment using an equipment detection model learned by the equipment learning unit, detects the parts from the detected equipment using a part detection model learned by the part learning unit, corrects the image data based on luminance information or identification AI of the image data in which the equipment or part is detected, and judges abnormalities in the equipment and / or parts using a previously trained anomaly detection model.

13. A line fitting abnormality detection device as described in any one of claims 9 to 12, wherein the data processing unit performs an inversion process on the image data based on the position of the detected object in the image data before determining an abnormality in equipment and / or components using the pre-trained anomaly detection model.

14. A method for detecting abnormalities in overhead line fittings, comprising: performing learning using image data generated by photographing overhead line fittings; and performing an abnormality judgment using the image data, wherein the performing learning includes learning to detect equipment and learning to detect parts of the equipment, and performing an abnormality judgment includes detecting the equipment using an equipment detection model learned by learning to detect the equipment, detecting the parts using a part detection model learned by learning to detect the parts of the equipment from the detected equipment, correcting the image data in which the equipment or the parts are detected, and judging abnormalities in the equipment and / or the parts using a previously trained anomaly detection model.

15. A method for detecting abnormalities in overhead line fittings, comprising: performing learning using image data generated by photographing the overhead line fittings; and performing an abnormality judgment using the image data, wherein the performing learning includes learning to detect equipment and learning to detect parts of the equipment, and performing an abnormality judgment includes correcting the image data, detecting the equipment using an equipment detection model learned by learning to detect the equipment, detecting the parts from the detected equipment using a part detection model learned by learning to detect the parts of the equipment, and judging abnormalities in the equipment and / or parts using a previously trained anomaly detection model.

16. A method for detecting abnormalities in overhead line fittings, comprising: performing learning using image data generated by photographing overhead line fittings; and making an abnormality judgment using the image data, wherein the learning includes learning to detect equipment and learning to detect parts of the equipment, and wherein the abnormality judgment includes detecting the equipment using an equipment detection model learned by learning to detect the equipment, detecting the parts from the detected equipment using a part detection model learned by learning to detect the parts of the equipment, correcting the image data in which the equipment or the parts are detected based on the time of photographing and information about the vehicle's traveling position, and judging whether there is an abnormality in the equipment and / or the parts using a previously trained anomaly detection model.

17. A method for detecting abnormalities in overhead line fittings, comprising: performing learning using image data generated by photographing overhead line fittings; and making an abnormality judgment using the image data, wherein the learning includes learning to detect equipment and learning to detect parts of the equipment, and wherein the abnormality judgment includes detecting the equipment using an equipment detection model learned by learning to detect the equipment, detecting the parts from the detected equipment using a part detection model learned by learning to detect the parts of the equipment, correcting the image data based on luminance information or identification AI of the image data in which the equipment or the parts are detected, and judging abnormalities in the equipment and / or parts using a previously trained anomaly detection model.

18. A method for detecting abnormalities in overhead line fittings as described in any one of claims 14 to 17, wherein said determining an abnormality includes performing an inversion process on the image data based on the position of the detected object in the image data before determining an abnormality in equipment and / or components using the pre-trained anomaly detection model.

Citation Information

Patent Citations

  • Railway overhead line system fault detection method and device, electronic equipment and storage medium

    CN111784656A

  • Wire temperature monitoring device using image processing

    JP2016001140A

  • Overhead power line metal fitting detection device and method

    JP2019111942A

  • Overhead wire fitting abnormality detection device

    JP2022159010A

Cited By

  • Overhead wire fitting abnormality detection device and overhead wire fitting abnormality detection method

    JP7885894B1

  • Overhead wire fitting abnormality detection device and overhead wire fitting abnormality detection method

    WO2026168389A1