Overhead wiring metal fittings abnormality detection device and overhead wiring metal fittings abnormality detection method

Abstract

To provide a filament metal fittings abnormality detection device and a filament metal fittings abnormality detection method that can provide a data set for improving precision of detection on abnormality of wiring metal fittings.SOLUTION: An overhead wiring metal fittings abnormality detection device comprises: line sensor cameras 10L, 10R which are arranged on a vehicle roof and comprise fisheye lenses; a trimming part 141 which trims away parts other than photography object ranges of an image 50L photographed by the line sensor camera 10L and an image 50R photographed by the line sensor camera 10R; and an image right-left inversion processing part 142 which inverts the right and left of one of the trimmed image 50L and the trimmed image 50R about center positions of the line sensor cameras 10L, 10R so that photographed wiring metal fittings are in the same direction. The overhead wiring metal fittings abnormality detection device provides the image which does not have the right and left inverted and the inverted image which has the right and left inverted between the trimmed images as processed images for detecting abnormality of wiring metal fittings.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an abnormality detection device and an abnormality detection method for an overhead contact fitting. [Background technology]

[0002] Conventionally, maintenance and inspection work has been carried out on overhead line fittings, which are among the electric railway equipment, and inspectors visually check for abnormalities in the overhead line fittings to perform maintenance. Overhead line fittings are fittings that hold the overhead train lines (trolley wires, suspension wires, auxiliary suspension wires; hereafter referred to as "overhead lines") in a specified position to supply power to trains in electric railway equipment, or electrically connect or separate multiple overhead lines. However, visual inspection has limitations, and there is a possibility that abnormalities may be overlooked.

[0003] Therefore, Patent Documents 1 and 2 disclose a technology in which an image of the overhead line fittings is taken with a line sensor camera mounted on the roof of a railway vehicle to generate an input image, and then, based on the input image, machine learning is used to detect abnormalities in the overhead line fittings. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2020-147129 A [Patent Document 2] JP 2020-149286 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, in Patent Documents 1 and 2, when generating the input image, no image processing is performed on the captured input image. Therefore, if there is extra data in the image (metal fittings outside the range of the image to be captured, such as outside the main line), there is a risk that this data will affect the accuracy of anomaly detection.

[0006] Furthermore, when photographing a narrow range of subjects, there are many restrictions on the installation of equipment, such as the vehicle limits of the railway car, as well as the angle of view and depth of field of the line sensor camera, which significantly limits the selection of equipment.

[0007] Therefore, an object of the present invention is to provide a wire fitting abnormality detection device and a wire fitting abnormality detection method that are capable of providing a data set for improving the accuracy of detecting abnormalities in wire fittings. [Means for solving the problem]

[0008] In order to solve the above problems, a wire fitting abnormality detection device according to a first aspect of the present invention is as follows: a first line sensor camera and a second line sensor camera which are cameras for photographing overhead wire fittings using fisheye lenses while a railroad vehicle is traveling, the first line sensor camera and the second line sensor camera being disposed a predetermined distance apart on the left and right sides of the direction of travel of the railroad vehicle; a trimming unit that trims a portion outside a range of a subject to be photographed from a first image photographed by the first line sensor camera and trims a portion outside the range of a subject to be photographed from a second image photographed by the second line sensor camera; a left-right reversal processing unit that left-right reverses one of the trimmed first image and the trimmed second image so that an orientation of the captured overhead line fitting coincides with a center position of the first line sensor camera in the first image and a center position of the second line sensor camera in the second image, Among the trimmed first image or the trimmed second image, an image that has not been left-right reversed by the left-right reversal processing unit and an inverted image that has been left-right reversed by the left-right reversal processing unit are provided as processed images for detecting an abnormality in the overhead line fittings. It is characterized by:

[0009] In addition, the overhead line fitting abnormality detection device according to the second invention for solving the above problem is the above-mentioned first invention, an image left / right division processing unit that divides the first image into left and right parts from a center position of the first line sensor camera in the first image, and divides the second image into left and right parts from a center position of the second line sensor camera in the second image; an outer image deletion processing unit that deletes, from among the divided images divided from the first image, divided images that correspond to an outer side of the vehicle from a center position of the first line sensor camera, and deletes, from among the divided images divided from the second image, divided images that correspond to an outer side of the vehicle from a center position of the second line sensor camera, The trimming unit includes: trimming the outside of the range to be photographed from the first divided image that remains without being deleted, and trimming the outside of the range to be photographed from the second divided image that remains without being deleted; The left-right reversal processing unit is flipping one of the first divided image and the second divided image horizontally so that a direction of the overhead line fittings photographed in the trimmed first divided image coincides with a direction of the overhead line fittings photographed in the trimmed second divided image; It is characterized by:

[0010] In order to solve the above problems, a third aspect of the present invention provides an abnormality detection device for overhead line fittings, comprising: an image left / right division processing unit that divides the first image into left and right from a center position of the first line sensor camera in the first image, and divides the second image into left and right from a center position of the second line sensor camera in the second image, The trimming unit includes: trimming an area outside the range to be photographed from a first left divided image and a first right divided image obtained by dividing the first image into left and right parts, and trimming an area outside the range to be photographed from a second left divided image and a second right divided image obtained by dividing the second image into left and right parts, The left-right reversal processing unit is flipping one of the first left split image or the first right split image horizontally so that orientations of overhead line fittings in the trimmed first left split image and the trimmed first right split image coincide with each other, and flipping one of the second left split image or the second right split image horizontally so that orientations of overhead line fittings in the trimmed second left split image and the trimmed second right split image coincide with each other. It is characterized by:

[0011] Further, a fourth aspect of the present invention provides a method for detecting overhead line fittings, comprising: a step of photographing the overhead wire fittings while the railway vehicle is traveling, using a first line sensor camera and a second line sensor camera each equipped with a fisheye lens, the first line sensor camera and the second line sensor camera being disposed at a predetermined distance to the left and right of the direction of travel of the railway vehicle; trimming a portion outside a range of a subject captured from a first image captured by the first line sensor camera and trimming a portion outside a range of a subject captured from a second image captured by the second line sensor camera; a step of left-right inverting one of the trimmed first image and the trimmed second image so that an orientation of the photographed overhead line fitting coincides with a center position of the first line sensor camera in the first image and a center position of the second line sensor camera in the second image; providing, as processed images for detecting an abnormality in the overhead line fittings, an image that has not been horizontally inverted by the horizontal inversion step and an inverted image that has been horizontally inverted by the horizontal inversion step, out of the first trimmed image or the second trimmed image; The present invention is characterized by comprising:

[0012] In order to solve the above problems, a fifth aspect of the present invention provides a method for detecting an abnormality in a wire fitting, comprising the steps of: dividing the first image into left and right parts from a center position of the first line sensor camera in the first image, and dividing the second image into left and right parts from a center position of the second line sensor camera in the second image; a step of deleting, from among the divided images obtained by dividing the first image, a divided image that corresponds to an outer side of the vehicle from a center position of the first line sensor camera, and deleting, from among the divided images obtained by dividing the second image, a divided image that corresponds to an outer side of the vehicle from a center position of the second line sensor camera, The step of trimming the outside of the shooting range includes: trimming the outside of the range to be photographed from the first divided image that remains without being deleted, and trimming the outside of the range to be photographed from the second divided image that remains without being deleted; The step of flipping left and right includes: flipping one of the first divided image and the second divided image horizontally so that a direction of the overhead line fittings photographed in the trimmed first divided image coincides with a direction of the overhead line fittings photographed in the trimmed second divided image; It is characterized by:

[0013] In order to solve the above problems, a sixth aspect of the present invention provides a method for detecting an abnormality in a wire fitting, comprising the steps of: further comprising a step of dividing the first image into left and right parts from a center position of the first line sensor camera in the first image, and dividing the second image into left and right parts from a center position of the second line sensor camera in the second image, The step of trimming the outside of the shooting range includes: trimming an area outside the range to be photographed from a first left divided image and a first right divided image obtained by dividing the first image into left and right parts, and trimming an area outside the range to be photographed from a second left divided image and a second right divided image obtained by dividing the second image into left and right parts, The step of flipping left and right includes: flipping one of the first left split image or the first right split image horizontally so that orientations of overhead line fittings in the trimmed first left split image and the trimmed first right split image coincide with each other, and flipping one of the second left split image or the second right split image horizontally so that orientations of overhead line fittings in the trimmed second left split image and the trimmed second right split image coincide with each other. It is characterized by: Effect of the Invention

[0014] According to the present invention, it is possible to provide a data set for improving the accuracy of detecting an abnormality in an overhead line fitting. [Brief description of the drawings]

[0015] [Figure 1] 1 is a schematic diagram showing a schematic configuration of a wire fitting abnormality detection device 1 according to a first embodiment of the present invention. [Diagram 2] FIG. 1 is a schematic diagram for explaining advantages of line sensor cameras 10L and 10R using fisheye lenses in the first embodiment. [Diagram 3] 5 is a flowchart for explaining the operation of the overhead line fitting abnormality detection device 1 according to the first embodiment. [Figure 4] 3 is a conceptual diagram showing data transfer for explaining the operation of the overhead line fitting abnormality detection device 1 according to the first embodiment. FIG. [Diagram 5] 1 is a schematic diagram showing a state during imaging by the overhead line fitting abnormality detection device 1 according to the first embodiment. FIG. [Figure 6] 2 is a schematic diagram showing an image captured by the overhead line fitting abnormality detection device 1 according to the first embodiment. FIG. [Figure 7] FIG. 2 is a conceptual diagram for explaining a trimming range according to the first embodiment. [Figure 8] FIG. 11 is a conceptual diagram for explaining the trimming range for a left image 50L and a right image 50R according to the first embodiment. [Figure 9]FIG. 13 is a conceptual diagram showing an original image (left image 50L) before cropping according to the first embodiment and a left cropped image 51L obtained after cropping. [Figure 10] 10A and 10B are conceptual diagrams showing a left trimmed image 51L and a right trimmed image 51R after trimming in the first embodiment. [Figure 11] FIG. 11 is a conceptual diagram for explaining the advantage (effect) in learning and inference by performing trimming in the first embodiment. [Figure 12] FIG. 4 is a conceptual diagram for explaining left-right reversal processing according to the first embodiment. [Figure 13] 10 is a flowchart for explaining the operation of the overhead line fitting abnormality detection device 1 according to the second embodiment. [Figure 14] 10 is a conceptual diagram showing data transfer for explaining the operation of the overhead line fitting abnormality detection device 1 according to the second embodiment. FIG. [Figure 15] FIG. 11 is a conceptual diagram for explaining left / right image division processing according to the second embodiment. [Figure 16] FIG. 11 is a conceptual diagram for explaining outer image deletion processing according to the second embodiment. [Figure 17] 13 is a conceptual diagram for explaining the trimming range for a left divided image 54L and a right divided image 53R according to the second embodiment. FIG. [Figure 18] 13 is a conceptual diagram showing a left trimmed image 55L and a right trimmed image 55R after trimming in the second embodiment. FIG. [Figure 19] FIG. 11 is a conceptual diagram showing a left trimmed image 55L after trimming and a right inverted image 56R after image inversion in the second embodiment. [Figure 20] 10 is a flowchart for explaining the operation of the overhead line fitting abnormality detection device 1 according to the third embodiment. [Figure 21] 13 is a conceptual diagram showing data transfer for explaining the operation of the overhead line fitting abnormality detection device 1 according to the third embodiment. FIG. [Figure 22] FIG. 13 is a conceptual diagram for explaining left / right image division processing according to the third embodiment. [Diagram 23]13 is a conceptual diagram for explaining the trimming ranges for the left divided images 53L, 54L and the right divided images 53R, 54R according to the third embodiment. FIG. [Figure 24] 13A and 13B are conceptual diagrams showing left trimmed images 57L, 58L and right trimmed images 57R, 58R after trimming in the third embodiment. [Diagram 25] 13 is a conceptual diagram showing a left trimmed inverted image 59L, a left trimmed image 58L, a right trimmed inverted image 59R, and a right trimmed image 58R in the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings. The present invention aims to reduce the labor required for maintenance and inspection of overhead line fittings by making it possible to automatically detect overhead line fittings from images acquired from at least two line sensor cameras mounted on the roof of a railway vehicle, and by detecting abnormalities in the overhead line fittings based on the captured images in subsequent processing.

[0017] A. First embodiment Fig. 1 is a schematic diagram showing the schematic configuration of an overhead line fitting abnormality detection device 1 according to a first embodiment of the present invention. In Fig. 1, two line sensor cameras (first and second line sensor cameras) 10L, 10R equipped with fisheye lenses are installed on a vehicle roof 21 of a railway vehicle 2, spaced a predetermined distance apart in the vehicle width direction, and multiple lights 11, 11, 11 are installed facing upward. The line sensor cameras 10L, 10R are installed so that their angle of view is perpendicular to the vehicle roof 21. The line sensor cameras 10L, 10R photograph the overhead line fittings installed above the vehicle roof 21 along the tracks.

[0018] In the present invention, the overhead line fittings refer to the following: Hangers, droppers, connectors, double ears Curved overhead line fittings and supports These can be detected by the overhead line fitting abnormality detection device and overhead line fitting abnormality detection method according to the present invention.

[0019] An image processing device such as a personal computer (hereinafter referred to as on-board PC) 12 is mounted on the railway vehicle 2. The on-board PC 12 includes a data storage unit 13 that stores captured images taken by the line sensor cameras 10L and 10R, and an image processing unit 14 that performs various image processing on the captured images and performs preprocessing for detecting overhead line fittings by machine learning and detecting abnormalities in the overhead line fittings (distortion of the fittings, the presence or absence of screws, etc.).

[0020] In the present invention, the image processing unit 14 is equipped with a calculation device that implements an artificial intelligence function that learns using previously acquired images of overhead line fittings and images of things other than overhead line fittings, and detects overhead line fittings from the images based on the images generated by the preprocessing described above, and detects abnormalities in the detected overhead line fittings. However, since the process of detecting overhead line fittings from images by the calculation device and the process of detecting abnormalities in the detected overhead line fittings (distortion of the fittings, presence or absence of screws, etc.) are well known, a description thereof will be omitted.

[0021] Here, the reason (effect) of using fisheye lenses instead of wide-angle lenses for the line sensor cameras 10L and 10R in the present invention will be described. FIGS. 2(a) and 2(b) are schematic diagrams for explaining the advantages of the line sensor cameras 10L and 10R using fisheye lenses in the first embodiment. FIG. 2(a) shows a camera imaging range 31, a photographing target range 32, and a depth of field 33 in the case of a line sensor camera 30 using a wide-angle lens. FIG. 2(b) shows a camera imaging range 34 and a photographing target range 35 in the case of a line sensor camera 10L using a fisheye lens. In addition, in FIGS. 2(a) and 2(b), a vehicle limit 36 ​​is set around the vehicle. The vehicle limit 36 ​​is a standard for restricting the width, height, and other values ​​of a vehicle so that the vehicle can run safely on the track, and structures cannot be installed beyond this vehicle limit. Although the line sensor camera 10R is not shown in FIG. 2(b), the reason (effect) of using a fisheye lens is the same as that of the line sensor camera 10L, except that the camera imaging range 34 is shifted laterally.

[0022] In the case of the line sensor camera 30 using a wide-angle lens, due to the angle of view of the wide-angle lens, it is necessary to install the line sensor camera 30 at an angle as shown in the camera imaging range 31 in Fig. 2(a). In this case, since the image is captured from an oblique direction relative to the rectangular imaging target range 32, it is necessary to provide a deep depth of field. In addition, when a wide-angle lens is used, it is necessary to install the line sensor camera 30 at the edge of the vehicle roof 21, which may cause a collision with the vehicle limit 36 ​​described above.

[0023] On the other hand, as shown in FIG. 2(b), when a fisheye lens is used, the angle of view of the line sensor camera 10L is 180°, and the depth of field is deep, as shown in the camera imaging range 34. When a fisheye lens is used, it is not necessary to consider the angle of view of the line sensor camera 10L, and one of the characteristics of the fisheye lens is the deep depth of field, so that it is possible to capture a wider camera imaging range 34 than the line sensor camera 30 using a wide-angle lens. For the above reasons, by using a fisheye lens for the line sensor cameras 10L and 10R, it is no longer necessary to install them at the edge of the vehicle roof 21, and it is possible to significantly reduce installation constraints.

[0024] Fig. 3 is a flow chart for explaining the operation of the overhead line fitting abnormality detection device 1 according to the first embodiment. Fig. 4 is a conceptual diagram showing data transfer for explaining the operation of the overhead line fitting abnormality detection device 1 according to the first embodiment. Fig. 5 is a schematic diagram showing a state when an image is captured by the overhead line fitting abnormality detection device 1 according to the first embodiment. Fig. 6 is a schematic diagram showing an image captured by the overhead line fitting abnormality detection device 1 according to the first embodiment.

[0025] In the first embodiment, the following processing of steps S10 to S16 is repeated for a plurality of images (the number of images) captured by the line sensor camera at a predetermined time interval from the start to the end of image capture.

[0026] <Shooting process> First, the overhead line fitting abnormality detection device 1 captures images using the line sensor cameras 10L and 10R while the vehicle is traveling (step S10). The images captured by the line sensor cameras 10L and 10R are stored in the data storage unit 13 as shown in FIG.

[0027] Here, Fig. 5 is a schematic diagram showing the positional relationship between the vehicle roof 21, the line sensor cameras 10L and 10R, and the overhead lines A to C in this embodiment. During photography, the positional relationship between the vehicle roof 21, the line sensor cameras 10L and 10R, and the overhead lines A to C is as shown in Fig. 5. The overhead line A is composed of a trolley wire 40A, a suspension wire 41A suspending the trolley wire 40A, and an overhead line fitting 42A for connecting the trolley wire 40A and the overhead line 41A, and the overhead line B is composed of a trolley wire 40B, a suspension wire 41B suspending the trolley wire 40B, and an overhead line fitting 42B for connecting the trolley wire 40B and the overhead line 41B. The overhead lines A and B are included in the photography target range 35 of the line sensor cameras 10L and 10R. The overhead line fittings 42A and 42B of the overhead line A and B included in the photographing range 35 are targets for abnormality detection. On the other hand, the overhead line fitting 42C of the overhead line C not included in the photographing range 35 is a non-photographable fitting that is not a target for abnormality detection.

[0028] When the line sensor cameras 10L and 10R capture images of the overhead wires A and B located at the positions shown in Fig. 5, a left image (first image) 50L and a right image (second image) 50R are captured as shown in Fig. 6. The line sensor camera 10L captures the left image 50L shown in the upper part of Fig. 6, and the line sensor camera 10R captures the right image 50R shown in the lower part of Fig. 6. In Fig. 6, the lateral direction of the left image 50L and the right image 50R corresponds to the lateral direction of the vehicle roof 21. That is, the images are captured with a lateral shift by the difference in the installation positions of the line sensor cameras 10R and 10L.

[0029] In the left image 50L captured by the line sensor camera 10L, the overhead line A is captured to the left of the camera center, and the overhead line B is captured to the right of the camera center. At this time, since the line sensor camera 10L is oriented vertically, the upper parts of the overhead line fittings 42A1 and 42A2 of the overhead line A and the upper parts of the overhead line fittings 42B1 and 42B2 of the overhead line B are captured to be located on the camera center side. In the right image 50R captured by the line sensor camera 10R, the overhead line A and B are captured to the left of the camera center, and the overhead line C is captured to the right of the camera center. At this time, since the line sensor camera 10R is oriented vertically, the upper parts of the overhead line fittings 42B1 and 42B2 of the overhead line B and the upper parts of the overhead line fittings 42C1 and 42C2 of the overhead line C are captured to be located on the camera center side. In addition, since the overhead line fittings 42C1 and 42C2 of the overhead line C are not subject to abnormality detection, it is preferable that they are not included in the data for machine learning, and they are deleted by trimming, which will be described later.

[0030] <Trimming process> Next, the overhead line fitting abnormality detection device 1 performs trimming outside the imaging target range on the captured images (left image 50L, right image 50R) by the image processing unit 14 (step S12). More specifically, as shown in Fig. 4, the image processing unit 14 performs trimming on the captured images (left image 50L, right image 50R) stored in the data storage unit 13 by the outside imaging target range trimming unit (trimming unit) 141 to remove images outside the imaging target range, and stores the trimmed image (hereinafter referred to as a trimmed image) in the data storage unit 13. The trimming outside the imaging target range is for cutting out a part of the image of the overhead line C that is outside the imaging target range and is not an abnormality detection target, and by reducing unnecessary data, the load on the machine learning and inference processing can be reduced.

[0031] Here, Fig. 7 is a conceptual diagram for explaining the trimming range according to the first embodiment. Note that Fig. 7 explains only the line sensor camera 10L, but the same applies to the line sensor camera 10R. Fig. 8 is a conceptual diagram for explaining the trimming range for the left image 50L and the right image 50R according to the first embodiment.

[0032] The trimming range is determined based on the relationship between the positions of the line sensor cameras 10L and 10R (hereinafter, simply referred to as the camera positions) and the imaging target range 35. In order to check the relationship between the camera positions and the imaging target range 35, the positional relationship between the range unnecessary for imaging and the pixel position on the actual image is compared. In comparing the positional relationship, as shown in FIG. 7, marks (such as white markers) 61L and 62L are placed on the left and right ends of the minimum height 60 of the imaging target range 35L, and it is confirmed at which pixel position they appear on the actual image. The lower part of FIG. 7 shows the trimming ranges 63L and 64L and the imaging target range 35L in the left image 50L, and the trimming ranges 63R and 64R and the imaging target range 35R in the right image 50R. The ranges outside the pixel positions of the marks 61L and 62L or the marks 61R and 62R are the trimming ranges 63L and 64L and the trimming ranges 63R and 64R to be trimmed. Applying this to the captured left image 50L and right image 50R shown in FIG. 6, cropping is performed on the left image 50L within cropping ranges 63L and 64L, and cropping is performed on the right image 50R within cropping ranges 63R and 64R, as shown in FIG. 8.

[0033] FIG. 9 is a conceptual diagram showing an original image (left image 50L) before trimming according to the first embodiment and a left trimmed image 51L after trimming. Note that FIG. 9 shows a left image 50L as an original image and a left trimmed image (trimmed first image) 51L after trimming the left image 50L, but the same is true for the right image 50R. As shown in FIG. 9, in the original image immediately after shooting, that is, the left image 50L, the overhead line fittings 42A1 and 42A2 of the overhead line A and the overhead line fittings 42B1 and 42B2 of the overhead line B, which are data to be learned and inferred, and the overhead line fittings 42C1 and 42C2 of the overhead line C, which are non-target data not to be learned and inferred, are mixed in the same shape. On the other hand, in the left trimmed image 51L after trimming, parts of the overhead line fittings 42C1 and 42C2 of the overhead line C, which are non-target data not to be learned and inferred, are trimmed and cut out.

[0034] Fig. 10 is a conceptual diagram showing a left trimmed image 51L and a right trimmed image 51R after trimming in the first embodiment. As shown in Fig. 10, by trimming, a part of the overhead line C, which is non-target data not to be learned or inferred, is trimmed and cut out in both the left trimmed image 51L and the right trimmed image (trimmed second image) 51R. As a result, in the left trimmed image 51L, it becomes possible to distinguish the overhead line fittings 42A1, 42A2 of the overhead line A and the overhead line fittings 42B1, 42B2 of the overhead line B, which are data to be learned and inferred, from the overhead line fittings 42C1, 42C2 of the overhead line C, which are non-target data not to be learned and inferred, as being different in shape.

[0035] Moreover, it is possible to retain more image features when compressing an image. In machine learning, for convenience of processing, the image is resized to a square before learning and inference are performed. In the first embodiment, by trimming the width of the image, it is possible to reduce unnecessary information, and when the image is resized to a square, it is possible to perform efficient learning and inference.

[0036] FIG. 11 is a conceptual diagram for explaining the advantage (effect) of trimming in learning and inference in the first embodiment. Note that FIG. 11 shows a left image 50L as an original image and a left trimmed image 51L after the left image 50L is trimmed, but the same is true for the right image 50R. The upper part of FIG. 10 shows an image A in which the overhead wire fittings part is enlarged when the left image 50L without trimming is resized to a square for the purpose of improving the efficiency of learning and inference, and the lower part of the same figure shows an image B in which the overhead wire fittings part is enlarged when the left trimmed image 51L after trimming is resized to a square. As can be seen by comparing images A and B, in the first embodiment, when the left trimmed image 51L is resized to a square, more fitting information (high-definition image; coil-shaped fittings are spaced widely apart) can be left than when the left image 50L before trimming is resized to a square, that is, data loss due to compression can be reduced.

[0037] <Horizontal inversion processing> Next, the overhead line fitting abnormality detection device 1 performs left-right inversion of the right trimmed image 51R after the trimming (step S14). More specifically, as shown in Fig. 4, the image processing unit 14 performs left-right inversion of the right trimmed image 51R after the trimming stored in the data storage unit 13 by the image left-right inversion processing unit 142, and stores the image in the data storage unit 13 as a right inverted image 52R.

[0038] In the right image 50R, which is the original image, as shown in FIG. 6 described above, due to the installation position of the line sensor camera 10R, the upper part of the overhead wire fitting 42B of the overhead wire B is captured in a direction close to the center of the image. In machine learning, the accuracy is affected not only by the shape of the overhead wire fitting but also by pixel information on the image, so it is desirable that overhead wire fittings at the same position are captured in the same orientation in the left and right images. However, as shown in FIG. 10 described above, the left and right trimmed images 51L and 51R after trimming have different trimming ranges on the left and right, so the orientation of the overhead wire fittings differs depending on the left and right image positions. In the example of FIG. 10, the orientations of the overhead wire fittings 42B1 and 42B2 are different in the left and right trimmed images 51L and 51R. Therefore, a situation occurs in which the image coordinate information does not match the orientation information of the overhead wire fittings 42B1 and 42B2. To prevent this, of the left and right trimmed images 51L and 51R after trimming, the right trimmed image 51R is subjected to left-right inversion processing.

[0039] Fig. 12 is a conceptual diagram for explaining left-right inversion processing according to the first embodiment. As shown in Fig. 12, when left-right inversion processing is performed on the right trimmed image 51R, the camera center positions in the left trimmed image 51L and the right inverted image (inverted image) 52R are unified, so that the upper parts of the overhead wire fittings 42A1 and 42A2 in the left image are always located on the right, and the upper parts of the overhead wire fittings 42A1, 42A2, 42B1, and 42B2 in the right image are always located on the left, relative to the left and right camera centers. Therefore, the relationship between the positional relationship on the pixels of the image and the orientation of the overhead wire fittings 42B1 and 42B2 is also unified, and the accuracy of machine learning can be improved.

[0040] <Processed image provision processing> Next, the overhead line fitting abnormality detection device 1 supplies the left trimmed image 51L after trimming and the right inverted image 52R after left-right inversion processing as processed images to a machine learning system (arithmetic device) (not shown) (step S16). The machine learning system (not shown) performs machine learning based on the processed images, the left trimmed image 51L after trimming and the right inverted image 52R after left-right inversion processing, and detects abnormalities in the overhead line fittings (distortion of the fittings, presence or absence of screws, etc.).

[0041] According to the above-described first embodiment, by trimming, unnecessary data can be reduced, and the processing load for learning and inference can be reduced. Furthermore, by trimming, the loss of overhead line fitting information when resizing to a square can be mitigated. Furthermore, by image inversion, the relationship between the pixel position of the image and the orientation of the overhead line fittings can be unified, and the accuracy of machine learning can be improved. These effects make it possible to improve the accuracy of detecting abnormalities in overhead line fittings.

[0042] B. Second embodiment Next, a second embodiment of the present invention will be described. The configuration of the overhead line fitting abnormality detection device 1 is the same as that in Fig. 1, so a description thereof will be omitted. Fig. 13 is a flow chart for explaining the operation of the overhead line fitting abnormality detection device 1 according to the second embodiment. Fig. 14 is a conceptual diagram showing data transfer for explaining the operation of the overhead line fitting abnormality detection device 1 according to the second embodiment. Note that parts corresponding to those in Fig. 4 are given the same reference numerals and descriptions thereof will be omitted.

[0043] In the second embodiment, the processes of steps S20 to S30 are repeated for a plurality of images (the number of images) captured by the line sensor camera at a predetermined time interval from the start to the end of image capture.

[0044] <Shooting process> First, the overhead line fitting abnormality detection device 1 captures images using the line sensor cameras 10L and 10R while the vehicle is traveling (step S20). More specifically, the captured images captured by the line sensor cameras 10L and 10R are stored in the data storage unit 13 as shown in Fig. 14. The captured images become a left image 50L captured by the line sensor camera 10L and a right image 50R captured by the line sensor camera 10R as shown in Fig. 6 described above.

[0045] <Image left / right division processing> Next, the overhead line metal fitting abnormality detection device 1 performs a division process by the image processing unit 14 to divide each of the captured images (left image 50L, right image 50R) into left and right images at the center of the camera (step S22). More specifically, as shown in FIG. 14, the image processing unit 14 divides the captured images (left image 50L, right image 50R) stored in the data storage unit 13 into left and right images at the center of the camera by the left and right image division processing unit 143, and stores each of the divided images in the data storage unit 13. Here, FIG. 15 is a conceptual diagram for explaining the left and right image division process according to the second embodiment. As shown in FIG. 15, each of the left image 50L and the right image 50R is divided into left and right images (divided images) 53L and 54L at the center of the camera, and right divided images (divided images) 53R and 54R are obtained.

[0046] <Outer image deletion process> Next, the overhead line fitting abnormality detection device 1 deletes some images of the left and right divided images that are on the outside of the vehicle from the center of the camera (step S24). More specifically, as shown in Fig. 14, the image processing unit 14 deletes, by the outside image deletion processing unit 144, the images of the left and right divided images stored in the data storage unit 13 that are on the outside of the vehicle from the center of the camera, and stores the remaining left and right divided images in the data storage unit 13.

[0047] Fig. 16 is a conceptual diagram for explaining the outer image deletion process according to the second embodiment. As shown in the upper part of Fig. 16, of the left divided images (divided images divided from the first image) 53L and 54L, the left divided image (image corresponding to the outer side) 53L, which is on the outer side of the vehicle from the center of the camera, is deleted, and of the right divided images (divided images divided from the second image) 53R and 54R, the right divided image (image corresponding to the outer side) 54R, which is on the outer side of the vehicle from the center of the camera, is deleted. In other words, the right half of the left divided image 54L is left for the left image 50L, and the left half of the right divided image 53R is left for the right image 50R.

[0048] Generally, most overhead line fittings are installed near the center of the vehicle (at the position of overhead line B in Figs. 5 and 6). As described above, the upper parts of the overhead line fittings 42A, 42B are photographed so as to be located at the center of the image, as shown in Fig. 6. For this reason, in order to unify the up-down orientation of the overhead line fittings 42A, 42B in the image, the image of the part of the vehicle that is on the outer side of the camera center (the side where the overhead line B is not captured) where there are relatively few overhead line fittings to be photographed is deleted. As a result, although there are areas where abnormality detection that was conventionally performed by the left and right line sensor cameras 10L, 10R is performed only by one line sensor camera, 10R, as shown in the lower part of Fig. 16, it is possible to completely unify the orientation of the overhead line fittings in one original image.

[0049] In the example of Fig. 16, the orientations of the overhead line fittings 42B1, 42B2, 42C1, and 42C2 of the overhead lines B and C in the left divided image 54L match with those of the line sensor camera 10L, and the orientations of the overhead line fittings 42A1, 42A2, 42B1, and 42B2 of the overhead lines A and B in the right divided image 53R match with those of the line sensor camera 10R. In this case, an abnormality is detected for the overhead line fittings 42A1 and 42A2 of the overhead line A only by the line sensor camera 10R. Furthermore, an abnormality is detected for the overhead line fittings 42B1 and 42B2 of the overhead line B by both the left and right line sensor cameras 10L and 10R. Furthermore, no overhead line exists at the right end of the shooting range in this case, but if an overhead line exists, an abnormality would be detected only by the line sensor camera 10L.

[0050] <Trimming process> Next, in the overhead line fitting abnormality detection device 1, the image processing unit 14 performs outside-the-photographing-target-range trimming on the left divided image 54L and the right divided image 53R after the left and right divisions and the outer images have been deleted (step S26), in the same manner as in the first embodiment described above. More specifically, as shown in Fig. 14, the image processing unit 14 performs outside-the-photographing-target-range trimming on the left divided image and the right divided image after the outer images have been deleted and stored in the data storage unit 13, by the outside-the-photographing-target-range trimming unit 141, and stores them in the data storage unit 13 as a left trimmed image and a right trimmed image.

[0051] 17 is a conceptual diagram for explaining the trimming ranges for the left split image 54L and the right split image 53R according to the second embodiment. The trimming process is basically the same as that of the first embodiment. The trimming ranges according to the second embodiment are two, a trimming range (outside the range to be photographed) 64L at the right end of the left split image 54L and a trimming range (outside the range to be photographed) 64R at the left end of the right split image 53R.

[0052] Fig. 18 is a conceptual diagram showing a left trimmed image 55L and a right trimmed image 55R after trimming in the second embodiment. By trimming the left divided image (first divided image) 54L and the right divided image (second divided image) 53R, as shown in Fig. 18, it is possible to cut out a partial image of the overhead wire C that is outside the range of the image capture target and is not an abnormality detection target.

[0053] <Image inversion processing> Next, in the overhead line fitting abnormality detection device 1, similarly to the above-described first embodiment, the image processing unit 14 performs left-right flipping of the right trimmed image 55R after trimming (step S28). More specifically, as shown in Fig. 14, the image processing unit 14 performs left-right flipping of the right trimmed image after trimming stored in the data storage unit 13 using the image left-right flipping processing unit 142, and stores it in the data storage unit 13 as a right-inverted image.

[0054] FIG. 19 is a conceptual diagram showing a left trimmed image 55L after trimming and a right inverted image 56R after image inversion in the second embodiment. The upper part of FIG. 19 shows the left trimmed image 55L after trimming obtained from the image captured by the left line sensor camera 10L, and the lower part of the same figure shows the right inverted image 56R after trimming and inversion processing. In the second embodiment, by performing left-right inversion on the right trimmed image 55R after trimming, as shown in FIG. 19, the orientation of the overhead line fittings 42B1 and 42B2 of the overhead line B in the left trimmed image 55L is unified with the orientation of the overhead line fittings 42A1 and 42A2 of the overhead line A and the overhead line fittings 42B1 and 42B2 of the overhead line B in the right inverted image 56R, and the accuracy of machine learning can also be improved.

[0055] <Processed image provision processing> Next, the overhead line fitting abnormality detection device 1 supplies the trimmed left trimmed image 55L and the trimmed and inverted right inverted image 56R as processed images to a machine learning system (arithmetic device) (not shown) (step S30). The machine learning system (not shown) performs machine learning based on the trimmed left trimmed image 55L and the trimmed and inverted right inverted image 56R, which are the processed images, and detects abnormalities in the overhead line fittings (distortion of the fittings, the presence or absence of screws, etc.).

[0056] According to the second embodiment described above, compared to the first embodiment, the orientation of the overhead line fittings can be completely unified, and the accuracy of machine learning can be further improved. In addition, the loss of overhead line fitting information during resizing to a square can be mitigated. Furthermore, by deleting the outer images, unnecessary data can be further reduced, and the processing for learning and inference can be further reduced.

[0057] C. Third embodiment Next, a third embodiment of the present invention will be described. The configuration of the overhead line fitting abnormality detection device 1 is the same as that in Fig. 1, so a description thereof will be omitted. Fig. 20 is a flow chart for explaining the operation of the overhead line fitting abnormality detection device 1 according to the third embodiment. Fig. 21 is a conceptual diagram showing data transfer for explaining the operation of the overhead line fitting abnormality detection device 1 according to the third embodiment. Note that parts corresponding to those in Fig. 14 are given the same reference numerals and descriptions thereof will be omitted.

[0058] In the third embodiment, the processes of steps S40 to S48 are repeated for a plurality of images (the number of images) captured by the line sensor camera at a predetermined time interval from the start to the end of image capture.

[0059] <Shooting process> First, the overhead line fitting abnormality detection device 1 photographs the overhead line fittings with the line sensor cameras 10L, 10R while the vehicle is traveling (step S40). More specifically, the images captured by the line sensor cameras 10L, 10R are stored in the data storage unit 13 as shown in Fig. 21. The captured images are a left image 50L captured by the line sensor camera 10L and a right image 50R captured by the line sensor camera 10R as shown in Fig. 6 described above.

[0060] <Image left / right division processing> Next, in the overhead line fitting abnormality detection device 1, similarly to the second embodiment, the image processing unit 14 performs a division process to divide each of the captured images (left image 50L, right image 50R) into left and right images at the center of the camera (step S42). More specifically, as shown in Fig. 21, the image processing unit 14 divides the captured images (left image 50L, right image 50R) stored in the data storage unit 13 into left and right images at the center of each camera by the left / right image division processing unit 143, and stores them in the data storage unit 13 as left / right divided images.

[0061] Fig. 22 is a conceptual diagram for explaining the left / right image division process according to the third embodiment. As shown in Fig. 22, each of a left image 50L and a right image 50R is divided left and right at the center of the camera to obtain left divided images (first left divided image, first right divided image) 53L, 54L and right divided images (second left divided image, second right divided image) 53R, 54R.

[0062] <Trimming process> Next, in the overhead line fitting abnormality detection device 1, similarly to the first and second embodiments described above, the image processing unit 14 performs outside-the-photographing-target-range trimming on the left divided image 53L, 54L and the right divided image 53R, 54R after left and right division (step S44). More specifically, as shown in Fig. 21, the image processing unit 14 performs outside-the-photographing-target-range trimming on the left divided image and the right divided image stored in the data storage unit 13 using the outside-the-photographing-target-range trimming unit 141, and stores the trimmed images after trimming in the data storage unit 13.

[0063] 23 is a conceptual diagram for explaining the trimming ranges for the left split images 53L, 54L and the right split images 53R, 54R according to the third embodiment. The trimming process is basically the same as that of the first embodiment. The trimming ranges according to the trimming process of the third embodiment are a total of four areas, including a trimming range 63L at the left end of the left split image 53L, a trimming range 64L at the right end of the left split image 54L, a trimming range 63R at the left end of the right split image 53R, and a trimming range 64R at the right end of the right split image 54R.

[0064] Fig. 24 is a conceptual diagram showing left trimmed images 57L, 58L and right trimmed images 57R, 58R after trimming in the third embodiment. As shown in Fig. 24, by trimming the left end of the left divided image 53L and the right end of the left divided image 54L, and the left end of the right divided image 53R and the right end of the right divided image 54R, it is possible to cut out a partial image of the overhead wire C that is outside the range to be photographed and is not subject to abnormality detection.

[0065] <Image inversion processing> Next, in the overhead line fitting abnormality detection device 1, as in the first and second embodiments described above, the image processing unit 14 performs left-right flipping on some of the images after division and trimming, i.e., the left trimmed image 57L and the right trimmed image 57R (step S28). More specifically, as shown in Fig. 21, the image processing unit 14 performs left-right flipping on the left trimmed image and the right trimmed image after division and trimming stored in the data storage unit 13, using the image left-right flipping processing unit 142, and stores them in the data storage unit 13 as the left trimmed image and the right trimmed image, respectively.

[0066] FIG. 25 is a conceptual diagram showing a left trimmed inverted image 59L, a left trimmed image 58L, a right trimmed inverted image 59R, and a right trimmed image 58R in the third embodiment. The upper part of FIG. 25 shows a left trimmed image 58L after trimming, a left trimmed inverted image 59L further inverted after trimming, and a right trimmed inverted image 58R after trimming, and a right trimmed inverted image 59R further inverted after trimming. In the third embodiment, as shown in FIG. 25, the orientation of the overhead line fittings 42A1 and 42A2 of the overhead line A in the left trimmed inverted image 59L, the orientation of the overhead line fittings 42B1 and 42B2 of the overhead line B in the left trimmed image 58L, the orientation of the overhead line fittings 42A1 and 42A2 of the overhead line A in the right trimmed inverted image 59R, and the orientation of the overhead line fittings 42B1 and 42B2 of the overhead line B are all unified, so that the accuracy of machine learning can be improved. In addition, when the overhead line metal fitting to be detected is not present in the image, as in the case of the right trimmed image 58R in FIG. 25, it is not used in the learning data set.

[0067] <Processed image provision processing> Next, the overhead line fitting abnormality detection device 1 supplies the left cropped inverted image 59L, the left cropped image 58L, the right cropped inverted image 59R, and the right cropped image 58R as processed images to a machine learning system (arithmetic device) (not shown) (step S48). The machine learning system (not shown) performs machine learning based on the processed images, the left cropped inverted image 59L, the left cropped image 58L, and the right cropped inverted image 59R, and detects abnormalities in the overhead line fittings (distortion of the fittings, the presence or absence of screws, etc.).

[0068] According to the overhead line fitting abnormality detection device and the overhead line fitting abnormality detection method of the present embodiment configured as described above, it is possible to improve the detection accuracy of abnormalities in overhead line fittings. Also, by reducing the amount of data through trimming, it is possible to reduce the burden on machine learning and inference processing. Furthermore, by using fisheye lenses for the line sensor cameras 10L and 10R, it is possible to configure the equipment in a manner that is less affected by installation conditions.

[0069] In the above-described first to third embodiments, the right image is inverted, but this is not limited to the above, and one of the images may be inverted so that the orientation of the overhead line fittings coincides with the center of the camera.

[0070] In addition, it is possible to select and discard the configurations given in the above embodiment, or to change them to other configurations as appropriate, without departing from the spirit of the present invention. [Explanation of symbols]

[0071] 1. Cable fitting abnormality detection device 2. Railway vehicles 21 Vehicle roof 10L, 10R Line sensor camera 11, 11, 11 Lighting 12 In-vehicle PC 13 Data Storage Section 14 Image processing section 141 Cropping area outside the shooting range 142 Image left-right inversion processing unit 143 Image left / right division processing unit 144 Outer image deletion processing unit A~C overhead lines 40A~40C Contact wire 41A~41C Suspension wire 42A~42C Overhead Wiring Fittings 42A1, 42A2, 42B1, 42B2, 42C1, 42C2 overhead line fittings 51L Left cropped image 51R Right cropped image 52R Right inverted image 53L, 54L Left split image 53R, 54R Right split image

Claims

1. a first line sensor camera and a second line sensor camera which are cameras for photographing overhead wire fittings using fisheye lenses while a railroad vehicle is traveling, the first line sensor camera and the second line sensor camera being disposed a predetermined distance apart on the left and right sides of the direction of travel of the railroad vehicle; a trimming unit that trims a portion outside a range of a subject to be photographed from a first image photographed by the first line sensor camera and trims a portion outside the range of a subject to be photographed from a second image photographed by the second line sensor camera; a left-right reversal processing unit that left-right reverses one of the trimmed first image and the trimmed second image so that an orientation of the captured overhead line fitting coincides with a center position of the first line sensor camera in the first image and a center position of the second line sensor camera in the second image, providing, as processed images for detecting an abnormality in the overhead line metal fittings, an image that has not been horizontally inverted by the horizontal inversion processing unit and an inverted image that has been horizontally inverted by the horizontal inversion processing unit, out of the first trimmed image or the second trimmed image; A wire fitting abnormality detection device characterized by the above.

2. an image left / right division processing unit that divides the first image into left and right parts from a center position of the first line sensor camera in the first image, and divides the second image into left and right parts from a center position of the second line sensor camera in the second image; an outer image deletion processing unit that deletes, from among the divided images divided from the first image, divided images that correspond to an outer side of the vehicle with respect to a center position of the first line sensor camera, and deletes, from among the divided images divided from the second image, divided images that correspond to an outer side of the vehicle with respect to a center position of the second line sensor camera, The trimming unit includes: trimming the outside of the range to be photographed from the first divided image remaining without being deleted, and trimming the outside of the range to be photographed from the second divided image remaining without being deleted; The left-right reversal processing unit is flipping one of the first divided image and the second divided image horizontally so that a direction of the overhead line fittings photographed in the trimmed first divided image coincides with a direction of the overhead line fittings photographed in the trimmed second divided image; 2. The overhead line fitting abnormality detection device according to claim 1.

3. an image left / right division processing unit that divides the first image into left and right from a center position of the first line sensor camera in the first image, and divides the second image into left and right from a center position of the second line sensor camera in the second image, The trimming unit includes: trimming an area outside the imaging target range from a first left divided image and a first right divided image obtained by dividing the first image into left and right parts, and trimming an area outside the imaging target range from a second left divided image and a second right divided image obtained by dividing the second image into left and right parts, The left-right reversal processing unit is flipping one of the first left split image and the first right split image left and right so that orientations of overhead line fittings in the trimmed first left split image and the first right split image match, and flipping one of the second left split image and the second right split image left and right so that orientations of overhead line fittings in the trimmed second left split image and the second right split image match; 2. The overhead line fitting abnormality detection device according to claim 1.

4. a step of photographing the overhead line fittings while the railway vehicle is traveling, using a first line sensor camera and a second line sensor camera each equipped with a fisheye lens, the first line sensor camera and the second line sensor camera being disposed at a predetermined distance to the left and right of the direction of travel of the railway vehicle; trimming a portion outside a range of a subject captured from a first image captured by the first line sensor camera and trimming a portion outside a range of a subject captured from a second image captured by the second line sensor camera; a step of left-right inverting one of the trimmed first image and the trimmed second image so that an orientation of the captured overhead line fitting coincides with a center position of the first line sensor camera in the first image and a center position of the second line sensor camera in the second image; providing, as processed images for detecting an abnormality in the overhead line fittings, an image that has not been horizontally inverted by the horizontal inversion step and an inverted image that has been horizontally inverted by the horizontal inversion step, out of the first trimmed image or the second trimmed image; A method for detecting an abnormality in a wire fitting, comprising:

5. dividing the first image into left and right parts from a center position of the first line sensor camera in the first image, and dividing the second image into left and right parts from a center position of the second line sensor camera in the second image; a step of deleting, from among the divided images obtained by dividing the first image, a divided image that corresponds to an outer side of the vehicle from a center position of the first line sensor camera, and deleting, from among the divided images obtained by dividing the second image, a divided image that corresponds to an outer side of the vehicle from a center position of the second line sensor camera, The step of trimming the outside of the shooting range includes: trimming the outside of the range to be photographed from the first divided image remaining without being deleted, and trimming the outside of the range to be photographed from the second divided image remaining without being deleted; The step of flipping left and right includes: flipping one of the first divided image and the second divided image horizontally so that a direction of the overhead line fittings photographed in the trimmed first divided image coincides with a direction of the overhead line fittings photographed in the trimmed second divided image; 5. The method for detecting an abnormality in a metal fitting according to claim 4.

6. further comprising a step of dividing the first image into left and right parts from a center position of the first line sensor camera in the first image, and dividing the second image into left and right parts from a center position of the second line sensor camera in the second image, The step of trimming the outside of the shooting range includes: trimming an area outside the imaging target range from a first left divided image and a first right divided image obtained by dividing the first image into left and right parts, and trimming an area outside the imaging target range from a second left divided image and a second right divided image obtained by dividing the second image into left and right parts, The step of flipping left and right includes: flipping one of the first left split image and the first right split image left and right so that orientations of overhead line fittings in the trimmed first left split image and the first right split image match, and flipping one of the second left split image and the second right split image left and right so that orientations of overhead line fittings in the trimmed second left split image and the second right split image match; 5. The method for detecting an abnormality in a metal fitting according to claim 4.

Images