Abnormality detection method, abnormality detection device, and computer program

Abstract

To provide an abnormality detection method, abnormality detection device and computer program, which enable efficient detection of troubles of installations including electrical equipment mounted on utility poles.SOLUTION: An abnormality detection method is provided, comprising acquiring a time-series of captured images of installations including electrical equipment mounted on utility poles provided along a road, and detecting abnormality in the installations on the basis of the obtained time-series of images.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an abnormality detection method, an abnormality detection device, and a computer program. [Background technology]

[0002] Low-voltage distribution lines from distribution transformers to consumers use electrical equipment including low-voltage lines and electric light lines attached to utility poles along roads. Electrical equipment on such low-voltage distribution lines must be inspected once at legally specified intervals. However, with the huge number of utility poles installed, there are a great many inspection points, and visually discovering defects in the electrical equipment requires a great deal of effort. In addition, the number of inspection points is so large that it is not realistic to increase the frequency of patrols.

[0003] Patent Document 1 discloses a technology for measuring the temperature of pole-mounted equipment and detecting abnormalities in the pole-mounted equipment by combining visible images obtained by photographing a utility pole equipped with pole-mounted equipment with a visible light camera and thermal images obtained by photographing the pole-mounted equipment with an infrared camera. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2002-366953 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, the technology of Patent Document 1 only considers temperature abnormalities in pole-mounted equipment. Abnormalities in equipment, including electrical equipment mounted on utility poles, include not only temperature abnormalities but also various other abnormal conditions. For this reason, it is desirable to improve the efficiency of the work of detecting malfunctions in equipment, including electrical equipment.

[0006] The present invention has been made in consideration of the above circumstances, and aims to provide an anomaly detection method, an anomaly detection device, and a computer program that can efficiently detect malfunctions in equipment, including electrical equipment mounted on utility poles. [Means for solving the problem]

[0007] The present application includes multiple means for solving the above-mentioned problems, and one example is an anomaly detection method that acquires a time series of images of equipment, including electrical equipment mounted on utility poles installed along roads, and detects an abnormality in the equipment based on the acquired time series of images. Effect of the Invention

[0008] According to the present invention, malfunctions in facilities including electrical equipment mounted on utility poles can be efficiently detected. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram showing an example of inspection of electrical equipment according to an embodiment of the present invention. [Diagram 2] FIG. 1 is a diagram illustrating an example of a structure of an electrical device. [Diagram 3] FIG. 2 is a diagram illustrating an example of a configuration of an imaging device. [Figure 4] FIG. 4 is a diagram showing an example of shooting information. [Diagram 5] FIG. 1 illustrates an example of a configuration of an abnormality detection device. [Figure 6] FIG. 2 is a diagram showing an example of the configuration of a utility pole information DB. [Figure 7] A figure showing an example of detection of electrical equipment and detection of abnormalities in electrical equipment using a first learning model. [Figure 8] FIG. 2 illustrates an example of tracking an object. [Figure 9] FIG. 11 is a diagram showing a first example of a detection result. [Figure 10] FIG. 11 is a diagram showing a second example of a detection result. [Figure 11]FIG. 1 is a diagram illustrating an example of electrical equipment that is a target for abnormality detection. [Figure 12] FIG. 10 is a diagram illustrating an example of a procedure for detecting an abnormality by the abnormality detection device. [Figure 13] FIG. 1 is a diagram showing an example of a crow's nest as an abnormality detection target. [Figure 14] FIG. 13 is a diagram illustrating an example of a vine as an abnormality detection target. [Figure 15] 1A and 1B are diagrams illustrating examples of signs and guide boards as targets for abnormality detection; [Figure 16] FIG. 1 is a diagram illustrating an example of a communication facility as a target for anomaly detection. [Figure 17] A figure showing an example of equipment detection and equipment anomaly detection using a first learning model. [Figure 18] FIG. 2 illustrates an example of tracking an object. [Figure 19] FIG. 13 is a diagram illustrating an example of an abnormality detection target. [Figure 20] 11 is a diagram illustrating another example of a procedure for detecting an abnormality by the abnormality detection device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] (First embodiment) Hereinafter, the present invention will be described with reference to the drawings showing the embodiments. FIG. 1 is a diagram showing an example of the inspection of electrical equipment according to this embodiment. A utility pole 20 is installed along a road, and electrical equipment 25 is mounted on the utility pole 20. The electrical equipment 25 includes an arm 21 as a second electrical equipment fixed to the utility pole 20, and an insulator (e.g., a low-voltage pin insulator) 22 fixed to the arm 21. The arm 21 is, for example, a plate-shaped member with a cross section bent into a U-shape, and is fixed to the utility pole 20 with a bolt or the like. The insulator 22 is fixed to both ends of the arm 21, and a low-voltage line (200V) 30 is fixed to the insulator 22 with a binding wire. Note that the electrical equipment 25 mounted on the utility pole 20 is only an example, and is not limited to the structure shown in FIG. 1. Also, an electric light line (100V) may be used instead of the low-voltage line 30.

[0011] A contractor who performs inspection work on electrical equipment 25 photographs the electrical equipment 25 on each utility pole 20 with the photographing device 10 while driving a vehicle 1 equipped with the photographing device 10. As shown in Fig. 1, when the vehicle 1 drives near the utility pole 20, photographing the electrical equipment 25 with the photographing device 10 makes it possible to obtain a plurality of images (videos) in a time series. The number of images in the time series can be, for example, about 50 to 100 per utility pole 20, but is not limited to this number.

[0012] As described above, in this embodiment, a time series of images of electrical equipment 25 mounted on a utility pole 20 is obtained using the photographing device 10 from a traveling vehicle 1, thereby making the inspection work more efficient than when an inspection worker inspects the equipment visually.

[0013] In the following, in this specification, an abnormality in the electrical equipment 25 will be described taking as an example an abnormality in a binding wire for fixing the low-voltage wire 30 to the insulator 22. Note that, as will be described later, the abnormality in the electrical equipment 25 is not limited to an abnormality in the binding wire.

[0014] Fig. 2 is a diagram showing a schematic example of the structure of electrical equipment 25. Fig. 2A shows a schematic example of a case where the bind wire is normal, and Fig. 2B shows a schematic example of a case where the bind wire is abnormal. As shown in Fig. 2A, in a normal facility state, the low-voltage wire 30 is held to the insulator 22 by the bind wire (vinyl-coated aluminum bi-and wire) 23. The bind wire 23 is wound around the insulator 22 and the low-voltage wire 30 so that the low-voltage wire 30 is fixed to the insulator 22. Since the low-voltage wire 30 is fixed to a predetermined position on the insulator 22, it is separated by a required distance from metal parts such as the cross arm 21.

[0015] As shown in Fig. 2B, if the binding wire 23 comes undone or breaks, the low-voltage wire 30 comes off the insulator 22 and falls onto the cross arm 21. If the low-voltage wire 30 is swayed by an external force such as wind, the low-voltage wire 30 rubs against the cross arm 21, causing the insulating coating of the low-voltage wire 30 to wear away, and the electric wire inside the low-voltage wire 30 may come into contact with the metal part of the cross arm 21, leading to a short circuit. In this embodiment, an abnormality in the binding wire 23 as shown in Fig. 2B can be efficiently detected.

[0016] 3 is a diagram showing an example of the configuration of the photographing device 10. The photographing device 10 can be configured, for example, as a smartphone or a tablet terminal. The photographing device 10 includes a control unit 11 that controls the entire device, a camera unit 12, a GPS unit 13, a communication unit 14, and a storage unit 15.

[0017] The control unit 11 may be configured by incorporating a required number of CPUs (Central Processing Units), MPUs (Micro-Processing Units), GPUs (Graphics Processing Units), etc. The control unit 11 may also be configured by combining DSPs (Digital Signal Processors), FPGAs (Field-Programmable Gate Arrays), etc.

[0018] The camera unit 12 can capture video (time-series images). The camera unit 12 can automatically start and end image capture under the control of the control unit 11. For example, the vehicle 1 equipped with the image capture device 10 starts image capture when it enters the image capture area of ​​the utility pole 20, and ends image capture when it leaves the image capture area. The image capture area may be determined based on the range in which the electrical equipment 25 mounted on the utility pole 20 can be captured at a required size, based on the position information of the utility pole 20 and the position information of the road.

[0019] The control unit 11 has a timing function and can identify the time point at which a video image captured by the camera unit 12 was captured.

[0020] The GPS unit 13 acquires the position information of the photographing device 10, i.e., the position information of the vehicle 1 on which the photographing device 10 is mounted. The position information can be expressed by longitude and latitude. Under the control of the control unit 11, the GPS unit 13 acquires the position information at the time when the video is being photographed by the camera unit 12. This makes it possible to acquire the position information of the vehicle 1 in association with the positions of each utility pole 20 installed along the road.

[0021] The communication unit 14 includes a communication module, and realizes a communication function with an abnormality detection device 50, which will be described later.

[0022] The storage unit 15 can be configured, for example, with a semiconductor memory or the like, and can store time-series images captured by the camera unit 12. The time-series images can be stored in the storage unit 15 after being divided for each utility pole 20 (i.e., for each image captured from the start to the end of image capture).

[0023] The photographing device 10 transmits photographing information, including time-series images photographed by the camera unit 12 and stored in the storage unit 15, via the communication unit 14 to the abnormality detection device 50 described below.

[0024] FIG. 4 is a diagram showing an example of the photographing information. The photographing information includes a time-series image group ID, a time-series image, a position ID, position information, and time information. The time-series image group ID is an identifier for identifying a plurality of time-series images obtained by photographing the electrical equipment 25 of one utility pole 20 as one image group. For example, if 300 pieces of electrical equipment 25 of utility poles 20 are photographed during an inspection patrol at a certain timing, there are 300 time-series image group IDs. The time-series images are the number of time-series images (number of frames) included in the time-series image group ID. The position ID is an identifier for identifying the position information obtained when the time-series images corresponding to the time-series image group ID were photographed. The position information represents the position information for each position ID and is represented by latitude and longitude. The time information is the date and time when the time-series images corresponding to the time-series image group ID were photographed. As shown in FIG. 4, the position information and the time information are assigned to the image for each frame of the time-series image group.

[0025] 5 is a diagram showing an example of the configuration of an abnormality detection device 50. The abnormality detection device 50 is installed in an office or facility of a contractor who performs inspection work on electrical equipment 25. The abnormality detection device 50 includes a control unit 51 that controls the entire device, a communication unit 52, a memory 53, a display unit 54, an operation unit 55, a storage unit 56, and a utility pole information DB (database) 60. The abnormality detection device 50 can be configured, for example, by a PC (personal computer), a smartphone, or a tablet terminal.

[0026] The control unit 51 may be configured by incorporating a required number of CPUs, MPUs, GPUs, etc. The control unit 51 may also be configured by combining a DSP, an FPGA, etc.

[0027] The communication unit 52 includes a communication module and has a function of communicating with the image capturing device 10 .

[0028] The display unit 54 includes a liquid crystal display panel, an organic EL display panel, etc. The display unit 54 may be an external display device connected to the abnormality detection device 50 via a communication network.

[0029] The operation unit 55 is composed of, for example, a mouse, a keyboard, etc. The operation unit 55 is also composed of a touch panel, etc., and can perform operations on characters, images, or icons displayed on the display unit 54. The operation unit 55 may also be one that operates an external display device.

[0030] The storage unit 56 can be configured, for example, with a hard disk or a semiconductor memory, and stores a computer program 57 (program product), a first learning model 58, a second learning model 59, and required information.

[0031] The computer program 57 is a computer program that runs on the abnormality detection device 50. The computer program 57 can be read by a recording medium reading unit from a recording medium (for example, an optically readable disk storage medium such as a CD-ROM) on which the computer program 57 is recorded, and the read computer program 57 can be stored in the storage unit 56. The computer program 57 may also be downloaded from an external device via the communication unit 52 and stored in the storage unit 56.

[0032] Memory 53 can be configured with a semiconductor memory such as a static random access memory (SRAM), a dynamic random access memory (DRAM), or a flash memory. A computer program 57 can be loaded into memory 53, and control unit 51 can execute computer program 57. Control unit 51 can execute processing defined by computer program 57. In other words, processing by control unit 51 is also processing by computer program 57.

[0033] Details of the first learning model 58 and the second learning model 59 will be described later.

[0034] The utility pole information DB 60 is a database that records information about utility poles. The utility pole information DB 60 may be provided on a data server or a cloud separate from the abnormality detection device 50.

[0035] FIG. 6 is a diagram showing an example of the configuration of the utility pole information DB 60. The utility pole information DB 60 includes information such as a utility pole ID, location information, electrical equipment details, and inspection implementation date and time. The utility pole ID is an identifier for identifying each utility pole 20. The location information indicates the installation location of the utility pole identified by the utility pole ID using latitude and longitude. The electrical equipment details include information that shows the details such as the ID, number, and type of electrical equipment installed on the utility pole identified by the utility pole ID. In the example of FIG. 6, the information includes the ID of the insulator, binding wire, etc. The inspection implementation date and time indicates the date and time of the most recent inspection. Based on the inspection implementation date and time, it is possible to identify the utility pole to be inspected in the next patrol inspection.

[0036] Next, a method for detecting an abnormality in an electrical facility by the abnormality detection device 50 of this embodiment will be described.

[0037] The control unit 51 (abnormality detection device 50) acquires from the photographing device 10 time-series images of the electrical equipment 25 mounted on the utility pole 20 installed along the road, and detects abnormalities in the electrical equipment 25 based on the acquired time-series images. In this way, since time-series images of the electrical equipment 25 mounted on the utility pole 20 are acquired, the inspection work can be made more efficient than when an inspection worker inspects the equipment visually.

[0038] Furthermore, the control unit 51 can obtain time-series images of electrical equipment mounted on utility poles using the imaging device 10 provided on the vehicle 1 (moving body) moving along a road.

[0039] Next, a specific method for detecting an abnormality will be described.

[0040] 7 is a diagram showing an example of detection of electrical equipment and detection of anomalies in electrical equipment by the first learning model 58. For example, a convolutional neural network (CNN) method is used for the first learning model 58. More specifically, a YOLO (You Only Look Once) method based on a convolutional neural network can be used.

[0041] When the control unit 51 receives a time series of images of the electrical equipment 25, it can detect abnormalities in the electrical equipment 25 by inputting the acquired time series images into a first learning model 58, which outputs the presence or absence of an abnormality in the electrical equipment 25 for each time series image.

[0042] The first learning model 58 detects objects and classifies the detected objects. The first learning model 58 applies grid cells to each of the input time-series images, and each cell applies a bounding box to an object in the image, and if an object exists, classifies the class of the object. In this embodiment, the objects include the cross arm 21 and the insulator 22 (OBJ1, OBJ2), and the classes are classified into two classes: the bind wire of the insulator 22 is normal, and the bind wire of the insulator 22 is abnormal. The "bind wire of the insulator 22 is normal" class indicates that the electrical equipment (first electrical equipment) is normal (OK), and the "bind wire of the insulator 22 is abnormal" class indicates that the electrical equipment (first electrical equipment) is abnormal (NG). The first learning model 58 outputs class probabilities.

[0043] As shown in Fig. 7, time-series images obtained by photographing a utility pole 20 are frames 1 to 68. When the image of frame 1 is input to first learning model 58, first learning model 58 outputs OBJ1: OK and OBJ2: NG. OBJ1 and OBJ2 are insulators 22, OK indicates that the bind wire is normal, and NG indicates that the bind wire is abnormal.

[0044] Next, suppose that when the image of frame 2 is input to the first learning model 58, the first learning model 58 outputs OBJ1: OK and OBJ2: NG. Similar processing is performed for the images of subsequent frames, and when the image of the final frame 68 is input to the first learning model 58, the first learning model 58 outputs OBJ1: OK and OBJ2: NG.

[0045] And let's say that for a total of 68 frames, the number of OKs for OBJ1 is 68 and the number of NGs is 0, while the number of OKs for OBJ2 is 3 and the number of NGs is 65. If the normality / abnormality judgment threshold for OBJ is Th, then if the number of OKs is equal to or greater than the judgment threshold Th, then OBJ is judged to be normal, and if the number of NGs is equal to or greater than the judgment threshold Th, then OBJ is judged to be abnormal. For example, if the judgment threshold Th is set to 90% of the total number of frames, then it can be detected that OBJ1 is normal and OBJ2 is abnormal.

[0046] As described above, when the control unit 51 inputs a time series of images taken of the electrical equipment 25, the control unit 51 can input the acquired time series of images to the first learning model 58, which detects the first electrical equipment (insulator 22) for each time series image and outputs the presence or absence of an abnormality in the first electrical equipment, and detect an abnormality in the first electrical equipment (an abnormality in the bind wire).

[0047] Furthermore, the control unit 51 can calculate the probability of abnormality of the electrical equipment 25 based on the number of abnormalities of the binding lines (abnormalities of one electrical equipment) detected for each time-series image taken over a predetermined period or taken a predetermined number of times. For example, if the total number of frames is N and the number of frames in which the binding lines are detected as abnormal is n, the probability of abnormality is (n / N). Note that the method of calculating the probability of abnormality is not limited to the above-mentioned method. For example, a value obtained by statistically processing (e.g., average value, median value, etc.) the probability (probability) of abnormality output by the first learning model 58 for all frames for the image of each frame may be used. Similarly, a value obtained by statistically processing (e.g., average value, median value, etc.) the probability (probability) of normality output by the first learning model 58 for all frames for the image of each frame may be used.

[0048] To train (generate) the first learning model 58, a plurality of training images of electrical equipment and teacher data in which objects in the images are classified into classes may be used. For example, a large number of images showing normal OBJs and teacher labels indicating that the OBJs are normal are prepared, and the first learning model 58 is trained to output that the OBJs are normal when the images are input to the first learning model 58. Also, a large number of images showing abnormal OBJs and teacher labels indicating that the OBJs are abnormal are prepared, and the first learning model 58 is trained to output that the OBJs are abnormal when the images are input to the first learning model 58. Note that the first learning model 58 is not limited to YOLO, and other methods such as R-CNN and SSD (Single Shot MultiBox Detector) may be used.

[0049] Next, object tracking will be described.

[0050] Among the electrical equipment 25 mounted on the utility pole 20, the insulator 22 to which the bind wire is fixed has a relatively small size. For this reason, it may be difficult to directly track the insulator 22 using time-series images captured by the imaging device 10 mounted on the vehicle 1. On the other hand, if the cross arm 21, whose relative position with respect to the insulator 22 is predetermined and whose shape is larger than that of the insulator 22, is used, tracking can be made easier.

[0051] The second learning model 59 is an object tracking model, and may use a method such as Bytetrack, but is not limited thereto, and may use other methods such as SMILEtrack, BotSort, StrongSort, etc. The second learning model 59 can track the cross arm 21 (bounding box) and the insulator 22 (bounding box) detected by the first learning model 58 for each frame of a time series of images.

[0052] That is, the control unit 51 detects the position of the cross arm 21 (second electrical equipment) on each of the acquired time-series images, and based on the detected position of the cross arm 21 and the relative position of the cross arm 21 and the insulator 22 (first electrical equipment), it can detect the position of the insulator 22 on each of the acquired time-series images.

[0053] FIG. 8 is a diagram showing an example of object tracking. For convenience, three frames, frame (i-1), frame (i), and frame (i+1), are considered from the time series of images. OBJ, OBJ1, and OBJ2 represent the cross arm 21, the insulator 22 on one side, and the insulator 22 on the other side, respectively. It is assumed that OBJ, OBJ1, and OBJ2 are detected in the image of frame (i-1). The second learning model 59 specifies the positions (positions of the bounding boxes) of the two insulators 22 from the reference position (e.g., the upper left position) of the bounding box of the cross arm 21 based on the relative positions of the cross arm 21 and the two insulators 22. Similarly, it is assumed that OBJ, OBJ1, and OBJ2 are detected in the image of frame (i). The second learning model 59 identifies the positions of the two insulators 22 (positions of the bounding boxes) from the reference position of the bounding box of the cross arm 21 (for example, the upper left position) based on the relative positions of the cross arm 21 and each of the two insulators 22. The same is true for the image of frame (i+1). In this way, the second learning model 59 can track the positions of the two insulators 22 in each frame based on the position of the cross arm 21.

[0054] Next, a display example of the detection result will be described.

[0055] FIG. 9 is a diagram showing a first example of the detection result. The detection result screen 100 is displayed on the display unit 54 or an external display device. The detection result screen 100 displays a selection field for selecting an inspection area. By selecting an inspection area, the person in charge displays the inspection results and inspection date and time in the selected inspection area. In the example of FIG. 9, a road map of the inspection area is displayed. The road map can be enlarged and reduced, and moved in 360 degrees including north, south, east and west. The position of the utility pole where the inspection was carried out is displayed on the road map. In the example of FIG. 9, rectangular marks 101 and 102 are displayed at the positions of the utility poles. The mark 101 indicates that the electrical equipment is normal, and the mark 102 indicates that the electrical equipment is abnormal. In the example of FIG. 9, the mark 102 indicating an abnormality is filled with a predetermined color (for example, red) so as to be noticeable. Note that the marks indicating normality and abnormality are not limited to the example of FIG. 9, and may be displayed in a manner that allows the normality and abnormality to be distinguished from each other. For example, the shape of the mark may be made different between normality and abnormality, or the color or pattern may be made different between normality and abnormality. When the person in charge selects the mark 102 and performs a predetermined operation (for example, a click operation, a touch operation, etc.), a detection result screen 110 described below can be displayed.

[0056] As described above, the control unit 51 can display on a map the position of the utility pole on which the electrical equipment in which an abnormality has been detected is installed. As a result, the position of the utility pole on which the electrical equipment abnormality has been detected is displayed on a map, it is possible to easily grasp the utility pole in need of replacement or repair of the electrical equipment, and it is possible to take prompt action.

[0057] FIG. 10 is a diagram showing a second example of the detection result. As described above, when the abnormality mark 102 is operated on the detection result screen 100, the detection result screen 110 is displayed. On the detection result screen 110, the ID of the utility pole where the abnormality of the electrical equipment was detected, the abnormality probability, a detailed image of the abnormality location, and the number of frames detected as normal and the number of frames detected as abnormal for each insulator 22 in the entire time-series image are displayed. In the example of FIG. 10, of the two insulators 22, insulator ID: 215 has 50 frames of OK and 0 frames of NG, and insulator ID: 216 has 5 frames of OK and 45 frames of NG. As a result, the detailed image displays that insulator ID: 215 is normal (OK) and insulator ID: 216 is abnormal (NG).

[0058] In FIG. 10, the detailed image may be an image of one frame, or images of all or part of a plurality of frames of the time-series image group may be displayed in a selectable manner. When an image of one frame is displayed, for example, an image with the highest probability of abnormality may be displayed among all the images of the frames of the time-series image group. When images of a plurality of frames are displayed in a selectable manner, for example, a predetermined number of images may be displayed in order of the probability of abnormality. In addition, by operating the "NG" portion in the detailed image, an enlarged image of an image including the insulator 22 detected as abnormal may be displayed. This makes it easy to grasp the defective state of the binding wire (for example, the urgency of repair, etc.). In addition, by operating the "OK" portion in the detailed image, an enlarged image of an image including the insulator 22 detected as normal may be displayed, and the person in charge can confirm that there is no mistake that it is normal (false detection).

[0059] As described above, the control unit 51 can output an image of the electrical equipment in which an abnormality has been detected, thereby making it possible to objectively grasp the abnormal state of the electrical equipment.

[0060] 11 is a diagram showing an example of electrical equipment subject to abnormality detection. As described above, the first electrical equipment includes the insulator 22, the second electrical equipment includes the cross arm 21 to which the insulator 22 is fixed, and an abnormality in the first electrical equipment includes an abnormality in a binding wire for fixing the low-voltage line 30 (or electric light line) to the insulator 22. Note that the electrical equipment is not limited to these.

[0061] 11, the first electric equipment may include a discharge clamp insulator, the second electric equipment may include a cross arm to which the discharge clamp insulator is fixed, and the abnormality of the first electric equipment may include an abnormality in the discharge clamp. In this case, the control unit 51 detects the position of the cross arm (second electric equipment) on each of the acquired time-series images, and can detect the position of the discharge clamp insulator on each of the acquired time-series images based on the detected position of the cross arm and the relative position between the cross arm and the discharge clamp insulator (first electric equipment).

[0062] 11, the first electrical equipment may include a bushing or gasket, the second electrical equipment may include a pole transformer to which the bushing or gasket is fixed, and the abnormality of the first electrical equipment may include an abnormality in the bushing or gasket. In this case, the control unit 51 can detect the position of the pole transformer (second electrical equipment) on each of the acquired time-series images, and detect the position of the bushing or gasket on each of the acquired time-series images based on the detected position of the pole transformer and the relative position between the pole transformer and the bushing or gasket (first electrical equipment).

[0063] Although not shown, the abnormality detection device 50 of this embodiment can also be applied to detecting vines wrapped around utility poles and support wires, and detecting crow nests.

[0064] 12 is a diagram showing an example of a procedure for detecting an abnormality by the abnormality detection device 50. For convenience, the following description will be given with the control unit 51 acting as the main actor in the process. The control unit 51 acquires time-series images of the electrical equipment 25 mounted on the utility pole 20 (S11), and detects the cross arm 21 based on the acquired images (S12). The control unit 51 detects the insulator 22 based on the detected position of the cross arm 21 and the relative position of the cross arm 21 and the insulator 22 (S13). The control unit 51 detects the presence or absence of an abnormality in the bind wire for fixing the low-voltage line 30 (or electric light line) to the insulator 22 (S14).

[0065] The processing of steps S12 to S14 described above includes detection of the cross arm 21 and insulator 22 by the first learning model 58, and detection processing of whether the insulator 22 is normal or abnormal (more specifically, detection of whether the bind wire is normal or abnormal), and tracking processing by the second learning model 59 of the cross arm 21 and insulator 22 detected by the first learning model 58 in the images of each frame.

[0066] The control unit 51 determines whether or not the processing of all the time-series images of the utility pole 20 has been completed (S15), and if the processing has not been completed (NO in S15), the control unit 51 continues the processing from step S12 onwards for the image of the next frame. If the processing of all the time-series images has been completed (YES in S15), the control unit 51 acquires position information corresponding to the group of time-series images of the electrical equipment 25 of the utility pole 20 (S16).

[0067] The control unit 51 calculates the number of abnormalities in the binding lines (the number of frames detected as abnormal) and the number of normal binding lines (the number of frames detected as normal) for each time-series image (S17), and detects the presence or absence of an abnormality in the binding lines of the utility pole 20 (S18). The control unit 51 outputs the detection result (S19), and ends the process.

[0068] As described above, since the insulator 22 (the binding wire fixed to the insulator 22) is small in size among the electrical equipment 25, it may become invisible if it overlaps with electric wires or other equipment. In such a situation, it is not realistic from the viewpoint of detection accuracy to detect the presence or absence of an abnormality in the binding wire based on a single image taken at a required distance. However, in this embodiment, the presence or absence of an abnormality in the electrical equipment is detected based on a plurality of time-series images taken over a predetermined period of time, rather than a single image, so that not only can the detection accuracy be improved, but also the efficiency of the inspection work can be improved compared to visual inspection work.

[0069] In addition, when detecting the presence or absence of an abnormality in electrical equipment using multiple time-series images, even if the electrical equipment being detected for an abnormality (e.g., an insulator 22, a binding wire fixed to the insulator 22, etc.) is small in shape and difficult to track between frames, it is possible to track electrical equipment (e.g., arm 21) that is larger than the electrical equipment being detected for an abnormality, while tracking the smaller electrical equipment based on the relative position of the larger electrical equipment from a reference position, so that it is possible to detect the presence or absence of an abnormality even in small electrical equipment.

[0070] Furthermore, rather than detecting whether there is an abnormality in the electrical equipment from just one image, the final normality / abnormality of the electrical equipment is determined based on the number of normal / abnormal frames detected in each of multiple time-series images. Therefore, even if, for example, the target electrical equipment becomes invisible at a certain point during shooting, the presence or absence of an abnormality can be detected with high accuracy.

[0071] Second embodiment In the above-described first embodiment, insulators, discharge clamp insulators, bushings, gaskets, etc. have been given as examples of electrical equipment as targets for abnormality detection, but the targets for abnormality detection are not limited to these electrical equipment. Below, abnormalities in equipment including electrical equipment will be described. In this specification, "equipment" includes "electrical equipment" and "equipment other than electrical equipment." "Electrical equipment" is equipment that uses electricity, and "equipment other than electrical equipment" is, for example, equipment that does not use electricity.

[0072] First, the object of abnormality detection will be described.

[0073] FIG. 13 is a diagram showing an example of a crow's nest as an abnormality detection target. As shown in FIG. 13, a low-voltage wire 30 is held by a binding wire (vinyl-coated aluminum bi-and wire) 23 on an insulator 22. The binding wire 23 is wound around the insulator 22 and the low-voltage wire 30 so that the low-voltage wire 30 is fixed to the insulator 22. Since the low-voltage wire 30 is fixed at a predetermined position on the insulator 22, it is separated from metal parts such as the cross arm 21 by a required distance. As shown in FIG. 13, for example, a bird's nest such as a crow's nest is built on the upper surface of the cross arm 21. If the material of the nest includes a metal wire such as a hanger, an abnormality in the electric wire (e.g., poor insulation or short circuit) may occur when the metal wire comes into contact with an electric wire such as the low-voltage wire 30. The abnormality detection target is a bird's nest, and the abnormality of the equipment includes an abnormality in an electric wire (e.g., a low-voltage wire or a high-voltage wire).

[0074] Fig. 14 is a diagram showing an example of vines as an abnormality detection target. As shown in Fig. 13, a ground support wire is provided on a utility pole 20 to balance the force acting on the utility pole 20 and prevent the utility pole 20 from falling over. When vines get entangled with the ground support wire and extend upward along the ground support wire, the tip of the vine may come into contact with an electric wire such as low-voltage line 30, or may enter a storage box or the like that stores the electric wire and come into contact with the electric wire, which may cause an abnormality in the electric wire (e.g., a ground fault). The abnormality detection target is vines, and the equipment abnormalities include abnormalities in electric wires (e.g., low-voltage lines, high-voltage lines, etc.).

[0075] FIG. 15 is a diagram showing an example of a sign and a guide board as an abnormality detection target. As shown in FIG. 15A, a sign is attached to a utility pole. The example in FIG. 15A is an example showing a speed limit, but the sign also includes signs for no parking and temporary stop. FIGS. 15B and 15C show an example of a guide board. The guide board includes the name of an intersection point as in FIG. 15B, and destination information as in FIG. 15C. The sign and the guide board are facilities other than electrical facilities, and abnormalities in the facilities include abnormalities in the sign and the guide board themselves. Note that the example in FIG. 15 is merely an example, and is not limited thereto. The abnormality detection target is a sign or a guide board, and abnormalities in the facilities are abnormalities in the sign or the guide board.

[0076] FIG. 16 is a diagram showing an example of communication equipment as an abnormality detection target. As shown in FIG. 16, a fastener for fixing a suspension line for suspending a communication line is provided on an arm of a utility pole, and the suspension line is fixed by the fastener. A communication line is suspended from the suspension line by a hanger. Although not shown, the communication lines are connected to each other by a junction box. Abnormalities in the communication equipment include, for example, a fastener provided on the arm coming off, a broken or sagging suspension line or hanger, damage to a junction box or a lid coming off, rust or loosening (sagging) of a communication line, and the like. The abnormality detection target is communication equipment, and an abnormality in the equipment is an abnormality in the communication equipment.

[0077] Next, a method for detecting an abnormality in equipment including electrical equipment will be described.

[0078] FIG. 17 is a diagram showing an example of equipment detection and equipment anomaly detection by the first learning model 58. In the following, a bird nest will be described as an example of an anomaly detection target. The first learning model 58 is similar to the one illustrated in FIG. 7, and therefore a description thereof will be omitted. When a time series of images of equipment including electrical equipment is input, the control unit 51 can detect anomalies in the equipment including electrical equipment by inputting the acquired time series of images into the first learning model 58, which outputs the presence or absence of anomalies in the equipment including electrical equipment for each time series image.

[0079] The first learning model 58 detects objects and classifies the detected objects. The first learning model 58 applies grid cells to each of the input time-series images, and each cell applies a bounding box to an object in the image, and if an object exists, classifies the class of the object. In this embodiment, the objects are electric poles (OBJ1) and bird nests (OBJ2), and the classes are classified into two classes: the presence or absence of bird nests (OBJ2). If a bird nest exists, this is represented by "NG" in FIG. 17. The first learning model 58 can also output class probabilities.

[0080] 17, frames 1 to 60 are images of a time series obtained by photographing a utility pole 20. When the image of frame 1 is input to first learning model 58, first learning model 58 outputs OBJ2:NG (i.e., "birds nesting present").

[0081] Next, assume that when the image of frame 2 is input to the first learning model 58, the first learning model 58 outputs OBJ2:NG. Similar processing is performed for the images of the subsequent frames, and when the image of the final frame 60 is input to the first learning model 58, the first learning model 58 outputs OBJ2:NG.

[0082] Then, for all 60 frames, the number of OBJ2:NG is 60. If the determination threshold for OBJ2:NG is Th, then if the number of NG is equal to or greater than the determination threshold Th, it can be determined that there is a bird nest, i.e., that there is an abnormality in the equipment.

[0083] The control unit 51 can calculate the probability of an abnormality in a piece of equipment based on the number of abnormalities detected in each time-series image taken over a predetermined period of time or a predetermined number of images. For example, if the total number of frames is N and the number of frames detected as an abnormality in the equipment is n, the probability of an abnormality in the equipment is (n / N).

[0084] Next, object tracking will be described.

[0085] The second learning model 59 is an object tracking model, and may use a method such as Bytetrack, but is not limited thereto, and may use other methods such as SMILEtrack, BotSort, StrongSort, etc. The second learning model 59 can track the utility pole (OBJ1) and bird nest (OBJ2) detected by the first learning model 58 for each frame of a time series of images.

[0086] FIG. 18 is a diagram showing an example of object tracking. For convenience, three frames, frame (i-1), frame (i), and frame (i+1), are focused on among images in a time series. OBJ1 represents a utility pole, and OBJ2 represents a bird's nest. It is assumed that OBJ1 and OBJ2 are detected in the image of frame (i-1). The second learning model 59 identifies the position of the bird's nest (the position of the bounding box) from the reference position (e.g., the lower left position) of the bounding box of the utility pole based on the relative positions of the utility pole (OBJ1) and the bird's nest (OBJ2). Similarly, it is assumed that OBJ1 and OBJ2 are detected in the image of frame (i). The second learning model 59 identifies the position of the bird's nest (the position of the bounding box) from the reference position (e.g., the lower left position) of the bounding box of the utility pole based on the relative positions of the utility pole (OBJ1) and the bird's nest (OBJ2). The same is true for the image of frame (i+1). In this way, the second learning model 59 can track the location of the bird nest in each frame relative to the position of the utility pole.

[0087] In addition, since abnormalities in equipment (including electrical equipment) due to bird nests may occur in locations where bird nests exist, the locations of bird nests can be regarded as the locations of equipment that is the target of abnormality detection. In this specification, the locations of bird nests are considered to be the locations of equipment that is the target of abnormality detection.

[0088] As described above, the control unit 51 detects the position of the utility pole on each of the acquired time-series images, and based on the detected position of the utility pole and the relative position between the utility pole and the equipment, can detect the position of the equipment on each of the acquired time-series images.

[0089] The method illustrated in Figures 17 and 18 is not limited to bird nests as an abnormality detection target, but can also be applied to vines. Note that, in the case of vines, since they exist along the ground support line, the detection target can be the tip of the vine, and the position of the vine is assumed to be the position of the equipment for which abnormality is detected.

[0090] FIG. 19 is a diagram showing an example of an abnormality detection target. As shown in FIG. 19, the abnormality detection targets include bird nests, vines, guide boards, signs, and communication facilities. The reference facility during object tracking may be a utility pole. Specifically, the reference facility may be the center of the utility pole in the height direction, but the reference position is not limited to this. When the abnormality detection target is communication facilities, the reference position may be the cross arm 21, as exemplified in FIG. 8.

[0091] When the target of anomaly detection is bird nesting, the equipment anomaly is an anomaly in an electric wire (including, for example, a low-voltage line, a high-voltage line, etc.), and specifically includes the possibility of a ground fault or a short circuit in the electric wire.

[0092] When the abnormality detection target is vines, the abnormality in the facility is an abnormality in an electric wire (including, for example, a low-voltage line, a high-voltage line, etc.), and specifically includes the possibility of a ground fault in the electric wire.

[0093] When the target of abnormality detection is a guide board, the abnormality in the equipment is an abnormality in the guide board, and specifically includes damage, corrosion (rust, holes, missing parts), bending, detachment, and the like of the guide board.

[0094] When the target of anomaly detection is a sign, the equipment anomaly is an anomaly in the sign, and specifically includes damage, corrosion (rust, holes, missing parts), bending, detachment, and the like of the sign.

[0095] When the target of abnormality detection is communication equipment, the abnormality in the equipment is an abnormality in the communication equipment, specifically including a fastener coming loose, a broken suspension line or hanger, a sagging communication line, damage to a connection box, a lid coming loose, etc.

[0096] The detection of bird nests and vines and object tracking can be performed by the methods exemplified in Figs. 17 and 18. The detection of guide boards and signs and object tracking can also be performed by the methods exemplified in Figs. 17 and 18. The detection of communication equipment and object tracking may be performed by the methods exemplified in Figs. 17 and 18, or the methods exemplified in Figs. 7 and 8. In the case of communication equipment, the object to be detected may be, for example, a fastener provided on the cross arm 21, a suspension line, a hanger, a connection box, etc. Furthermore, the detection of an abnormality in communication equipment can be performed based on the number of OKs and the number of NGs of an object, as exemplified in Fig. 7. Moreover, the display examples of the detection results shown in Figs. 9 and 10 can be similarly applied to bird nests, vines, guide boards, signs, and communication equipment.

[0097] FIG. 20 is a diagram showing another example of the procedure for anomaly detection by anomaly detection device 50. The control unit 51 acquires a time series of images of equipment mounted on utility pole 20 (S21), and detects utility pole 20 based on the acquired images (S22). The position of utility pole 20 to be detected may be the center part in the height direction of utility pole 20. The control unit 51 detects the detection object based on the relative position of the detected utility pole 20 position and the detection object (S23). The detection object includes, for example, at least one of bird nests, vines, guide boards, signs, and communication equipment. The control unit 51 detects the presence or absence of an abnormality in the equipment (S24).

[0098] The control unit 51 determines whether or not the processing of all the time-series images of the utility pole 20 has been completed (S25), and if the processing has not been completed (NO in S25), the control unit 51 continues the processing from step S22 onwards for the image of the next frame. If the processing of all the time-series images has been completed (YES in S25), the control unit 51 acquires position information corresponding to the group of time-series images of the electrical equipment 25 of the utility pole 20 (S26).

[0099] The control unit 51 calculates the number of equipment abnormalities for each time-series image (S27) and detects the presence or absence of an abnormality in the equipment of the utility pole 20 (S28). If the total number of equipment abnormalities in all the time-series images is equal to or greater than a predetermined threshold, it can be determined that the equipment is abnormal. The control unit 51 outputs the detection result (S29) and ends the process.

[0100] (Supplementary Note 1) The anomaly detection method involves acquiring a time series of images of equipment, including electrical equipment mounted on utility poles along roads, and detecting anomalies in the equipment based on the acquired time series of images.

[0101] (Supplementary Note 2) In the anomaly detection method in Supplementary Note 1, when a time-series image of the equipment is input, the acquired time-series images are input into a learning model which outputs the presence or absence of an anomaly in the equipment for each image in the time series, and an anomaly in the equipment is detected.

[0102] (Supplementary Note 3) In the anomaly detection method in Supplementary Note 2, the electrical equipment includes a first electrical equipment and a second electrical equipment larger in shape than the first electrical equipment, and when time-series images of the electrical equipment are input, the learning model detects the first electrical equipment for each of the time-series images and outputs the presence or absence of an abnormality in the first electrical equipment, and detects an abnormality in the first electrical equipment by inputting the acquired time-series images into the learning model.

[0103] (Supplementary Note 4) The anomaly detection method in Supplementary Note 3 detects the position of the second electrical equipment on each of the acquired time-series images, and detects the position of the first electrical equipment on each of the acquired time-series images based on the detected position of the second electrical equipment and the relative position of the second electrical equipment and the first electrical equipment.

[0104] (Appendix 5) In the anomaly detection method according to appendix 1 or appendix 2, the position of the utility pole is detected on each of the acquired time-series images, and the position of the equipment is detected on each of the acquired time-series images based on the detected position of the utility pole and the relative position of the utility pole and the equipment.

[0105] (Supplementary Note 6) The anomaly detection method according to any one of Supplementary Note 1 to Supplementary Note 5 calculates the probability of an anomaly in a piece of equipment based on the number of anomalies detected in each of a time-series of images taken over a predetermined period of time or a predetermined number of images.

[0106] (Supplementary Note 7) In the abnormality detection method according to Supplementary Note 3 or Supplementary Note 4, the first electrical equipment includes an insulator, the second electrical equipment includes an arm to which the insulator is fixed, and the abnormality of the first electrical equipment includes an abnormality in a bind wire for fixing a low-voltage line or an electric lighting line to the insulator.

[0107] (Appendix 8) The abnormality detection method according to any one of appendix 3, appendix 6, or appendix 7, wherein the first electrical equipment includes a discharge clamp insulator, the second electrical equipment includes a cross arm to which the discharge clamp insulator is fixed, and the abnormality of the first electrical equipment includes an abnormality in a discharge clamp.

[0108] (Appendix 9) An abnormality detection method according to any one of appendixes 3, 6 to 8, wherein the first electrical equipment includes a bushing or a gasket, the second electrical equipment includes a pole-mounted transformer to which the bushing or gasket is attached, and the abnormality of the first electrical equipment includes an abnormality in the bushing or gasket.

[0109] (Supplementary Note 10) The abnormality detection method according to any one of Supplementary Note 1, Supplementary Note 2, or Supplementary Note 5, wherein the equipment includes at least one of an electric wire, a sign, a guide board, and a communication equipment.

[0110] (Supplementary Note 11) The abnormality detection method is any one of Supplementary Note 1 to Supplementary Note 10, which outputs an image of equipment in which an abnormality is detected.

[0111] (Appendix 12) The anomaly detection method is any one of Appendices 1 to 11, in which the position of a utility pole on which equipment in which an anomaly has been detected is mounted is displayed on a map.

[0112] (Appendix 13) The anomaly detection method is in any one of Appendices 1 to 12, in which a time-series image of equipment mounted on a utility pole is acquired using an imaging device provided on a mobile object moving along a road.

[0113] (Addendum 14) The abnormality detection device includes a control unit, which acquires a time series of images of equipment, including electrical equipment mounted on utility poles installed along roads, and detects abnormalities in the equipment based on the acquired time series of images.

[0114] (Appendix 15) The computer program acquires a time series of images of equipment, including electrical equipment mounted on utility poles along roads, and detects abnormalities in the equipment based on the acquired time series of images.

[0115] The matters described in each embodiment can be combined with each other. In addition, the independent claims and dependent claims described in the claims can be combined with each other in any and all combinations regardless of the citation format. Furthermore, the claims use a format in which a claim cites two or more other claims (multiple claim format), but this is not limited to this. A multiple claim (multi-multi claim) that cites at least one multiple claim may also be used. [Explanation of symbols]

[0116] 1 vehicle 10. Imaging Device 11 Control section 12 Camera section 13 GPS section 14 Communications Department 15 Storage section 20 Electric pole 21 Arms 22 Insulator 23 Binding Wire 25 Electrical Equipment 30 Low voltage line 50 Anomaly detection device 51 Control section 52 Communications Department 53 Memory 54 Display section 55 Operation section 56 Memory section 57 Computer Programs 58 First Learning Model 59 Second Learning Model 60 Utility pole information DB

Claims

1. A time series of images is acquired of facilities including electrical equipment mounted on utility poles along a road, Detecting an abnormality in the facility based on the acquired time-series images. Anomaly detection methods.

2. When time-series images of the equipment are input, a learning model outputs the presence or absence of an abnormality in the equipment for each time-series image, and detects an abnormality in the equipment by inputting the acquired time-series images. The anomaly detection method according to claim 1 .

3. The electrical equipment includes: A first electrical device and a second electrical device having a larger size than the first electrical device, When time-series images of the electrical equipment are input, the learning model detects the first electrical equipment for each time-series image and outputs the presence or absence of an abnormality in the first electrical equipment, and detects an abnormality in the first electrical equipment by inputting the acquired time-series images into the learning model. The anomaly detection method according to claim 2 .

4. Detecting a position of the second electrical equipment on each of the acquired time-series images; detecting a position of the first electrical equipment on each of the acquired time-series images based on the detected position of the second electrical equipment and a relative position between the second electrical equipment and the first electrical equipment; The anomaly detection method according to claim 3 .

5. Detecting the position of the utility pole on each of the acquired time-series images; Detecting the position of the equipment on each of the acquired time-series images based on the detected position of the utility pole and the relative position between the utility pole and the equipment. The anomaly detection method according to claim 2 .

6. Calculating the probability of an abnormality in a piece of equipment based on the number of abnormalities detected in each time-series image taken over a predetermined period of time or a predetermined number of images taken; The method for detecting an anomaly according to any one of claims 1 to 5.

7. the first electrical equipment includes an insulator; the second electrical equipment includes a cross arm to which the insulator is fixed, The abnormality of the first electrical equipment includes an abnormality of a binding wire for fixing a low-voltage line or an electric light line to the insulator. The abnormality detection method according to claim 3 or 4.

8. the first electrical device includes a discharge clamp insulator; the second electrical equipment includes a cross arm to which the discharge clamp insulator is fixed, The abnormality of the first electrical equipment includes an abnormality of a discharge clamp. The abnormality detection method according to claim 3 or 4.

9. the first electrical device includes a bushing or a gasket; The second electrical equipment includes a pole-mounted transformer to which the bushing or gasket is attached. Including, The abnormality of the first electrical equipment includes an abnormality of the bushing or gasket. The abnormality detection method according to claim 3 or 4.

10. The facilities include at least one of electric wires, signs, guide boards, and communication facilities; The method for detecting an anomaly according to claim 1 , 2 or 5 .

11. Outputting images of equipment where an abnormality was detected, The method for detecting an anomaly according to any one of claims 1 to 5.

12. Display on a map the location of the utility pole on which the equipment where the abnormality was detected is installed. The method for detecting an anomaly according to any one of claims 1 to 5.

13. A time-series image of the equipment mounted on the utility pole is acquired using an image capturing device mounted on a mobile object moving along a road. The method for detecting an anomaly according to any one of claims 1 to 5.

14. A control unit is provided, The control unit is A time series of images is acquired of facilities including electrical equipment mounted on utility poles along a road, Detecting an abnormality in the facility based on the acquired time-series images. Anomaly detection device.

15. A time series of images is acquired of facilities including electrical equipment mounted on utility poles along a road, Detecting an abnormality in the facility based on the acquired time-series images. A computer program that causes a computer to carry out processing.

Images