Abnormal location detection device, abnormal location detection method, and abnormal location detection program

Abstract

This abnormal location detection device comprises: an abnormal location detection unit that detects abnormal location candidates, which are candidates for an abnormal location, from image features contained in electric wire area images (S102); an abnormality periodicity determination unit that determines periodicity of the abnormal location candidates (S103), and outputs abnormal location candidates without periodicity (S104: No) as abnormal locations (S110); an abnormal image comparison unit that compares images of abnormal location candidates with periodicity (S104: Yes) to calculate degrees of similarity among the abnormal location candidates (S106); and a normal location removal unit that excludes abnormal location candidates with a high degree of similarity calculated by the abnormal image comparison unit (S107: Yes) as normal locations (S108), and outputs abnormal location candidates excluding the normal locations as abnormal locations (S110).

Description

Abnormality detection device, abnormality detection method, and abnormality detection program

[0001] The present disclosure relates to an abnormality detection device, an abnormality detection method, and an abnormality detection program.

[0002] There is a technology that detects abnormalities such as damage to electric wires by analyzing images of electric wires such as power transmission lines. However, electric wires may have components such as snow-resistant rings attached at equal intervals to prevent snow from accumulating on the wires. In such cases, there is a risk that the components such as snow-resistant rings may also be detected as abnormalities.

[0003] Patent Document 1 discloses a technology in which an electric wire area is acquired from image data photographed from a helicopter, a representative value of each pixel in the horizontal direction is calculated, and an optical fiber crossing portion is identified from a change in the representative value, and an anomaly detection method is changed depending on whether or not an optical fiber crosses. In Patent Document 1, when a specified number or more of anomalies are detected from the image data, the periodicity of the anomalies is determined, and if periodicity is detected, the anomaly detection threshold is lowered so that the anomaly is not detected.

[0004] Japanese Patent Application Laid-Open No. 2007-310828

[0005] However, in the technology described in Patent Document 1, when the threshold is lowered due to the periodicity of multiple abnormal locations, there is a risk that minor abnormalities will be eliminated in addition to normal locations. Also, in Patent Document 1, when the threshold for detecting an abnormality is high, multiple structures that exist at equal intervals on the electric wire, such as snow-resistant rings, will be detected as abnormal, which causes a problem that checking the detection results and subsequent processing will become cumbersome.

[0006] The present disclosure aims to provide an abnormal part detection device, an abnormal part detection method, and an abnormal part detection program that can clearly distinguish between an abnormal part and a normal part even when multiple abnormal parts have periodicity.

[0007] The abnormal part detection device disclosed herein is characterized by comprising an abnormal part detection unit that detects candidate abnormal parts that are candidates for abnormal parts from image features included in photographed images of an elongated object to be inspected; an abnormal periodicity determination unit that determines the periodicity of the candidate abnormal parts and outputs the candidate abnormal parts that do not have periodicity as abnormal parts; an abnormal image comparison unit that compares images of the candidate abnormal parts that have periodicity and calculates the similarity of each candidate abnormal part; and a normal part removal unit that excludes candidate abnormal parts that have a high degree of similarity calculated by the abnormal image comparison unit as normal parts and outputs the candidate abnormal parts from which the normal parts have been excluded as the abnormal parts.

[0008] The abnormal part detection method disclosed herein is a computer-executed method that includes the steps of: detecting candidate abnormal parts that are candidates for abnormal parts from image features included in a photographed image of an elongated object to be inspected; determining the periodicity of the candidate abnormal parts and outputting the candidate abnormal parts that do not have periodicity as abnormal parts; comparing the images of the candidate abnormal parts that have periodicity to calculate the similarity of each candidate abnormal part; and excluding candidate abnormal parts with a high calculated similarity as normal parts and outputting the candidate abnormal parts excluding the normal parts as the abnormal parts.

[0009] The abnormal part detection program disclosed herein causes a computer to execute the following steps: detecting candidate abnormal parts from image features contained in photographed images of an elongated object to be inspected; determining the periodicity of the candidate abnormal parts and outputting the candidate abnormal parts that do not have periodicity as abnormal parts; comparing the images of the candidate abnormal parts that have periodicity to calculate the similarity of each candidate abnormal part; and excluding candidate abnormal parts with a high calculated similarity as normal parts and outputting the candidate abnormal parts excluding the normal parts as the abnormal parts.

[0010] According to the present disclosure, it is possible to provide an abnormal part detection device, an abnormal part detection method, and an abnormal part detection program that can clearly distinguish between an abnormal part and a normal part even when multiple abnormal parts have periodicity.

[0011] FIG. 1 is a functional configuration diagram showing an abnormal point detection device according to a first embodiment. FIG. 2 is a schematic diagram showing an example of an electric wire that is an inspection target. FIG. 3 is a graph in an electric wire image of an electric wire area, in which the horizontal axis represents the length of the electric wire and the vertical axis represents the average brightness value of each column of the electric wire area. FIG. 4 is a hardware configuration diagram showing an abnormal point detection device according to the first embodiment. FIG. 5 is a flowchart showing an example of a determination process of the abnormal point detection device according to the first embodiment. FIG. 6 is a functional configuration diagram showing an abnormal point detection device according to a second embodiment. FIG. 7 is a flowchart showing an example of a determination process of the abnormal point detection device according to the second embodiment. FIG. 8 is a functional configuration diagram showing an abnormal point detection device according to a third embodiment. FIG. 9 is a flowchart showing an example of a determination process of the abnormal point detection device according to the third embodiment. FIG. 10 is a functional configuration diagram showing an abnormal point detection device according to a fourth embodiment. FIG. 11 is a flowchart showing an example of a determination process of the abnormal point detection device according to the fourth embodiment.

[0012] An abnormality detection device according to an embodiment will be described below with reference to the drawings. The following embodiments are merely examples, and the embodiments can be appropriately combined and modified.

[0013] 1 is a functional configuration diagram showing an abnormal point detection device 100 according to embodiment 1. The abnormal point detection device 100 includes an abnormal point detection unit 10 that detects abnormal point candidates that are candidates for abnormal points from image features of an input electric wire image 50 in an electric wire region, an abnormal periodicity determination unit 12 that determines the periodicity of the abnormal point candidates when there are multiple detected abnormal point candidates, an abnormal image comparison unit 14 that compares images of the abnormal point candidates and determines the similarity between the abnormal point candidates when the abnormal point candidates are periodic, and a normal point removal unit 16 that removes abnormal point candidates that have a high degree of similarity from the abnormal points detected by the abnormal point detection unit 10 as normal points and outputs the result as an abnormal point 52.

[0014] The electric wire area electric wire image (hereinafter abbreviated as "electric wire area image") 50 is an image of an electric wire area such as a power transmission line 400, which is an inspection object having a long shape, as shown in Figure 2, and is obtained by aerial photography using an aircraft such as a helicopter or an unmanned aircraft such as a drone.

[0015] The abnormal part detection unit 10 is constructed, for example, by training a mathematical model such as a convolutional neural network (CNN) through machine learning using as training data an image of an electric wire region including the abnormal parts 52, 420. After training, the abnormal part detection unit 10 detects candidate abnormal parts from image features extracted from the electric wire region image 50 by image comparison, brightness value comparison, or the like.

[0016] The abnormality periodicity determination unit 12 determines whether or not there is periodicity in the abnormal part candidates detected by the abnormal part detection unit 10. For example, there is a tendency for abnormal part candidates to be detected consecutively in the snow-resistant rings 410 and finned electric wires. The abnormality periodicity determination unit 12 determines the periodicity of the abnormal part candidates based on, for example, any of (1) the magnitude of the variance of the distances between the abnormal part candidates, (2) the degree of deviation of each value of the distances between the abnormal part candidates from the median value of the distances between the abnormal part candidates, and (3) the brightness value of the pixels in the electric wire region.

[0017] The determination based on the variance of the distance between the abnormality candidates in (1) is specifically based on the distance d between the adjacent abnormality candidates. i (i=1, 2, 3, ..., n: n is the total number of samples of the distance between adjacent abnormality location candidates) is obtained, and the distance d i Variance of s 2 The abnormality periodicity determining unit 12 determines whether there is a variation in the distance between the candidate abnormal parts from the value of s. The distance between the candidate abnormal parts is measured based on the number of pixels between the abnormal parts in the electric wire area image 50. 2 When the value of is equal to or less than a predetermined variance threshold, it is determined that the candidate abnormal part detected by the abnormal part detection unit 10 has periodicity. The variance threshold is, for example, a variance s calculated from each of the electric wire region image of the electric wire to which the member such as the snow accretion resistant ring 410 is attached and the electric wire region image of the electric wire having an abnormal part such as damage in addition to the member such as the snow accretion resistant ring 410. 2Alternatively, if the abnormal periodicity determination unit 12 is constructed by machine learning using, as training data, an image of a wire region of a wire to which a component such as the snow accretion resistant ring 410 is attached and an image of a wire region of a wire that has an abnormal portion such as damage in addition to a component such as the snow accretion resistant ring 410, the abnormal periodicity determination unit 12 may hold a variance threshold as a result of the machine learning.

[0018]

[0019] (2) When making a determination based on the degree of deviation of each value of the distance between the candidate abnormalities from the median of the distances between the candidate abnormalities, the abnormality periodicity determination unit 12, similar to the above (1), measures the distance between the candidate abnormalities based on the number of pixels between the candidate abnormalities in the electric wire region image 50 and extracts the median of the distances between adjacent candidate abnormalities. The abnormality periodicity determination unit 12 then calculates the number of samples of the distance between the candidate abnormalities that exist within a certain range from the median, and determines that the candidate abnormalities detected by the abnormality portion detection unit 10 are periodic if the ratio of this number to the total number of samples n is equal to or greater than a predetermined ratio threshold. The certain range from the median and the ratio threshold are, for example, the distance d between the candidate abnormalities calculated from each of the electric wire region image of the electric wire to which a component such as the snow-resistant ring 410 is attached and the electric wire region image of the electric wire that has an abnormality such as damage in addition to the component such as the snow-resistant ring 410. i Alternatively, if the abnormal periodicity determination unit 12 is constructed by machine learning using, as training data, an image of a wire region of a wire to which a component such as the snow accretion resistant ring 410 is attached and an image of a wire region of a wire that has an abnormal portion such as damage in addition to a component such as the snow accretion resistant ring 410, the abnormal periodicity determination unit 12 may hold a certain range and a ratio threshold as a result of the machine learning.

[0020] (3) The determination based on the luminance values ​​of the pixels in the electric wire region involves calculating the average luminance value of each column in the electric wire region, and determining whether the candidate abnormalities exhibit periodicity from a graph, such as that shown in FIG. 3, with the electric wire length (longitudinal distance) on the horizontal axis and the average luminance value of each column in the electric wire region on the vertical axis. Each column in the electric wire region refers to each column of pixels in the electric wire region image 50 that intersects perpendicularly with the row of pixels representing the electric wire. The abnormal periodicity determination unit 12 determines that the candidate abnormalities detected by the abnormality portion detection unit 10 exhibit periodicity when the graph shows locations with similar luminance values ​​at approximately equal intervals. For example, since the snow-resistant ring 410 is often made of black resin, the luminance values ​​exhibit minimum values ​​at regular intervals in FIG. 3.

[0021] If the candidate abnormal part detected by the abnormal part detection part 10 has periodicity, the abnormal periodicity determination part 12 transfers the processing to the subsequent abnormal image comparison part 14. On the other hand, if the candidate abnormal part detected by the abnormal part detection part 10 does not have periodicity, the abnormal periodicity determination part 12 outputs the candidate abnormal part as an abnormal part 52.

[0022] When the candidate abnormality regions are periodic, the abnormal image comparison unit 14 compares the images of the candidate abnormality regions and calculates the similarity of the images. The similarity of the images is, for example, the sum of squares of the deviations of the brightness values ​​of pixels located at corresponding positions in the images of the two candidate abnormality regions being compared. The closer this sum of squares is to 0, the higher the similarity between the two images. The abnormal image comparison unit 14 determines that the similarity between the two images is at least a certain level if the calculated sum of squares is equal to or less than a predetermined similarity threshold. For example, the similarity threshold is determined by comparing and considering the value of the sum of squares of the deviations of the brightness values ​​of pixels located at corresponding positions in the images of the two candidate abnormality regions to be compared, extracted from an image of a wire region of an electric wire to which a component such as the snow-resistant ring 410 is attached and an image of a wire region of an electric wire having an abnormality such as damage in addition to the component such as the snow-resistant ring 410. Alternatively, if the abnormal periodicity determination unit 12 is constructed by machine learning using, as training data, an image of the wire area of ​​an electric wire to which a component such as the snow-resistant ring 410 is attached and an image of the wire area of ​​an electric wire that has an abnormal part such as damage in addition to a component such as the snow-resistant ring 410, the abnormal periodicity determination unit 12 may be configured to retain a similarity threshold as a result of the machine learning.

[0023] If the similarity of the image of the abnormal part candidate is equal to or greater than a certain level, the normal part removal unit 16 determines that the abnormal part candidate is a periodically occurring snow-resistant ring 410 or an electric wire with a special shape, such as a finned electric wire. The normal part removal unit 16 regards the abnormal part candidate with a similarity equal to or greater than a certain level as a normal part and removes it from the abnormality detection result, and outputs the removed abnormal part candidate as an abnormal part 52.

[0024] 4 is a hardware configuration diagram showing the abnormal point detection device 100 according to embodiment 1. The abnormal point detection device 100 includes a CPU (Central Processing Unit) 210, a main memory 220, an input / output interface (I / O interface) 230, and a storage unit 240, which are connected to each other via a system bus 250. The abnormal point detection device 100 may be composed of multiple computers.

[0025] The CPU 210 is an integrated circuit (IC) that performs arithmetic processing. Apart from the CPU 210, a computing element such as a digital signal processor (DSP) or a graphics processing unit (GPU) may be used. By executing an abnormality detection program, the CPU 210 operates as an abnormality location detection function that detects abnormality location candidates from the electric wire area image 50, an abnormal periodicity determination function that determines whether the abnormality location candidates detected by the abnormality location detection function have periodicity, an abnormality image comparison function that compares the images of the abnormality location candidates and calculates the similarity of the images if the abnormality location candidates have periodicity, and a normal location elimination function that considers abnormality location candidates with a similarity equal to or greater than a certain level as normal locations and excludes them from the abnormality detection result, and outputs the excluded abnormal location candidates as abnormal locations 52. As a result, by executing the abnormal part detection program, the CPU 210 functions as the abnormal part detection unit 10, the abnormal periodicity determination unit 12, the abnormal image comparison unit 14, and the normal part removal unit 16. The abnormal part detection program is provided, for example, as a recording medium on which it is recorded.

[0026] The main memory 220 is configured by a volatile storage device such as a RAM (Random Access Memory) or a non-volatile storage device such as a ROM (Read Only Memory). The storage unit 240 is configured by a non-volatile storage device such as a HDD (Hard Disk Drive) or a flash memory.

[0027] The I / O interface 230 is a port to which the camera 300 serving as an input device and the display device 310 serving as an output device are connected. Specific examples of the I / O interface 230 include a USB (Universal Serial Bus) terminal, an IEEE 1394 terminal, a Thunderbolt terminal, or the like, and further includes a communication interface such as Ethernet (registered trademark). The camera 300 is an imaging device that acquires an electric wire area image 50 by aerial photography or the like, and inputs the acquired electric wire area image 50 to the abnormal point detection device 100 by connecting to the I / O interface 230. In the embodiment, it is assumed that the camera 300 is connected to the I / O interface 230 after aerial photography is performed by an aircraft such as a helicopter. However, if wireless communication is possible between the I / O interface 230 and the camera 300, the electric wire area image 50 acquired by the camera 300 may be input to the I / O interface 230 in real time via wireless communication. The display device 310 is a display or the like, but may also include a printer or the like that outputs information.

[0028] 5 is a flowchart showing an example of the determination process of the abnormal point detection device 100 according to Embodiment 1. In step S101, the electric wire area image 50 captured by the camera 300 is acquired via the I / O interface 230.

[0029] In step S102, the abnormal part detection unit 10 detects abnormal part candidates by image comparison or brightness value comparison, and creates a list of the detected abnormal part candidates.

[0030] In step S103, the abnormality periodicity determination unit 12 calculates the period of the candidate abnormal part. In the first embodiment, as described above, the periodicity of the candidate abnormal part is calculated based on any one of (1) the variance of the distance between the candidate abnormal part, (2) the distance between the candidate abnormal part with respect to the median value of the distance between the candidate abnormal part, or (3) the luminance value of the pixel in the electric wire region.

[0031] In step S104, the abnormality periodicity determination unit 12 determines whether the abnormality location candidate has periodicity. If the abnormality location candidate has periodicity in step S104, the procedure proceeds to step S105, and if the abnormality location candidate does not have periodicity, the abnormality location candidate detected by the abnormality location detection unit 10 is considered to be the abnormality location 52, and the procedure proceeds to step S110.

[0032] In step S105, loop 1 is started, in which the procedures from step S106 to step S108 are repeated for each candidate abnormal region. In step S106, the abnormal image comparison unit 14 calculates the similarity between the images of adjacent candidate abnormal regions. As described above, in the first embodiment, the similarity between the images is the sum of squares of the deviations of the luminance values ​​of pixels located at corresponding positions in each of the images of the two candidate abnormal regions being compared. The closer this sum of squares is to 0, the higher the similarity between the two images.

[0033] In step S107, the abnormal image comparison unit 14 determines whether the similarity is equal to or greater than a predetermined standard. If the sum of squares of the deviations of the luminance values ​​of the pixels of the two compared abnormal area candidates is equal to or less than a predetermined similarity threshold, the abnormal image comparison unit 14 determines that the similarity between both images is equal to or greater than the standard. If the similarity is equal to or greater than the standard in step S107, the procedure proceeds to step S108. If the similarity is not equal to or greater than the standard, the procedure proceeds to step S109. In the first embodiment, an abnormal area candidate whose similarity is not equal to or greater than the standard is likely to be an abnormal area 52. However, in the first embodiment, the similarity between adjacent abnormal area candidates is determined as described above, and therefore the two adjacent abnormal area candidates cannot be determined to be abnormal areas 52 solely based on their similarity. This is because even if two abnormal area candidates do not have a mutual similarity equal to or greater than the standard, there may be cases in which the similarity between each of the two adjacent abnormal area candidates is equal to or greater than the standard. In the first embodiment, normal parts are excluded from the detection results, so that after the processing of loop 1 is performed for all the candidate abnormal parts, the candidate abnormal parts that have not been excluded as normal parts become the abnormal parts 52.

[0034] In step S108, the normal part removal unit 16 regards the abnormal part candidates whose similarity is equal to or greater than the standard as normal parts, and excludes them from the abnormal part candidates determined to have periodicity by the abnormal periodicity determination unit 12. In the first embodiment, the similarity between adjacent abnormal part candidates is determined as described above, and therefore, when the similarity is equal to or greater than the standard, each of the two adjacent abnormal part candidates related to the calculation of the similarity is regarded as a normal part.

[0035] In step S109, the normal part removal unit 16 determines whether or not the similarity has been determined for all of the abnormal part candidates determined to have periodicity. If the similarity has been determined for all of the abnormal part candidates determined to have periodicity, the normal part removal unit 16 ends the processing of loop 1 and proceeds to step S110. If the similarity has not been determined for all of the abnormal part candidates determined to have periodicity, the normal part removal unit 16 executes the processing of loop 1 from step S106 to step S108.

[0036] In step S110, the abnormal part candidates determined not to have periodicity in step S104, together with the abnormal part candidates obtained by excluding the abnormal part candidates that the normal part removal unit 16 has deemed to be normal parts from the abnormal part candidates determined to have periodicity, are output to the display device 310 as abnormal parts 52, and the processing ends.

[0037] As described above, in the first embodiment, the abnormal periodicity determination unit 12 determines whether the candidate abnormal portions are periodic. If the candidate abnormal portions are periodic, the abnormal image comparison unit 14 compares the images of the candidate abnormal portions with each other to determine the degree of similarity. If the candidate abnormal portions have a high degree of similarity, the normal portion removal unit 16 assumes that the candidate abnormal portions are snow-resistant rings 410 uniformly distributed on the electric wire or portions of an electric wire with a special shape, such as a finned electric wire, and removes the corresponding portions from the abnormality detection results. As a result, by removing normal portions such as snow-resistant rings 410 from the candidate abnormal portions, it is possible to clearly distinguish between the abnormal portions 52 and normal portions even when multiple abnormal portions 52 are periodic.

[0038] According to the first embodiment, it is possible to remove non-anomalous parts from the candidate anomaly parts that are detected in a state where minor anomaly parts can be detected, and therefore it is possible to accurately detect the abnormal part excluding normal parts while maintaining high detection sensitivity for the candidate anomaly parts. Furthermore, by removing normal parts that are not abnormal from the candidate anomaly parts, it is possible to shorten the time for checking the results and the time for subsequent processing.

[0039] Second Embodiment Next, an abnormal point detection device 110 according to the second embodiment will be described. The abnormal point detection device 110 according to the second embodiment shown in FIG. 6 differs from the first embodiment in that it includes a line section information database (DB) 60 in which line sections (sections) using specially shaped electric wires, such as snow-resistant rings 410 or finned electric wires, are registered in advance, and a processing necessity determination unit 22 that refers to the line section information DB 60 to determine whether or not to execute the processing for removing normal points according to the first embodiment. However, since the other configurations are the same as those of the first embodiment, the same components as those of the first embodiment are denoted by the same reference numerals as those of the first embodiment, and detailed description thereof will be omitted. Furthermore, since the hardware configuration of the second embodiment is the same as that of the first embodiment, detailed description thereof will be omitted. However, in the second embodiment, the CPU 210 functions as an abnormal point detection function, an abnormal period determination function, an abnormal image comparison function, and a normal point removal function, and also functions as a processing necessity determination function that refers to the line section information DB 60 to determine whether or not to execute the processing for removing normal points according to the first embodiment. As a result, the CPU 210 functions as an abnormal part detection unit 10, an abnormal periodicity determination unit 12, an abnormal image comparison unit 14, a normal part removal unit 16, and a processing necessity determination unit 22 by executing the abnormal part detection program.

[0040] 7 is a flowchart showing an example of the determination processing of the abnormality point detection device 110 according to the second embodiment. The flowchart shown in FIG. 7 differs from the first embodiment in that it includes step S201 of acquiring, together with the electric wire area image 50, line section information, which is information about a line section using electric wires with special shapes such as snow-resistant rings 410 or finned electric wires, from the line section information DB 60, and step S202 of determining, by referring to the line section information, whether or not the line section in which the abnormality point candidate detected by the abnormality point detection unit 10 is located corresponds to a line section using electric wires with special shapes. However, since the other steps are the same as those in the first embodiment, the same steps as those in the first embodiment are denoted by the same reference numerals and detailed description thereof will be omitted.

[0041] In step 201, the abnormality part detection unit 10 acquires the electric wire area image 50, and the processing necessity determination unit 22 acquires the line section information by referring to the line section information DB 60. As in the first embodiment, in step S102, the abnormality part detection unit 10 detects abnormality part candidates from the electric wire area image 50 by image comparison or brightness value comparison, and creates a list of the detected abnormality part candidates.

[0042] In step S202, the processing necessity determination unit 22 determines whether the abnormality location candidate detected by the abnormality location detection unit 10 belongs to a line section using an electric wire with a special shape. If the detected abnormality location candidate belongs to a line section using an electric wire with a special shape in step S202, the procedure proceeds to step S103, and if the detected abnormality location candidate does not belong to a line section using an electric wire with a special shape, the abnormality location candidate is regarded as the abnormality location 52, and the procedure proceeds to step S110.

[0043] In step S110, the abnormal points 52 detected in the processing of loop 1, which is composed of the procedures from steps S105 to S109 as in embodiment 1, and the abnormal points 52 detected in steps S104 and S202 are each output to the display device 310, and the processing is terminated.

[0044] As described above, in the second embodiment, if the candidate abnormality location detected by the abnormality location detection unit 10 is present in a location registered in the line section information DB 60, the process of removing the normal location according to the first embodiment is executed, and if the candidate abnormality location is not present in a location registered in the line section information DB 60, it is output as the abnormality location 52. As a result, the likelihood of erroneously removing the abnormality location 52 as a normal location is reduced, and the abnormality location 52 in the electric wire can be accurately detected.

[0045] Third Embodiment Next, an abnormal point detection device 120 according to a third embodiment will be described. The abnormal point detection device 120 according to the third embodiment shown in FIG. 8 differs from the first embodiment in that the normal point removal unit 36 ​​separates and outputs abnormal points 52 and decision-required points 54 that should ultimately be decided by personnel based on the similarity between the candidate abnormal points. However, since the other configurations are the same as those of the first embodiment, the same components as those of the first embodiment are denoted by the same reference numerals as those of the first embodiment, and detailed description thereof will be omitted. Furthermore, since the hardware configuration of the third embodiment is the same as that of the first embodiment, detailed description thereof will be omitted.

[0046] 9 is a flowchart showing an example of the determination processing of the abnormal point detection device 120 according to the third embodiment. In the flowchart shown in FIG. 9 , in the processing of loop 1 from step S305 to step S311, an abnormal point candidate whose similarity between adjacent abnormal point candidates is equal to or greater than a first standard is regarded as a normal point, and further, when an abnormal point candidate whose similarity is not equal to or greater than the first standard is equal to or greater than a second standard that is lower than the first standard, the abnormal point candidate is classified as a determination-required point 54 whose similarity is to be determined by the user. However, since the other steps are the same as those of the first embodiment, the same reference numerals are used for steps similar to those of the first embodiment, and detailed description thereof will be omitted.

[0047] In step S305, loop 1 is started, in which the procedures from step S306 to step S310 are repeated for each candidate abnormal region. In step S306, the abnormal image comparison unit 14 calculates the similarity between the images of adjacent candidate abnormal regions. As in the first embodiment, the image similarity is the sum of squares of the deviations of the luminance values ​​of pixels existing at corresponding positions in each image of the two candidate abnormal regions being compared. The closer this sum of squares is to 0, the higher the similarity between the two images.

[0048] In step S307, the abnormal image comparison unit 14 determines whether the candidate abnormal areas determined to have periodicity in step S104 exhibit a high degree of similarity, specifically, whether the similarity is equal to or greater than a first criterion. If the sum of squares of the deviations of the pixel brightness values ​​of the two candidate abnormal areas being compared is equal to or less than a predetermined first similarity threshold, the abnormal image comparison unit 14 determines that the similarity between both images is equal to or greater than the first criterion. If the similarity is equal to or greater than the first criterion in step S307, the procedure proceeds to step S308, and if the similarity is not equal to or greater than the first criterion, the procedure proceeds to step S309.

[0049] In step S308, the normal part removal unit 36 ​​regards the abnormal part candidates whose similarity is equal to or greater than the first criterion as normal parts, and excludes them from the abnormal part candidates determined to have periodicity in step S104. In the third embodiment, the similarity between adjacent abnormal part candidates is determined as described above, and therefore, when the similarity is equal to or greater than the first criterion, each of the two adjacent abnormal part candidates related to the calculation of the similarity is regarded as a normal part.

[0050] In step S309, the abnormal image comparison unit 14 determines whether the abnormal portion candidate determined in step S307 to have a similarity less than the first standard exhibits a moderate similarity, specifically, whether the similarity is greater than or equal to a second standard lower than the first standard. The abnormal image comparison unit 14 determines that the similarity between the two images is greater than or equal to the second standard when the sum of squares of the deviations of the pixel brightness values ​​of the two compared abnormal portion candidates is less than or equal to a second similarity threshold higher than the first similarity threshold. In step S309, if the similarity is greater than or equal to the second standard, the procedure proceeds to step S310. If the similarity is less than or equal to the second standard, the procedure proceeds to step S311. In the third embodiment, an abnormal portion candidate whose similarity is less than the second standard is likely to be an abnormal portion 52. However, in the third embodiment, the similarity between adjacent abnormal portion candidates is determined as described above, and therefore, the two adjacent abnormal portion candidates cannot be determined to be abnormal portions 52 based solely on their similarity. This is because even if two candidate abnormal locations have a degree of similarity with each other that is not equal to or greater than the second standard, there may be cases in which the degree of similarity with each of the adjacent candidate abnormal locations is equal to or greater than the first standard or the second standard. In the third embodiment, normal locations and determination-required locations (described later) are excluded from the detection results, and therefore, after loop 1 processing has been performed on all candidate abnormal locations, the candidate abnormal locations that have not been excluded as normal locations or determination-required locations become abnormal locations 52.

[0051] As an example, the first similarity threshold and the second similarity threshold are each determined by comparing and considering the value of the sum of squares of the deviation of the brightness values ​​of pixels located at corresponding positions in each image of two candidate abnormal areas to be compared, which are extracted from each of an electric wire region image of an electric wire to which a component such as the snow-resistant ring 410 is attached and an electric wire region image of an electric wire that has an abnormal area such as damage in addition to a component such as the snow-resistant ring 410. Alternatively, if the abnormal periodicity determination unit 12 is constructed by machine learning using, as training data, an electric wire region image of an electric wire to which a component such as the snow-resistant ring 410 is attached and an electric wire region image of an electric wire that has an abnormal area such as damage in addition to a component such as the snow-resistant ring 410, the abnormal periodicity determination unit 12 may be configured to hold the first similarity threshold and the second similarity threshold as a result of the machine learning.

[0052] In step S310, the normal part removal unit 36 ​​regards the abnormal part candidates whose similarity is equal to or greater than the second standard as determination-required parts 54, and excludes them from the abnormal part candidates whose similarity is determined in step S307 to be less than the first standard. In the third embodiment, the similarity between adjacent abnormal part candidates is determined as described above, and therefore, when the similarity is equal to or greater than the second standard, each of the two adjacent abnormal part candidates related to the calculation of the similarity is regarded as a determination-required part.

[0053] In step S311, the normal part removal unit 36 ​​determines whether or not the similarity has been determined for all of the abnormal part candidates determined to have periodicity. If the similarity has been determined for all of the abnormal part candidates determined to have periodicity, the normal part removal unit 36 ​​ends the processing of loop 1 and proceeds to step S312. If the similarity has not been determined for all of the abnormal part candidates determined to have periodicity, the normal part removal unit 36 ​​executes the processing of loop 1 from step S306 to step S310.

[0054] In step S312, the abnormal portion 52 and the judgment-required portion 54 detected as described above are output to the display device 310, and the process ends.

[0055] As described above, in the third embodiment, as in the first or second embodiment, detected abnormal location candidates are assigned levels according to their similarity, and abnormal location candidates with a high level of similarity that are reliably considered to be normal locations are removed as normal locations, while abnormal location candidates with a low level of similarity are output as abnormal location 52. Then, abnormal location candidates that do not correspond to either normal locations or abnormal location 52 and have a medium level of similarity are presented to the user via display device 310, and the user is prompted to decide whether or not to exclude them. As a result, by incorporating the user's perspective, it becomes possible to more accurately remove abnormal location candidates that do not correspond to either normal locations or abnormal location 52.

[0056] Fourth Embodiment Next, an abnormal part detection device 130 according to the fourth embodiment will be described. The abnormal part detection device 130 according to the fourth embodiment shown in FIG. 10 differs from the first embodiment in that the normal part removal unit 46 determines abnormal part candidates having a similarity equal to or greater than a certain level as normal parts and removes them from the abnormal part candidates, and passes the result to the abnormal part detection unit 40. The abnormal part detection unit 40 then performs image comparison or brightness value comparison again on the abnormal part candidates from which the normal parts have been removed, and outputs the detected abnormal part candidates as abnormal parts 52. However, since the other configurations are the same as those of the first embodiment, the same components as those of the first embodiment are denoted by the same reference numerals as those of the first embodiment, and detailed description thereof will be omitted. Furthermore, since the hardware configuration of the fourth embodiment is the same as that of the first embodiment, detailed description thereof will be omitted.

[0057] 11 is a flowchart showing an example of the determination process of the abnormal part detection device 130 according to embodiment 2. The flowchart shown in FIG. 11 differs from embodiment 1 in that it includes step S401 in which image comparison or brightness value comparison is performed again on the candidate abnormal parts from which normal parts have been removed to detect the abnormal parts 52. However, the other steps are the same as those in embodiment 1, and therefore the same steps as those in embodiment 1 are denoted by the same reference numerals and detailed description thereof will be omitted.

[0058] In step S401, the abnormal part detection unit 40 again detects abnormal parts by image comparison, brightness value comparison, etc. for the abnormal part candidates from which normal parts have been removed by the processing of loop 1. Then, the abnormal part candidates detected as abnormal parts in step S401 are extracted as abnormal parts 52.

[0059] In step S110, the abnormality location candidate determined to have no periodicity in step S104 and the abnormality location candidate detected in step S401 are output as abnormality locations 52 to the display device 310, and the process ends.

[0060] As described above, in the fourth embodiment, the abnormal portion detection unit 40 again detects an abnormal portion as an abnormal portion from among the abnormal portion candidates from which normal portions have been excluded, and sets the abnormal portion as the abnormal portion 52. As described above, the abnormal image comparison unit 14 indicates the similarity by the sum of squares of the deviations of the luminance values ​​of pixels located at corresponding positions in each image of the two abnormal portion candidates being compared. For example, when comparing a normal portion with an abnormal portion 52, if the sum of squares of the deviations of the luminance values ​​of pixels located at corresponding positions indicates a value close to 0, the abnormal image comparison unit 14 may erroneously determine the abnormal portion 52 as a normal portion. In the fourth embodiment, abnormal portion detection is performed again for the abnormal portion candidates that may include the abnormal portion 52 erroneously determined as a normal portion, thereby detecting the abnormal portion 52 that has been erroneously determined as a normal portion. As a result, the abnormal portion 52 can be accurately detected.

[0061] <<Modifications>> In the first to fourth embodiments, the inspection target is an electric wire, but this is not limiting. For example, abnormalities may be detected in long structures such as railway overhead lines and tracks, as well as tunnels and pipelines. For railway overhead lines and tracks, abnormalities can be detected using the same procedures as those described in the first to fourth embodiments. For example, joints in tracks are evenly spaced, so they can be eliminated from the list of candidate abnormalities as normal locations with periodicity. For tunnels, candidate abnormalities, including cracks in the walls, can be detected from images of the tunnel interior. Joints that occur in each construction section during concrete pouring, for example, can be eliminated from the list of candidate abnormalities as normal locations by determining periodicity or similarity. For pipelines, candidate abnormalities, such as rust or deformation of pipes, can be detected from images of the pipeline exterior. Components such as flanges that join pipes can be eliminated from the list of candidate abnormalities as normal locations by determining periodicity or similarity.

[0062] 10 Abnormal part detection unit, 12 Abnormal periodicity determination unit, 14 Abnormal image comparison unit, 16 Normal part removal unit, 22 Processing necessity determination unit, 36 Normal part removal unit, 40 Abnormal part detection unit, 46 Normal part removal unit, 52 Abnormal part, 54 Determination required part, 60 Line section information DB, 100, 110, 120, 130 Abnormal part detection device, 210 CPU, 220 Main memory, 230 I / O interface, 240 Memory unit, 300 Camera, 310 Display device

Claims

1. an abnormality detection unit that detects candidate abnormalities from image features included in a captured image of a long inspection object; an abnormality periodicity determination unit that determines the periodicity of the abnormality location candidate; an abnormal image comparison unit that compares images of the periodic abnormal part candidates and calculates a similarity between the images of the periodic abnormal part candidates; a normal part removal unit that removes, as normal parts, the abnormal part candidates whose similarity calculated by the abnormal image comparison unit is higher than a predetermined threshold value, and outputs, as abnormal parts, the abnormal part candidates excluding the removed abnormal part candidates; An abnormality detection device comprising:

2. a line section information database in which sections where the inspection objects have special shapes are registered in advance; a processing necessity determination unit that refers to the line section information database, determines whether the abnormal location candidate belongs to the section having the special shape registered in the line section information database, and outputs the abnormal location candidate that does not belong to the section having the special shape as the abnormal location; and Furthermore, The abnormal part detection device according to claim 1 , wherein the abnormal periodicity determination unit determines the periodicity of the abnormal part candidate belonging to the section having the special shape using the processing necessity determination unit.

3. 2. The abnormal part detection device according to claim 1, wherein the normal part removal unit excludes the abnormal part candidate whose similarity is equal to or greater than a first standard as the normal part, and excludes, from the abnormal part candidate whose similarity is not equal to or greater than the first standard, the abnormal part candidate whose similarity is equal to or greater than a second standard that is smaller than the first standard as a decision-requiring part, and outputs the abnormal part candidate from which the normal parts and the decision-requiring parts have been excluded as the abnormal part together with the decision-requiring part.

4. 2. The abnormal part detection device according to claim 1, wherein the abnormal part detection unit detects abnormal part candidates again for the abnormal part candidates from which the normal parts have been excluded by the normal part removal unit, and outputs the detected abnormal part candidates as the abnormal parts.

5. 5. The abnormal part detection device according to claim 1, wherein the abnormality periodicity determination unit determines the periodicity for the plurality of abnormal part candidates based on a variance of distances between adjacent abnormal part candidates.

6. 5. The abnormal part detection device according to claim 1, wherein the abnormality periodicity determination unit determines the periodicity for the plurality of the abnormal part candidates based on a degree of deviation of each distance value between the abnormal part candidates from a median value of each distance between the adjacent abnormal part candidates.

7. 5. The abnormal part detection device according to claim 1, wherein the abnormal periodicity determination unit calculates an average value of brightness values ​​of a row of pixels of the inspection object that is perpendicular to the longitudinal direction of the inspection object in a photographed image of the inspection object, and determines that the candidate abnormal part has the periodicity if the average brightness values ​​indicate brightness values ​​that are approximately equal to each other over a distance in the longitudinal direction of the inspection object.

8. A computer-implemented method for detecting an abnormality, comprising: detecting a candidate abnormal portion from image features included in a captured image of a long inspection object; determining the periodicity of the anomaly candidate; a step of comparing images of the periodic abnormality candidate and calculating a similarity between the images of the periodic abnormality candidate; a step of excluding, as normal parts, the abnormal part candidates whose calculated similarity is higher than a predetermined threshold value, and outputting, as abnormal parts, the abnormal part candidates excluding the excluded abnormal part candidates; An abnormality detection method comprising:

9. detecting a candidate abnormal portion from image features included in a captured image of a long inspection object; determining the periodicity of the anomaly candidate; a step of comparing images of the periodic abnormality candidate and calculating a similarity between the images of the periodic abnormality candidate; a step of excluding, as normal parts, the abnormal part candidates whose calculated similarity is higher than a predetermined threshold value, and outputting, as abnormal parts, the abnormal part candidates excluding the excluded abnormal part candidates; An abnormality detection program that causes a computer to execute the above.