Anomaly detection device, anomaly detection method, and anomaly detection program

Abstract

This abnormal location detection device comprises: an abnormal location detection unit that detects abnormal location candidates, which are candidates for an abnormal location, from image features contained in electric wire area images (S102); an abnormality periodicity determination unit that determines periodicity of the abnormal location candidates (S103), and outputs abnormal location candidates without periodicity (S104: No) as abnormal locations (S110); an abnormal image comparison unit that compares images of abnormal location candidates with periodicity (S104: Yes) to calculate degrees of similarity among the abnormal location candidates (S106); and a normal location removal unit that excludes abnormal location candidates with a high degree of similarity calculated by the abnormal image comparison unit (S107: Yes) as normal locations (S108), and outputs abnormal location candidates excluding the normal locations as abnormal locations (S110).

Description

Technical Field

[0001] The present disclosure relates to an abnormal location detection device, an abnormal location detection method, and an abnormal location detection program.

Background Art

[0002] There is a technique for detecting abnormal locations such as damage to an electric wire by analyzing an image of the electric wire such as a transmission line. However, members such as anti-snow rings for suppressing snow accumulation on the electric wire may be attached to the electric wire at equal intervals. In such a case, there is a risk that the members such as anti-snow rings may be included in the abnormal locations and detected.

[0003] Patent Document 1 discloses a technique for obtaining an electric wire region from image data taken from a helicopter, obtaining a representative value of each pixel in the horizontal direction, determining a fiber cross-section from the change in the value, and changing an abnormal detection method depending on the presence or absence of the cross-section. In Patent Document 1, when a specified number or more of abnormalities are detected from the image data, the periodicity of the abnormalities is determined, and if there is periodicity, the threshold value for abnormal detection is lowered so as not to be detected as an abnormality.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, in the technique described in Patent Document 1, if the threshold value is lowered because a plurality of abnormal locations have periodicity, there is a risk of simultaneously removing minor abnormalities other than normal locations. Further, in Patent Document 1, when the threshold value for abnormal detection is high, a plurality of structures existing at equal intervals on the electric wire such as anti-snow rings are detected as abnormalities, and there is a problem that confirmation of the detection result and subsequent processing become complicated.

[0006] The purpose of this disclosure is to provide an abnormal location detection device, an abnormal location detection method, and an abnormal location detection program that can distinguish between abnormal and normal locations even when multiple abnormal locations have a periodic pattern. [Means for solving the problem]

[0007] The abnormality detection device disclosed herein is elongated of An abnormality detection unit detects candidate abnormality locations from image features contained in the image of the object to be inspected, and determines the periodicity of the candidate abnormality locations. The difference A regular periodicity determination unit, an abnormal image comparison unit that compares the images of each of the abnormal location candidates having periodicity and calculates the similarity of each abnormal location candidate, and the similarity calculated by the abnormal image comparison unit If the predetermined threshold is greater than expensive The aforementioned Excluding potential abnormal areas as normal areas death , Excluding the aforementioned excluded candidate abnormal locations Candidate locations for abnormalities different It is characterized by comprising a normal location removal unit that outputs normal locations.

[0008] The abnormal location detection method disclosed herein is an abnormal location detection method performed by a computer, and is elongated of The steps include detecting candidate abnormal locations from image features contained in the image of the object to be inspected, and determining the periodicity of the candidate abnormal locations. Rus Steps include comparing the image of the step with each of the periodic abnormality candidate images to calculate the similarity of each abnormality candidate, and the calculated similarity If the predetermined threshold is greater than expensive The aforementioned Excluding potential abnormal areas as normal areas death , Excluding the aforementioned excluded candidate abnormal locations Candidate locations for abnormalities different It includes a step of outputting as a regular location.

[0009] The abnormality detection program in this disclosure is long-shaped of The steps include detecting candidate abnormal locations from image features contained in the image of the object to be inspected, and determining the periodicity of the candidate abnormal locations. RusStep of comparing each image of the abnormal location candidates having the periodicity with the steps and calculating the similarity of each abnormal location candidate, and the calculated similarity is If the predetermined threshold is greater than high The aforementioned Exclude the abnormal location candidate with a high similarity as a normal location death , Excluding the aforementioned excluded candidate abnormal locations Abnormal location candidate different Output as a normal location, and cause a computer to execute.

Effect of the Invention

[0010] According to the present disclosure, even when a plurality of abnormal locations have periodicity, it is possible to provide an abnormal location detection device, an abnormal location detection method, and an abnormal location detection program capable of distinguishing between abnormal locations and normal locations.

Brief Description of the Drawings

[0011] [Figure 1] is a functional configuration diagram showing an abnormal location detection device according to Embodiment 1. [Figure 2] is a schematic diagram showing an example of an electric wire that is an object to be inspected. [Figure 3] [[ID=3l]]is a graph in which the length of the electric wire is set on the horizontal axis and the average value of the luminance values of each column of the electric wire region is set on the vertical axis in the electric wire region electric wire image. [Figure 4] is a hardware configuration diagram showing an abnormal location detection device according to Embodiment 1. [Figure 5] is a flowchart showing an example of the determination process of the abnormal location detection device according to Embodiment 1. [[ID=三十九]] [Figure 6] is a functional configuration diagram showing an abnormal location detection device according to Embodiment 2. [Figure 7] is a flowchart showing an example of the determination process of the abnormal location detection device according to Embodiment 2. [Figure 8] is a functional configuration diagram showing an abnormal location detection device according to Embodiment 3. [Figure 9] is a flowchart showing an example of the determination process of the abnormal location detection device according to Embodiment 3. [Figure 10]This is a functional configuration diagram showing an abnormal location detection device according to Embodiment 4. [Figure 11] This is a flowchart showing an example of the determination process of the abnormal location detection device according to Embodiment 4. [Modes for carrying out the invention]

[0012] The abnormal location detection device according to the embodiment will be described below with reference to the drawings. The following embodiment is merely an example, and it is possible to combine the embodiments as appropriate and modify each embodiment as appropriate.

[0013] Embodiment 1 Figure 1 is a functional configuration diagram showing an abnormal location detection device 100 according to Embodiment 1. The abnormal location detection device 100 includes an abnormal location detection unit 10 that detects abnormal location candidates from the image features of an input wire area wire image 50, an abnormal periodicity determination unit 12 that determines the periodicity of the abnormal location candidates when there are multiple detected abnormal location candidates, an abnormal image comparison unit 14 that performs image comparison of the abnormal location candidates and determines the similarity between them when the abnormal location candidates have periodicity, and a normal location removal unit 16 that outputs the result of excluding abnormal location candidates with high similarity as normal locations from the abnormal locations detected by the abnormal location detection unit 10 as abnormal locations 52.

[0014] The wire area image (hereinafter abbreviated as "wire area image") 50 is an image of the wire area of ​​a long-length object to be inspected, such as a power transmission line 400, as shown in Figure 2, and is acquired by aerial photography using an aircraft such as a helicopter or an unmanned aircraft such as a drone.

[0015] The anomaly detection unit 10 is constructed, for example, by training a mathematical model such as a CNN (Convolutional Neural Network) using machine learning with the wire region image containing anomalies 52 and 420 as training data. After training, the anomaly detection unit 10 detects candidate anomalies from image features extracted from the wire region image 50 by image comparison or brightness value comparison, etc.

[0016] The abnormal periodicity determination unit 12 determines whether or not the abnormal location candidates detected by the abnormal location detection unit 10 have periodicity. For example, the snow-resistant ring 410 and the finned power line tend to have consecutive abnormal location candidates detected. The abnormal periodicity determination unit 12 determines the periodicity of the abnormal location candidates based on one of the following, as an example: (1) the magnitude of the variance of the distances between abnormal location candidates, (2) the degree of deviation of each value of the distance between abnormal location candidates from the median value of the distances between abnormal location candidates, or (3) the brightness value of the pixels in the power line area.

[0017] The determination in (1), based on the magnitude of the variance in the distances between candidate anomalies, specifically involves the distance d between adjacent candidate anomalies. i (i=1, 2, 3, ..., n: n is the interval between adjacent candidate anomalies) Obtain the total number of samples of distances, and the distance d shown in the following formula (1) i Variance s 2 The system determines whether there is variation in the distance between candidate abnormal locations based on the value of . The distance between candidate abnormal locations is measured based on the number of pixels between abnormal locations in the wire region image 50. The abnormal periodicity determination unit 12 determines the variance s 2 If the value is less than or equal to a predetermined variance threshold, the abnormal location detection unit 10 detects it. It is determined that the candidate abnormal locations have periodicity. The variance threshold is calculated, for example, from the variance s calculated from the wire region image of a wire to which components such as the snow-resistant ring 410 are attached, and from the wire region image of a wire to which components such as the snow-resistant ring 410 have abnormal locations such as damage. 2 The value The determination is made by comparison and consideration. Alternatively, if the abnormal periodicity determination unit 12 is constructed by machine learning using wire region images of electric wires to which components such as snow-resistant rings 410 are attached, and wire region images of electric wires that have abnormal areas such as damage in addition to components such as snow-resistant rings 410, respectively, then the abnormal periodicity determination unit 12 may maintain a variance threshold as a result of the machine learning.

[0018]

number

[0019] (2) When determining based on the degree of deviation of each value of the distance between candidate abnormal locations from the median of the distances between candidate abnormal locations, the abnormal periodicity determination unit 12 measures the distance between candidate abnormal locations based on the number of pixels between abnormal locations in the wire region image 50, as in (1) above, and extracts the median of the distances between adjacent candidate abnormal locations. The abnormal periodicity determination unit 12 then calculates the number of samples of the distances between candidate abnormal locations that are within a certain range from the median, and determines that the candidate abnormal locations detected by the abnormal location detection unit 10 have periodicity if the ratio of this number to the total number of samples n is equal to or greater than a predetermined ratio threshold. The certain range from the median and the ratio threshold are, as an example, the distance d between candidate abnormal locations calculated from the wire region image of a wire to which a component such as a snow-resistant ring 410 is attached, and the wire region image of a wire to which a component such as a snow-resistant ring 410 has abnormal locations such as damage. i Compare and consider the values. This is determined by the following: Alternatively, if the abnormal periodicity determination unit 12 is constructed by machine learning using wire region images of electric wires to which components such as snow-resistant rings 410 are attached, and wire region images of electric wires that have abnormal areas such as damage in addition to components such as snow-resistant rings 410, respectively, as training data, then the abnormal periodicity determination unit 12 may maintain a certain range and a percentage threshold as a result of the machine learning.

[0020] (3) The determination based on the brightness values ​​of pixels in the wire region is performed by calculating the average value of the brightness values ​​of each column in the wire region and determining whether the candidate abnormality location has periodicity from a graph, as shown in Figure 3, in which the horizontal axis is the length of the wire (longitudinal distance) and the vertical axis is the average value of the brightness values ​​of each column in the wire region. Each column in the wire region refers to each of the rows of pixels orthogonal to the row of pixels representing the wire in the wire region image 50. The abnormality periodicity determination unit 12 determines that the candidate abnormality location detected by the abnormality location detection unit 10 has periodicity if there are locations with similar brightness values ​​at approximately equal intervals in the graph described above. For example, the snow-resistant ring 410 is often made of black resin, so in Figure 3, the brightness value shows a minimum value at a constant period.

[0021] The abnormal periodicity determination unit 12 passes the processing to the subsequent abnormal image comparison unit 14 if the abnormal location candidate detected by the abnormal location detection unit 10 has periodicity. If the abnormal location candidate detected by the abnormal location detection unit 10 does not have periodicity, the abnormal periodicity determination unit 12 outputs the abnormal location candidate as abnormal location 52.

[0022] The abnormal image comparison unit 14, when the abnormal location candidates exhibit periodicity, compares each image of the abnormal location candidates and calculates the similarity of the images. The similarity of the images is, for example, the sum of the squares of the deviations in the brightness values ​​of pixels located at corresponding positions in each image of the two abnormal location candidates being compared. The closer the sum of squares is to 0, the higher the similarity between the two images. The abnormal image comparison unit 14 determines that the similarity between the two images is above a certain level if the calculated sum of squares is less than or equal to a predetermined similarity threshold. The similarity threshold is determined, for example, by comparing and considering the sum of the squares of the deviations in the brightness values ​​of pixels located at corresponding positions in each image of the two abnormal location candidates to be compared, extracted from both an image of the wire region of a power line to which a component such as a snow-resistant ring 410 is attached and an image of the wire region of a power line to which a component such as a snow-resistant ring 410 has abnormal locations such as damage in addition to the component. Alternatively, if the abnormal periodicity determination unit 12 is constructed by machine learning using wire region images of electric wires to which components such as snow-resistant rings 410 are attached, and wire region images of electric wires that have abnormal areas such as damage in addition to components such as snow-resistant rings 410, respectively, then the abnormal periodicity determination unit 12 may maintain a similarity threshold as a result of the machine learning.

[0023] The normal area removal unit 16 determines that if the similarity of the images of the abnormal area candidates is above a certain level, the abnormal area candidate is a special-shaped wire such as a periodically occurring snow-resistant ring 410 or a finned wire. The normal area removal unit 16 considers the abnormal area candidates with a similarity above a certain level as normal areas and excludes them from the abnormality detection results, outputting the excluded abnormal area candidates as abnormal areas 52.

[0024] Figure 4 is a hardware configuration diagram showing the abnormal location detection device 100 according to Embodiment 1. The abnormal location detection device 100 consists of a CPU (Central Processing Unit) 210, main memory 220, input / output interface (I / O interface) 230, and storage unit 240, each of which is connected to one another via a system bus 250. The abnormal location detection device 100 may consist of multiple computers.

[0025] The CPU 210 is an integrated circuit (IC) that performs arithmetic processing. In addition to the CPU 210, other processing elements such as a digital signal processor (DSP) or a graphics processing unit (GPU) may also be used. The CPU 210 operates as an abnormal location detection function that detects abnormal location candidates from the wire region image 50 by executing the abnormal location detection program, an abnormal period determination function that determines whether the abnormal location candidates detected by the abnormal location detection function have periodicity, an abnormal image comparison function that, if the abnormal location candidates have periodicity, compares each image of the abnormal location candidates and calculates the similarity of the images, and a normal location removal function that considers abnormal location candidates with a similarity above a certain level as normal locations and excludes them from the abnormal detection results, outputting the excluded abnormal location candidates as abnormal locations 52. As a result, the CPU 210 functions as an abnormal location detection unit 10, an abnormal periodicity determination unit 12, an abnormal image comparison unit 14, and a normal location removal unit 16 by executing the abnormal location detection program. The abnormal location detection program is provided, for example, on a recording medium on which these are recorded.

[0026] The main memory 220 is composed of a volatile storage device such as RAM (Random Access Memory) or a non-volatile storage device such as ROM (Read Only Memory). The storage unit 240 is composed of a non-volatile storage device such as an HDD (Hard Disk Drive) or flash memory.

[0027] The I / O interface 230 is a port to which the input device, camera 300, and the output device, display device 310, are connected. Specific examples of the I / O interface 230 include USB (Universal Serial Bus) terminals, IEEE 1394 terminals, or Thunderbolt terminals, and also include communication interfaces such as Ethernet (registered trademark). The camera 300 is an imaging device that acquires power line area images 50 through aerial photography, etc., and by being connected to the I / O interface 230, the acquired power line area images 50 are input to the abnormal location detection device 100. In this embodiment, it is assumed that the camera 300 is connected to the I / O interface 230 after aerial photography by an aircraft such as a helicopter, but if wireless communication is possible for both the I / O interface 230 and the camera 300, the power line area images 50 acquired by the camera 300 may be input to the I / O interface 230 in real time via wireless communication. The display device 310 is a display, etc., but may also include a printer, etc., that outputs information.

[0028] Figure 5 is a flowchart showing an example of the determination process of the abnormal location detection device 100 according to Embodiment 1. In step S101, the wire area image 50 captured by the camera 300 is acquired via the I / O interface 230.

[0029] In step S102, the abnormality detection unit 10 detects candidate abnormalities by comparing images or brightness values, and creates a list of the detected candidate abnormalities.

[0030] In step S103, the abnormal periodicity determination unit 12 calculates the period of the candidate abnormal location. In Embodiment 1, as described above, the periodicity of the candidate abnormal location is calculated based on one of the following: (1) the variance of the distances between candidate abnormal locations, (2) the distance between candidate abnormal locations relative to the median of the distances between candidate abnormal locations, or (3) the brightness value of the pixels in the wire region.

[0031] In step S104, the abnormal periodicity determination unit 12 determines whether the candidate abnormal location exhibits periodicity. If the candidate abnormal location exhibits periodicity in step S104, the procedure proceeds to step S105. If the candidate abnormal location does not exhibit periodicity, the abnormal location detected by the abnormal location detection unit 10 is considered to be the abnormal location 52, and the procedure proceeds to step S110.

[0032] In step S105, for each candidate anomaly location, loop 1 is started, repeating the procedures from step S106 to step S108. In step S106, the anomaly image comparison unit 14 calculates the similarity between images of adjacent candidate anomalies. In Embodiment 1, as described above, the image similarity is the sum of the squares of the deviations in the brightness values ​​of pixels at corresponding positions in each image of the two candidate anomalies being compared. The closer this sum of squares is to 0, the higher the similarity between the two images.

[0033] In step S107, the abnormal image comparison unit 14 determines whether the similarity is above a predetermined standard. The abnormal image comparison unit 14 determines that the similarity of both images is above the standard if the sum of the squares of the deviations in the brightness values ​​of the pixels of the two abnormal location candidates being compared is less than or equal to a predetermined similarity threshold. In step S107, if the similarity is above the standard, the procedure proceeds to step S108; if the similarity is not above the standard, the procedure proceeds to step S109. In Embodiment 1, an abnormal location candidate whose similarity is not above the standard has a high probability of being abnormal location 52. However, in Embodiment 1, as described above, the similarity of adjacent abnormal location candidates is determined, so it is not possible to determine that two adjacent abnormal location candidates are abnormal location 52 based solely on the similarity of those two adjacent abnormal location candidates. This is because even if two abnormal location candidates do not have a similarity to each other above the standard, it is possible that their similarity to other adjacent abnormal location candidates may be above the standard. In Embodiment 1, normal locations are excluded from the detection results. After the processing of Loop 1 is performed for all candidate abnormal locations, any candidate abnormal locations that were not excluded as normal locations become abnormal locations 52.

[0034] In step S108, the normal location removal unit 16 considers abnormal location candidates with a similarity score above the standard as normal locations and excludes them from the abnormal periodicity determination unit 12's determined abnormal location candidates to have periodicity. In Embodiment 1, as described above, the similarity score of adjacent abnormal location candidates is determined, so if the similarity score is above the standard, each of the two adjacent abnormal location candidates involved in the calculation of the similarity score is considered a normal location.

[0035] In step S109, the normal area removal unit 16 determines whether it has determined the similarity of all abnormal area candidates that have been determined to have periodicity. If it has determined the similarity of all abnormal area candidates that have been determined to have periodicity, it terminates the processing of loop 1 and proceeds to step S110. If it has not determined the similarity of all abnormal area candidates that have been determined to have periodicity, it executes the processing of loop 1 from step S106 to step S108.

[0036] In step S110, the normal location removal unit 16 outputs the abnormal location candidates, which were determined not to have periodicity in step S104, along with the abnormal location candidates that were deemed to have periodicity, as abnormal location candidates to the display device 310 as abnormal location 52, and terminates the process.

[0037] As described above, in Embodiment 1, the abnormal periodicity determination unit 12 determines whether or not the candidate abnormal location has periodicity. If the candidate abnormal location has periodicity, the abnormal image comparison unit 14 performs an image comparison with other abnormal location candidates that have periodicity to determine the degree of similarity. Then, if the degree of similarity between the abnormal location candidates is high, the normal location removal unit 16 assumes that the location is a part of a specially shaped wire, such as a snow-resistant ring 410 or a finned wire, that is uniformly present on the wire, and removes the corresponding location from the abnormality detection result. As a result, by removing normal locations such as snow-resistant rings 410 from the abnormal location candidates, it is possible to distinguish between abnormal locations 52 and normal locations even if multiple abnormal locations 52 have periodicity.

[0038] According to Embodiment 1, since non-abnormal areas can be removed from the abnormal area candidates detected in a state where minor abnormal areas can be detected, abnormal areas can be accurately detected while maintaining a high detection sensitivity for abnormal area candidates and excluding normal areas. Furthermore, by reducing the number of normal areas from the abnormal area candidates, the time required for result confirmation and subsequent processing can be shortened.

[0039] Embodiment 2 Next, the abnormal location detection device 110 according to Embodiment 2 will be described. The abnormal location detection device 110 according to Embodiment 2 shown in Figure 6 differs from Embodiment 1 in that it includes a line section information database (DB) 60 that pre-registers line sections (sections) that use specially shaped electric wires such as snow-resistant rings 410 or finned electric wires, and a processing necessity determination unit 22 that refers to the line section information DB 60 to determine whether or not to execute the process of removing normal locations according to Embodiment 1. However, the other configurations are the same as those of Embodiment 1, so the same reference numerals as in Embodiment 1 are used for the same components, and their detailed descriptions are omitted. Also, the hardware configuration of Embodiment 2 is the same as that of Embodiment 1, so a detailed description is omitted, but in Embodiment 2, the CPU 210 acts as an abnormal location detection function, an abnormal period determination function, an abnormal image comparison function, and a normal location removal function, as well as a processing necessity determination function that refers to the line section information DB 60 to determine whether or not to execute the process of removing normal locations according to Embodiment 1. As a result, the CPU 210 functions as an abnormal location detection unit 10, an abnormal periodicity determination unit 12, an abnormal image comparison unit 14, a normal location removal unit 16, and a processing necessity determination unit 22 by executing the abnormal location detection program.

[0040] Figure 7 is a flowchart showing an example of the determination process of the abnormal location detection device 110 according to Embodiment 2. The flowchart shown in Figure 7 differs from Embodiment 1 in that it includes a step S201 in which, along with the wire area image 50, line section information, which is information on line sections using specially shaped wires such as snow-resistant rings 410 or finned wires, is obtained from the line section information DB 60, and a step S202 in which, by referring to the line section information, it is determined whether or not the line section where the abnormal location candidate detected by the abnormal location detection unit 10 is located corresponds to a line section using specially shaped wires. However, the other procedures are the same as in Embodiment 1, so the same reference numerals are used for the same procedures as in Embodiment 1 and detailed explanations are omitted.

[0041] In step 201, the abnormality detection unit 10 acquires the wire area image 50, and the processing necessity determination unit 22 acquires line section information by referring to the line section information DB 60. Similar to Embodiment 1, in step S102, the abnormality detection unit 10 detects candidate abnormalities from the wire area image 50 by image comparison or brightness value comparison and creates a list of detected candidate abnormalities.

[0042] In step S202, the processing necessity determination unit 22 determines whether the candidate abnormal location detected by the abnormal location detection unit 10 belongs to a line section using specially shaped electric wires. If the detected candidate abnormal location in step S202 belongs to a line section using specially shaped electric wires, the procedure proceeds to step S103. If the detected candidate abnormal location does not belong to a line section using specially shaped electric wires, the candidate abnormal location is considered an abnormal location 52, and the procedure proceeds to step S110.

[0043] In step S110, the abnormal location 52 detected in the loop 1 process, which is structured in the same way as in Embodiment 1, from steps S105 to S109, and the abnormal location 52 detected in steps S104 and S202 are each output to the display device 310, and the process ends.

[0044] As described above, in Embodiment 2, if a candidate for an abnormal location detected by the abnormal location detection unit 10 is located in a location registered in the line section information DB 60, the process of removing the normal location according to Embodiment 1 is executed. If the candidate is not located in a location registered in the line section information DB 60, it is output as an abnormal location 52. As a result, the probability of mistakenly removing an abnormal location 52 as a normal location is suppressed, and abnormal locations 52 in the power lines can be accurately detected.

[0045] Embodiment 3 Next, the abnormal location detection device 120 according to Embodiment 3 will be described. The abnormal location detection device 120 according to Embodiment 3 shown in Figure 8 differs from Embodiment 1 in that the normal location removal unit 36 ​​separates and outputs abnormal locations 52 and locations requiring final judgment 54, which must be judged by personnel, based on the similarity between the abnormal location candidates. However, the other configurations are the same as those of Embodiment 1, so the same reference numerals as in Embodiment 1 are used for the same components, and their detailed descriptions are omitted. Also, the hardware configuration of Embodiment 3 is the same as that of Embodiment 1, so a detailed description is omitted.

[0046] Figure 9 is a flowchart showing an example of the determination process of the abnormal location detection device 120 according to Embodiment 3. In the flowchart shown in Figure 9, the process of loop 1 from step S305 to step S311 differs from Embodiment 1 in that abnormal location candidates whose similarity to adjacent abnormal location candidates is equal to or greater than the first criterion are considered normal locations, and if the similarity of an abnormal location candidate whose similarity is not equal to or greater than the first criterion is equal to or greater than the second criterion which is lower than the first criterion, it is classified as an abnormal location 54 requiring determination by the user for similarity determination. However, the other procedures are the same as in Embodiment 1, so the same reference numerals are used for the same procedures as in Embodiment 1 and detailed explanations are omitted.

[0047] In step S305, for each candidate anomaly location, loop 1 is started, repeating the procedures from step S306 to step S310. In step S306, the anomaly image comparison unit 14 calculates the similarity between images of adjacent candidate anomalies. Similar to Embodiment 1, the image similarity is the sum of the squares of the deviations in the brightness values ​​of pixels at corresponding positions in each image of the two candidate anomalies being compared. The closer this sum of squares is to 0, the higher the similarity between the two images.

[0048] In step S307, the abnormal image comparison unit 14 determines whether the candidate abnormal location determined to have periodicity in step S104 shows a high degree of similarity, specifically whether the similarity is equal to or greater than the first criterion. The abnormal image comparison unit 14 determines that the similarity of both images is equal to or greater than the first criterion if the sum of the squares of the deviations in the brightness values ​​of the pixels of the two candidate abnormal location being compared is less than or equal to a predetermined first similarity threshold. In step S307, if the similarity is equal to or greater than the first criterion, the procedure proceeds to step S308; otherwise, the procedure proceeds to step S309.

[0049] In step S308, the normal location removal unit 36 ​​considers abnormal location candidates with a similarity of the first criterion or higher as normal locations and excludes them from the abnormal location candidates that were determined to have periodicity in step S104. In Embodiment 3, as described above, the similarity of adjacent abnormal location candidates is determined, so if the similarity is equal to or higher than the first criterion, each of the two adjacent abnormal location candidates involved in the calculation of that similarity is considered a normal location.

[0050] In step S309, the abnormal image comparison unit 14 determines whether the abnormal location candidate, which was determined in step S307 to have a similarity of less than or equal to the first criterion, exhibits a moderate similarity, specifically whether the similarity is lower than the first criterion but higher than or equal to the second criterion. The abnormal image comparison unit 14 determines that the similarity of both images is at or above the second criterion if the sum of the squares of the deviations in the brightness values ​​of the pixels of the two abnormal location candidates being compared is less than or equal to the second similarity threshold, which is greater than the first similarity threshold. In step S309, if the similarity is at or above the second criterion, the procedure proceeds to step S310; otherwise, the procedure proceeds to step S311. In Embodiment 3, an abnormal location candidate whose similarity is not at or above the second criterion has a high probability of being abnormal location 52. However, in Embodiment 3, as described above, the similarity of adjacent abnormal location candidates is determined, so it is not possible to determine that two adjacent abnormal location candidates are abnormal location 52 based solely on the similarity of those two adjacent abnormal location candidates. Even if two candidate abnormal locations do not have a similarity score of at least the second criterion, it is possible that their similarity to other adjacent candidate abnormal locations may be at or above the first criterion, or at or above the second criterion. In Embodiment 3, each normal location or location requiring further consideration (described later) is excluded from the detection results. After the processing of Loop 1 is performed for all candidate abnormal locations, any candidate abnormal locations that were not excluded as normal locations or locations requiring further consideration become abnormal locations 52.

[0051] The first similarity threshold and the second similarity threshold are determined, for example, by comparing and considering the sum of the squared deviations of the brightness values ​​of pixels at corresponding positions in each image of two candidate abnormal locations to be compared, extracted from an image of the wire region of an electric wire to which a component such as a snow-resistant ring 410 is attached, and an image of the wire region of an electric wire to which a component such as a snow-resistant ring 410 is attached, as well as an abnormal location such as damage. Alternatively, if the abnormal periodicity determination unit 12 is constructed by machine learning using the image of the wire region of an electric wire to which a component such as a snow-resistant ring 410 is attached, and the image of the wire region of an electric wire to which a component such as a snow-resistant ring 410 is attached, as well as an abnormal location such as damage, as training data, the abnormal periodicity determination unit 12 may maintain the first similarity threshold and the second similarity threshold as a result of the machine learning.

[0052] In step S310, the normal area removal unit 36 ​​considers abnormal area candidates with a similarity of the second criterion or higher as areas requiring review 54, and excludes them from the abnormal area candidates that were determined in step S307 to have a similarity of less than the first criterion. In Embodiment 3, as described above, the similarity of adjacent abnormal area candidates is determined, so if the similarity is greater than or equal to the second criterion, each of the two adjacent abnormal area candidates involved in the calculation of that similarity is considered an area requiring review.

[0053] In step S311, the normal area removal unit 36 ​​determines whether it has determined the similarity of all abnormal area candidates that have been determined to have periodicity. If it has determined the similarity of all abnormal area candidates that have been determined to have periodicity, it terminates the processing of loop 1 and proceeds to step S312. If it has not determined the similarity of all abnormal area candidates that have been determined to have periodicity, it executes the processing of loop 1 from step S306 to step S310.

[0054] In step S312, the abnormal location 52 and the location requiring further consideration 54, as described above, are output to the display device 310, and the process is terminated.

[0055] As described above, in Embodiment 3, similar to Embodiment 1 or Embodiment 2, abnormal location candidates are assigned levels based on their similarity. Abnormal location candidates with a high similarity level that are reliably considered to be normal locations are removed as normal locations, while abnormal location candidates with a low similarity level are output as abnormal locations 52. Then, abnormal location candidates that do not fall into either the normal locations or abnormal locations 52, and have a moderate similarity level, are presented to the user via the display device 310, and the user is allowed to decide whether or not to exclude them. As a result, by incorporating the user's perspective, it becomes possible to remove normal locations with greater accuracy for abnormal location candidates that do not fall into either the normal locations or abnormal locations 52.

[0056] Embodiment 4 Next, the abnormal location detection device 130 according to Embodiment 4 will be described. The abnormal location detection device 130 according to Embodiment 4 shown in Figure 10 differs from Embodiment 1 in that the normal location removal unit 46 removes abnormal location candidates that have a similarity level above a certain level and considers them as normal locations, and passes the result of removal to the abnormal location detection unit 40. The abnormal location detection unit 40 then performs image comparison or brightness value comparison, etc., on the abnormal location candidates from which the normal locations have been removed, and outputs the detected abnormal location candidates as abnormal locations 52. However, the other configurations are the same as those of Embodiment 1, so the same reference numerals as in Embodiment 1 are used for the same configurations, and their detailed descriptions are omitted. Also, the hardware configuration of Embodiment 4 is the same as the hardware configuration of Embodiment 1, so a detailed description is omitted.

[0057] Figure 11 is a flowchart showing an example of the determination process of the abnormal location detection device 130 according to Embodiment 2. The flowchart shown in Figure 11 differs from Embodiment 1 in that it includes step S401, which involves performing image comparison or brightness value comparison, etc., again on the abnormal location candidates from which normal locations have been removed, in order to detect the abnormal location 52. However, the other procedures are the same as in Embodiment 1, so the same reference numerals are used for the same procedures as in Embodiment 1, and detailed explanations are omitted.

[0058] In step S401, the abnormality detection unit 40 performs image comparison or brightness value comparison, etc., on the abnormality candidates from which normal areas have been removed by the processing of loop 1. The abnormality candidates detected as abnormal areas in step S401 are then extracted as abnormality areas 52.

[0059] In step S110, the abnormal location candidates determined to have no periodicity in step S104 and the abnormal location candidates detected in step S401 are output to the display device 310 as abnormal location 52, and the process ends.

[0060] As explained above, in Embodiment 4, the abnormal location detection unit 40 identifies any abnormal location candidate that has been detected again as an abnormal location from the abnormal location candidates that have been excluded from the normal locations as abnormal location 52. As previously stated, the abnormal image comparison unit 14 indicates the similarity by the sum of the squares of the deviations in the brightness values ​​of the pixels located at the corresponding positions in each image of the two abnormal location candidates being compared. For example, when comparing a normal location and an abnormal location 52, if the sum of the squares of the deviations in the brightness values ​​of the pixels located at the corresponding positions is close to 0, the abnormal image comparison unit 14 may misidentify the abnormal location 52 as a normal location. In Embodiment 4, by performing abnormal location detection again on abnormal location candidates that may include the abnormal location 52 that was misidentified as a normal location, the abnormal location 52 that was misidentified by similarity is detected. As a result, the abnormal location 52 can be accurately detected.

[0061] Variant form In Embodiments 1 to 4, the object to be inspected was an electric wire, but the system is not limited to this. For example, abnormalities may be detected in railway overhead lines and tracks, as well as in tunnels and other long structures such as pipelines. Abnormalities in railway overhead lines and tracks can be detected using the same procedure as described in Embodiments 1 to 4. For example, joints in railway tracks are provided at equal intervals, so they can be excluded from the list of abnormal locations as normal locations with periodicity. In tunnels, abnormal locations, including cracks in the walls, can be detected from images of the inside of the tunnel, and joints that occur in each work section during concrete pouring can be removed from the list of abnormal locations as normal locations by determining their periodicity or similarity. In pipelines, abnormal locations, such as rust or deformation of pipes, can be detected from images of the exterior of the pipeline, and members such as flanges that join pipes together can be removed from the list of abnormal locations as normal locations by determining their periodicity or similarity. [Explanation of Symbols]

[0062] 10 Anomaly detection unit, 12 Anomaly periodicity determination unit, 14 Anomaly image comparison unit, 16 Normal area removal unit, 22 Processing necessity determination unit, 36 Normal area removal unit, 40 Anomaly detection unit, 46 Normal area removal unit, 52 Anomaly area, 54 Area requiring judgment, 60 Line section information DB, 100, 110, 120, 130 Anomaly detection device, 210 CPU, 220 Main memory, 230 I / O interface, 240 Storage unit, 300 Camera, 310 Display device

Claims

1. An abnormality detection unit detects candidate abnormality locations from image features contained in the captured image of a long object to be inspected, An abnormal periodicity determination unit for determining the periodicity of the candidate abnormal location, An abnormal image comparison unit compares each image of the aforementioned periodic abnormal location candidates and calculates the similarity of each abnormal location candidate, A normal area removal unit removes abnormal area candidates whose similarity calculated by the abnormal image comparison unit is higher than a predetermined threshold, and outputs the abnormal area candidates excluding the excluded abnormal area candidates as abnormal areas. An abnormal location detection device equipped with the following features.

2. A railway line information database in which sections having special shapes of the objects to be inspected are registered in advance, A processing necessity determination unit that refers to the aforementioned line section information database, determines whether the candidate abnormal location belongs to a section having the special shape registered in the aforementioned line section information database, and outputs the candidate abnormal location that does not belong to a section having the special shape as the abnormal location, Furthermore, The abnormality location detection device according to claim 1, wherein the abnormality periodicity determination unit determines the periodicity of the candidate abnormal location belonging to the section having the special shape, as determined by the processing necessity determination unit.

3. The abnormal location detection device according to claim 1, wherein the normal location removal unit excludes abnormal location candidates whose similarity is equal to or greater than the first criterion as normal locations, and among the abnormal location candidates whose similarity is not equal to or greater than the first criterion, abnormal location candidates whose similarity is less than the first criterion and equal to or greater than the second criterion are to be judged, and outputs the abnormal location candidates, excluding the normal locations and the locations to be judged, together with the locations to be judged as abnormal locations.

4. The abnormal location detection device according to claim 1, wherein the abnormal location detection unit detects abnormal location candidates again from the abnormal location candidates that have been excluded from the normal location removal unit, and outputs the detected abnormal location candidates as the abnormal location.

5. The abnormal periodicity determination unit determines the periodicity for a plurality of abnormal location candidates based on the dispersion of the distances between adjacent abnormal location candidates, as described in any one of claims 1 to 4.

6. An abnormal location detection device according to any one of claims 1 to 4, wherein the abnormal periodicity determination unit determines the periodicity for a plurality of abnormal location candidates based on the degree of deviation of the distance values ​​between each abnormal location candidate from the median value of the distances between adjacent abnormal location candidates.

7. An abnormal location detection device according to any one of claims 1 to 4, wherein the abnormal periodicity determination unit calculates the average value of the brightness values ​​of rows of pixels of the object to be inspected that are perpendicular to the longitudinal direction of the object to be inspected in the captured image of the object to be inspected, and determines that the candidate abnormal location has the periodicity when the average value of the brightness values ​​shows brightness values ​​that are approximately equal in intervals along the longitudinal distance of the object to be inspected.

8. A method for detecting abnormal locations performed by a computer, A step of detecting candidate abnormal areas from image features contained in the image of a long object to be inspected, A step of determining the periodicity of the candidate abnormal location, The steps include comparing images of each of the candidate abnormal locations having periodicity and calculating the similarity of each candidate abnormal location, The steps include: excluding the candidate abnormal location whose calculated similarity is higher than a predetermined threshold as a normal location, and outputting the candidate abnormal location excluding the excluded candidate abnormal location as an abnormal location; A method for detecting abnormal locations that have the following characteristics.

9. A step of detecting candidate abnormal areas from image features contained in the image of a long object to be inspected, A step of determining the periodicity of the candidate abnormal location, The steps include comparing images of each of the candidate abnormal locations having periodicity and calculating the similarity of each candidate abnormal location, The steps include: excluding the candidate abnormal location whose calculated similarity is higher than a predetermined threshold as a normal location, and outputting the candidate abnormal location excluding the excluded candidate abnormal location as an abnormal location; An anomaly detection program that is executed by a computer.

Images