Information processing device, detection method, and detection program

Abstract

The information processing device (100) has an acquisition unit (120) that acquires an image of the twisted wire, a division unit (130) that divides the image of the twisted wire, and a calculation unit (140) that calculates multiple similarities using multiple target images set from the multiple images obtained by the division and multiple comparison images set from the multiple images, normalizes the multiple similarities, calculates a class classification threshold using the multiple values ​​obtained by normalization, calculates one value for each class obtained by classifying the multiple values ​​based on the class classification threshold, and calculates the difference between the two calculated values ​​as an inter-class distance, and a determination unit (150) that determines that the twisted wire is abnormal if the inter-class distance is equal to or greater than a predetermined first threshold.

Description

[Technical field]

[0001] The present disclosure relates to an information processing device, a detection method, and a detection program. [Background technology]

[0002] Twisted wires are used. For example, twisted wires are used in electric cables. Here, a technology has been proposed for detecting anomalies using the average brightness value of an image (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 10-117415 Summary of the Invention [Problem to be solved by the invention]

[0004] A method for inspecting twisted wires using an average brightness value of an image is conceivable, but this method has low accuracy in detecting abnormalities.

[0005] An objective of the present disclosure is to detect anomalies with high accuracy. [Means for solving the problem]

[0006] An information processing device according to one aspect of the present disclosure includes an acquisition unit that acquires an image of a twisted wire, a division unit that divides the image of the twisted wire, a calculation unit that calculates a plurality of similarities using a plurality of target images selected from a plurality of images obtained by division and a plurality of comparison images selected from the plurality of images, normalizes the plurality of similarities, calculates a classification threshold using a plurality of values ​​obtained by normalization, calculates one value for each class obtained by classifying the plurality of values ​​based on the classification threshold, and calculates a difference between the two calculated values ​​as an inter-class distance, and a determination unit that determines that the twisted wire is abnormal if the inter-class distance is equal to or greater than a predetermined first threshold. Effect of the Invention

[0007] According to the present disclosure, abnormalities can be detected with high accuracy. [Brief description of the drawings]

[0008] [Figure 1] FIG. 2 is a diagram illustrating hardware included in an information processing device according to the first embodiment. [Diagram 2] 1 is a block diagram showing functions of an information processing device according to a first embodiment; [Diagram 3] FIG. 4 is a diagram illustrating an example of a process of calculating a plurality of similarities according to the first embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of normalization according to the first embodiment. [Diagram 5] FIG. 4 is a diagram illustrating an example of calculation of a classification threshold value according to the first embodiment. [Figure 6] 4 is a flowchart showing an example (part 1) of processing executed by the information processing device according to the first embodiment. [Figure 7] 13 is a flowchart showing a second example of the process executed by the information processing device according to the first embodiment. [Figure 8] FIG. 11 is a block diagram showing the functions of an information processing device according to a second embodiment. [Figure 9] 13 is a flowchart showing an example of processing executed by an information processing device according to the second embodiment. [Figure 10] FIG. 11 is a diagram illustrating an overview of a process executed by an information processing device according to a third embodiment. [Figure 11] 13 is a flowchart showing an example (part 1) of processing executed by an information processing device according to the third embodiment. [Figure 12] 13 is a flowchart showing a second example of the process executed by the information processing device according to the third embodiment. [Figure 13] 13 is a flowchart showing an example of processing executed by an information processing device according to a fourth embodiment. [Figure 14] 13 is a flowchart showing an example of processing executed by an information processing device according to a modification of the fourth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment will be described with reference to the drawings. The following embodiment is merely an example, and various modifications are possible within the scope of the present disclosure.

[0010] Embodiment 1 1 is a diagram showing hardware included in an information processing device according to embodiment 1. The information processing device 100 is a device that executes a detection method. The information processing device 100 includes a processor 101, a volatile storage device 102, and a non-volatile storage device 103.

[0011] The processor 101 controls the entire information processing device 100. For example, the processor 101 is a central processing unit (CPU) or a field programmable gate array (FPGA). The processor 101 may be a multiprocessor. The information processing device 100 may also include a processing circuit.

[0012] The volatile storage device 102 is a main storage device of the information processing device 100. For example, the volatile storage device 102 is a random access memory (RAM). The non-volatile storage device 103 is an auxiliary storage device of the information processing device 100. For example, the non-volatile storage device 103 is a hard disk drive (HDD) or a solid state drive (SSD).

[0013] Next, functions of the information processing device 100 will be described. 2 is a block diagram showing functions of the information processing device of embodiment 1. The information processing device 100 includes a storage unit 110, an acquisition unit 120, a division unit 130, a calculation unit 140, a determination unit 150, and an output unit 160.

[0014] The storage unit 110 may be realized as a storage area secured in the volatile storage device 102 or the non-volatile storage device 103 .

[0015] A part or all of the acquiring unit 120, the dividing unit 130, the calculating unit 140, the determining unit 150, and the output unit 160 may be realized by a processing circuit. Also, a part or all of the acquiring unit 120, the dividing unit 130, the calculating unit 140, the determining unit 150, and the output unit 160 may be realized as a module of a program executed by the processor 101. For example, the program executed by the processor 101 is also called a detection program. For example, the detection program is recorded on a recording medium.

[0016] The storage unit 110 stores various information.

[0017] The acquisition unit 120 may acquire an image including the twisted wire. For example, the acquisition unit 120 acquires the image from the storage unit 110. Alternatively, for example, the acquisition unit 120 acquires the image from a camera. Alternatively, for example, the acquisition unit 120 acquires the image from an external device. Note that the external device is, for example, a cloud server. Also, the illustration of the external device is omitted. For example, the stranded wire is an electric wire, a wire supporting a utility pole, a wire supporting a bridge, etc.

[0018] The acquiring unit 120 may acquire an extracted image of the twisted wire from an image including the twisted wire. In other words, the acquiring unit 120 may acquire an image area of ​​the twisted wire from an image including the twisted wire. The extraction process may be executed by the information processing device 100.

[0019] The acquisition unit 120 acquires an image of the twisted wire. As described above, the acquisition unit 120 may acquire an extracted image of the twisted wire. Alternatively, the acquisition unit 120 may acquire the image of the twisted wire from the storage unit 110 or an external device.

[0020] The dividing unit 130 divides the image of the twisted wire. In detail, the dividing unit 130 divides the image of the twisted wire into pieces of a predetermined length.

[0021] The calculation unit 140 calculates a plurality of similarities using a plurality of target images set from the plurality of images obtained by division and a plurality of comparison images set from the plurality of images. Here, the calculation of the plurality of similarities will be described using a concrete example.

[0022] Fig. 3 is a diagram showing an example of a process for calculating a plurality of similarities according to the embodiment 1. Fig. 3 shows an image 10 of a twisted wire. The dividing unit 130 divides the image 10 into four. The calculation unit 140 sets image 11 as a target image from among the multiple images obtained by division. The calculation unit 140 sets image 12 as a comparison image from among the multiple images. The calculation unit 140 calculates the similarity between image 11 and image 12 using a template matching technique. Note that, for example, template matching is normalized cross-correlation, SSD (Sum of Squared Difference), etc. Furthermore, the similarity may indicate the degree to which the two images are similar, or may indicate the degree to which the two images are dissimilar.

[0023] The calculation unit 140 sets the image 12 as a target image from among the multiple images obtained by division. The calculation unit 140 sets the image 13 as a comparison image from among the multiple images. The calculation unit 140 calculates the similarity between the image 12 and the image 13.

[0024] The calculation unit 140 sets the image 13 as a target image from among the multiple images obtained by division. The calculation unit 140 sets the image 14 as a comparison image from among the multiple images. The calculation unit 140 calculates the similarity between the image 13 and the image 14.

[0025] The calculation unit 140 sets the image 14 as a target image from among the multiple images obtained by division. The calculation unit 140 sets the image 13 as a comparison image from among the multiple images. The calculation unit 140 calculates the similarity between the image 13 and the image 14.

[0026] In this way, calculation unit 140 calculates multiple similarities using multiple target images set from the multiple images obtained by division and multiple comparison images set from the multiple images.

[0027] FIG. 3 shows a case where an image next to a target image is set as a comparison image. The comparison image does not have to be the image next to the target image. For example, the comparison image may be the image two images away from the target image. Specifically, when the target image is image 11, the comparison image may be image 13. Also, for example, a part of the comparison image and a part of the target image may be the same.

[0028] The calculation unit 140 normalizes the multiple similarities. A specific example of normalization will be described with reference to the drawings. 4 is a diagram showing an example of normalization according to the first embodiment. The calculation unit 140 normalizes the multiple similarities. For example, the calculation unit 140 normalizes the multiple similarities to a value where the minimum value among the multiple similarities is 0 and the maximum value is 1. As a result, the multiple similarities are converted into values ​​from 0 to 1. In other words, the converted values ​​are represented as "0≦value≦1".

[0029] The calculation unit 140 calculates a classification threshold value using a plurality of values ​​obtained by normalization. The calculation of the classification threshold value will be described with reference to the drawings. 5 is a diagram showing an example of calculation of a classification threshold value according to the first embodiment. The calculation unit 140 calculates the degree of separation using a plurality of values. Specifically, the calculation unit 140 calculates the degree of separation using formula (1).

[0030] Separation = between-class variance / within-class variance … (1)

[0031] The value at which the degree of separation is maximum is determined as the classification threshold.

[0032] The calculation unit 140 classifies the multiple values ​​based on the classification threshold value. That is, the calculation unit 140 performs two-class classification.

[0033] The calculation unit 140 calculates one value for each classified class. For example, the calculation unit 140 calculates an average value or a representative value for each classified class. For example, when an average value is calculated, the calculation unit 140 calculates the average value of the first class using a plurality of values ​​belonging to the first class, and calculates the average value of the second class using a plurality of values ​​belonging to the second class.

[0034] The calculation unit 140 calculates the difference between the two calculated values ​​as the inter-class distance. For example, the calculation unit 140 calculates the difference between the average value of the first class and the average value of the second class as the inter-class distance.

[0035] If the inter-class distance is equal to or greater than a predetermined threshold, the determination unit 150 determines that the twisted wire is abnormal. The threshold is also called a first threshold. For example, the threshold is 0.7.

[0036] The output unit 160 outputs the result. For example, the output unit 160 outputs the result to a display of the information processing device 100. Also, for example, the output unit 160 outputs the result to an external device. Note that the result is, for example, a result of the determination.

[0037] Next, the process executed by the information processing device 100 will be described with reference to a flowchart. FIG. 6 is a flowchart illustrating an example (part 1) of the process executed by the information processing device according to the first embodiment. (Step S11) The acquisition unit 120 acquires an image of the twisted wire. (Step S12) The division unit 130 divides the image of the twisted wire. (Step S13) Calculation unit 140 calculates a plurality of similarities using a plurality of target images set from the plurality of images obtained by division and a plurality of comparison images set from the plurality of images.

[0038] (Step S14) The calculation unit 140 normalizes the multiple similarities. (Step S15) The calculation unit 140 calculates a classification threshold value using a plurality of values ​​obtained by normalization. (Step S16) The calculation unit 140 calculates an average value for each class obtained by classifying the multiple values ​​based on the classification threshold value. Then, the process proceeds to step S21.

[0039] FIG. 7 is a flowchart illustrating a second example of the process executed by the information processing device according to the first embodiment. (Step S21) The calculation unit 140 calculates the difference between the two average values ​​as the inter-class distance. (Step S22) The determination unit 150 determines whether the inter-class distance is equal to or greater than a predetermined threshold. If the inter-class distance is equal to or greater than the threshold, the process proceeds to step S23. If the inter-class distance is smaller than the threshold, the process proceeds to step S24. (Step S23) The determination unit 150 determines that the twisted wire is abnormal. (Step S24) The determining unit 150 determines that the twisted wire is normal. (Step S25) The output unit 160 outputs the result.

[0040] According to the first embodiment, the information processing device 100 can detect an abnormality with high accuracy by performing the above process. For example, even if the image of the twisted wire has a lot of noise, the information processing device 100 can detect the abnormality with high accuracy by performing the above process. Moreover, even if a small abnormality exists in the twisted wire, the information processing device 100 can detect the abnormality with high accuracy by performing the above process. Thus, the information processing device 100 can detect the abnormality with high accuracy.

[0041] Embodiment 2 Next, a description will be given of embodiment 2. In embodiment 2, differences from embodiment 1 will be mainly described. In embodiment 2, descriptions of matters common to embodiment 1 will be omitted. In the first embodiment, a case has been described in which it is determined whether or not the twisted wire is normal. In the second embodiment, a case will be described in which, if the twisted wire is abnormal, an abnormal portion is detected.

[0042] 8 is a block diagram showing functions of an information processing device according to the second embodiment. The information processing device 100 further includes a detection unit 170. A part or all of the detection unit 170 may be realized by a processing circuit. Also, a part or all of the detection unit 170 may be realized as a module of a program executed by the processor 101. The function of the detection unit 170 will be described in detail later.

[0043] Next, the process executed by the information processing device 100 will be described with reference to a flowchart. Fig. 9 is a flowchart showing an example of a process executed by the information processing device of the embodiment 2. The process in Fig. 9 is different from the process in Fig. 7 in that steps S23a and S23b are executed. Therefore, steps S23a and S23b will be described in Fig. 9. Descriptions of the processes other than steps S23a and S23b will be omitted.

[0044] (Step S23a) The acquisition unit 120 acquires an abnormal part determination threshold. The abnormal part determination threshold is also referred to as a second threshold. The process of acquiring the abnormal part determination threshold will be described. The abnormal part determination threshold may be a predetermined value. When the abnormal part determination threshold is a predetermined value, the acquisition unit 120 acquires the abnormal part determination threshold from the storage unit 110 or an external device. The calculation unit 140 may calculate the abnormal part determination threshold. For example, the calculation unit 140 may calculate the abnormal part determination threshold by using formula (2).

[0045] Abnormality determination threshold = Classification threshold × 2 … (2)

[0046] Furthermore, the calculation unit 140 may multiply the classification threshold by a value other than “2.” When the calculation unit 140 calculates the abnormal part determination threshold, the acquisition unit 120 acquires the calculated abnormal part determination threshold.

[0047] (Step S23b) The detection unit 170 detects a value equal to or greater than the abnormal portion determination threshold from among the multiple values ​​obtained by normalization. The detection unit 170 detects a portion indicated by the image corresponding to the detected value as an abnormal portion. The detection process of an abnormal portion will be described with reference to Fig. 3. The detected value is a value obtained by normalizing the similarity corresponding to the image 12. The detection unit 170 detects the portion indicated by the image 12 corresponding to the detected value as an abnormal portion.

[0048] According to the second embodiment, the information processing device 100 can detect an abnormality.

[0049] Embodiment 3 Next, a description will be given of embodiment 3. In embodiment 3, differences from embodiments 1 and 2 will be mainly described. In embodiment 3, descriptions of matters common to embodiments 1 and 2 will be omitted.

[0050] The process performed by the information processing device 100 in the third embodiment will be briefly described. Fig. 10 is a diagram showing an outline of the process executed by the information processing device of the embodiment 3. Fig. 10 shows an image 20 of a twisted wire. The image 20 shows an abnormality 21 and an abnormality 22. In the method of the second embodiment, there may be an abnormality that is not detected in one detection process. For example, the abnormality 21 is detected. However, the abnormality 22 is not detected. Therefore, the information processing device 100 deletes the similarity corresponding to the image including the abnormality 21. Then, the information processing device 100 executes the detection process of the abnormality again. As a result, the information processing device 100 can detect the abnormality 22.

[0051] Next, the process executed by the information processing device 100 will be described with reference to a flowchart. Fig. 11 is a flowchart showing an example (part 1) of the process executed by the information processing device of the third embodiment. The process in Fig. 11 differs from the process in Fig. 6 in that step S14a is executed. Therefore, step S14a will be described in Fig. 11. Descriptions of the processes other than step S14a will be omitted.

[0052] (Step S14a) The calculation unit 140 normalizes the multiple similarities. Furthermore, when step S14a is executed after step S23d, the similarities corresponding to images including abnormal areas are excluded from the multiple similarities.

[0053] Fig. 12 is a flowchart showing an example (part 2) of the process executed by the information processing device of the third embodiment. The process of Fig. 12 differs from the process of Fig. 9 in that steps S23c and S23d are executed. Therefore, steps S23c and S23d will be described in Fig. 12. Descriptions of the processes other than steps S23c and S23d will be omitted. However, when step S23a is repeated, the acquisition unit 120 acquires an abnormal part determination threshold calculated based on the class classification threshold.

[0054] (Step S23c) The detection unit 170 deletes the similarity corresponding to the image including the abnormal portion. (Step S23d) The detection unit 170 determines whether or not the process has been repeated a predetermined number of times. If the process has been repeated a predetermined number of times, the process proceeds to step S25. If the process has not been repeated a predetermined number of times, the process proceeds to step S14a.

[0055] In step S23c, the detection unit 170 may not delete the similarity corresponding to the image including the abnormal portion. If the similarity is not deleted, in step S14a, the calculation unit 140 performs normalization by excluding the similarity corresponding to the image including the abnormal portion from among the multiple similarities.

[0056] In this way, when an abnormality is detected and then a process for detecting another abnormality is performed, and multiple similarities are normalized, the similarity corresponding to the image containing the abnormality is excluded from the multiple similarities.

[0057] According to the third embodiment, the information processing device 100 can detect a small abnormality even when a large abnormality exists nearby.

[0058] Embodiment 4 Next, a fourth embodiment will be described. In the fourth embodiment, differences from the first and second embodiments will be mainly described. In the fourth embodiment, descriptions of the same matters as the first and second embodiments will be omitted.

[0059] Metal fittings and the like are attached to the twisted wire. In the method of the second embodiment, there is a possibility that a location where a metal fitting is attached may be mistakenly determined as an abnormal location. Also, for example, in the method of the second embodiment, there is a possibility that a location where a shadow of another twisted wire exists may be mistakenly determined as an abnormal location. In this way, there is a possibility that a normal location may be mistakenly determined as an abnormal location. Therefore, in the fourth embodiment, a case will be described in which a normal location is prevented from being mistakenly detected as an abnormal location.

[0060] Fig. 13 is a flowchart showing an example of processing executed by the information processing device of the fourth embodiment. The processing in Fig. 13 differs from the processing in Fig. 9 in that steps S23e and 23f are executed. Therefore, steps S23e and 23f will be explained in Fig. 13. Explanations of the processing other than steps S23e and 23f will be omitted. Also, the abnormality detected in step S23b may be considered as a provisionally detected abnormality.

[0061] (Step S23e) The acquiring unit 120 acquires the trained model. For example, the acquiring unit 120 acquires the trained model from the storage unit 110 or an external device. Note that the trained model is generated by using an image including a normal part (for example, a metal fitting) as training data. (Step S23f) Detection unit 170 uses the image including the abnormal portion and the trained model to determine whether or not the portion is abnormal. In other words, detection unit 170 uses the image including the provisionally detected abnormal portion and the trained model to determine whether or not the portion is abnormal. In detail, detection unit 170 inputs an image including an abnormal portion to the trained model, and the trained model outputs information indicating whether or not the portion indicated by the input image is an abnormal portion. If the portion indicated by the input image is not an abnormal portion (i.e., if the portion indicated by the input image is a normal portion), detection unit 170 detects the portion indicated by the input image as a normal portion.

[0062] Furthermore, when multiple abnormal locations are detected, the detection unit 170 uses the multiple images and the trained model to determine whether or not the locations indicated by each of the multiple images are abnormal locations. The detection unit 170 excludes normal locations from the multiple detected abnormal locations. As a result, when multiple abnormal locations are detected in step S23b, a result in which the normal locations have been excluded is output in step S25. In other words, only the original abnormal locations are output as a result.

[0063] According to the fourth embodiment, the information processing device 100 can prevent a normal portion from being erroneously detected as an abnormal portion. Furthermore, there may be an increase in the number of parts, etc., that are to be excluded as normal portions. In such a case, the trained model is retrained using an image that includes the parts, etc., that are to be excluded. Then, by using the retrained trained model, the information processing device 100 can deal with the case where the number of parts, etc., that are to be excluded increases.

[0064] A variation of the fourth embodiment. Next, a description will be given of a variation of embodiment 4. In the variation of embodiment 4, a case in which embodiment 3 and embodiment 4 are combined will be described. Fig. 14 is a flowchart showing an example of processing executed by an information processing device according to a modification of the fourth embodiment. The processing in Fig. 14 differs from the processing in Fig. 12 in that steps S23e and 23f are executed. The processing contents of steps S23e and 23f are the same as those described above. Therefore, the description of the processing of steps S23e and 23f will be omitted.

[0065] According to the modification of the fourth embodiment, the information processing device 100 can prevent a normal portion from being erroneously detected as an abnormal portion.

[0066] The features of each of the embodiments described above can be combined with each other as appropriate. [Explanation of symbols]

[0067] 10 image, 11 image, 12 image, 13 image, 14 image, 20 image, 21 abnormal area, 22 abnormal area, 100 information processing device, 101 processor, 102 volatile storage device, 103 non-volatile storage device, 110 storage unit, 120 acquisition unit, 130 division unit, 140 calculation unit, 150 judgment unit, 160 output unit, 170 detection unit.

Claims

1. An acquisition unit for acquiring an image of the stranded wire; A division unit that divides the image of the stranded wire; a calculation unit that calculates a plurality of similarities using a plurality of target images set from the plurality of images obtained by the division and a plurality of comparison images set from the plurality of images, normalizes the plurality of similarities, calculates a classification threshold using a plurality of values ​​obtained by the normalization, calculates one value for each class obtained by classifying the plurality of values ​​based on the classification threshold, and calculates a difference between the two calculated values ​​as an inter-class distance; a determination unit that determines that the twisted wire is abnormal when the inter-class distance is equal to or greater than a predetermined first threshold; An information processing device having the above configuration.

2. The determination unit determines that the twisted wire is normal when the inter-class distance is smaller than the first threshold value. The information processing device according to claim 1 .

3. Further comprising a detection unit, The acquisition unit acquires a second threshold value; The detection unit detects a value equal to or greater than the second threshold from among the plurality of values ​​obtained by normalization, and detects a portion indicated by an image corresponding to the detected value as an abnormal portion.

3. The information processing device according to claim 1 or 2.

4. In a case where a process for detecting another abnormality is executed after the abnormality is detected, and the plurality of similarities are normalized, a similarity corresponding to an image including the abnormality is excluded from the plurality of similarities; The acquisition unit acquires the second threshold value calculated based on the class classification threshold value when the detection process of another abnormal portion is executed. The information processing device according to claim 3 .

5. The acquisition unit acquires a trained model, The detection unit uses an image including the abnormal portion and the trained model to determine whether or not a portion indicated by the image is the abnormal portion, and if the portion indicated by the image is not an abnormal portion, detects the portion indicated by the image as a normal portion. The information processing device according to claim 3 .

6. Further, an output unit for outputting the result is provided.

3. The information processing device according to claim 1 or 2.

7. An information processing device, Acquire an image of the stranded wire, Segmenting the image of the stranded wire; Calculating a plurality of similarities using a plurality of target images set from the plurality of images obtained by the division and a plurality of comparison images set from the plurality of images; normalizing the plurality of similarities; A classification threshold is calculated using the multiple values ​​obtained by normalization; calculating one value for each class obtained by classifying the plurality of values ​​based on the classification threshold; The difference between the two calculated values ​​is calculated as the inter-class distance. If the inter-class distance is equal to or greater than a predetermined first threshold, the twisted wire is determined to be abnormal. Detection methods.

8. In the information processing device, Acquire an image of the stranded wire, Segmenting the image of the stranded wire; Calculating a plurality of similarities using a plurality of target images set from the plurality of images obtained by the division and a plurality of comparison images set from the plurality of images; normalizing the plurality of similarities; A classification threshold is calculated using the multiple values ​​obtained by normalization; calculating one value for each class obtained by classifying the plurality of values ​​based on the classification threshold; The difference between the two calculated values ​​is calculated as the inter-class distance. If the inter-class distance is equal to or greater than a predetermined first threshold, the twisted wire is determined to be abnormal. The detection program that causes the process to run.

Images