Program, information processing method, and information processing device

Abstract

To provide a program, etc. with which it is possible to efficiently perform determination regarding the reuse of insulators.SOLUTION: The program causes a computer to execute processing of: acquiring an infrared image in which an insulator recovered from a power distribution facility is captured using an infrared illumination and an infrared camera; inputting the acquired infrared image including the insulator to a learning model having been trained to output information on an appearance state of the insulator when an infrared image including the insulator is inputted, so as to acquire, from the learning model, the information on the appearance state of the insulator; deriving information on reuse of the insulator on the basis of the acquired information on the appearance state of the insulator; and outputting the derived information on reuse of the insulator.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a program, an information processing method, and an information processing device. [Background technology]

[0002] For example, Patent Document 1 describes matters related to electrical equipment such as transformers, cables, circuit breakers, and capacitors, and further refers to a method for evaluating deterioration of electrical insulating oil used in these electrical equipment. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2022-65858 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the degradation evaluation method disclosed in Patent Document 1, no consideration is given to efficiently making a decision regarding the reuse of insulators.

[0005] The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a program or the like that can efficiently perform a determination regarding the reuse of insulators. [Means for solving the problem]

[0006] A program according to one aspect of the present disclosure causes a computer to execute a process of acquiring infrared images of insulators recovered from power transmission and distribution equipment using infrared lighting and an infrared camera, inputting the acquired infrared images including insulators into a learning model trained to output information regarding the external condition of the insulators when an infrared image including insulators is input, acquiring information regarding the external condition of the insulators from the learning model, deriving information regarding the reuse of the insulators based on the acquired information regarding the external condition of the insulators, and outputting the derived information regarding the reuse of the insulators.

[0007] In this embodiment, for example, after a power transmission and distribution facility such as a power transmission line, a power distribution line, or a pole-mounted transformer is used, the insulators used in the power transmission and distribution facility are removed and collected. That is, it is assumed that the insulators on the poles are periodically taken down and their appearances are inspected to determine their deterioration state. The collected (taken down) insulators are imaged by an infrared camera (infrared imaging device) under infrared illumination using an infrared light source having a wavelength of, for example, 830 nm. The infrared light source is, for example, a light-emitting diode (infrared LED) that emits near-infrared light from about 700 nm to 1500 nm. By imaging the insulators using near-infrared light having a longer wavelength than visible light, it is possible to extract and photograph relatively deep scratches in the external state of the insulator. By inputting an image (infrared image) in which the relatively deep scratches are extracted in this way into a learning model (infrared model), it is possible to improve the inference accuracy of the learning model, i.e., the accuracy of detecting the presence or absence of the deep scratches on the outer surface or outer periphery of the insulator. In the photographing environment, the insulator may be surrounded by light shielding objects and placed on a monochromatic floor surface (green cloth) of green or the like, and the insulator may be photographed. Furthermore, a polarizing filter may be attached to the infrared camera to suppress (prevent) disturbance or halation. The control unit of the information processing device may preprocess the acquired image including the insulator, for example, by keying the background of the insulator, to make the background monochromatic and make it easier to extract the insulator. In this case, the preprocessed image is input to the learning model. By performing the keying process or the like in this way, the background of the insulator can be substantially removed, and the inference accuracy by the learning model can be improved. The learning model may be configured, for example, by R-CNN (Region CNN), FastR-CNN, FasterR-CNN, YOLO (You Only Look Once), or SSD (Single Shot MultiBox Detector). Alternatively, for example, an artificial intelligence chatbot such as ChatGPT may be used to perform object detection on an image including an insulator (outputting information regarding the external appearance of the insulator).In this case, the image including the insulator may be stored in a storage server that can be specified by a URL on the Internet that ChatGPT can access in advance, and the image including the insulator may be input to ChatGPT by specifying the URL in a prompt of ChatGPT. In this case, ChatGPT functions as a learning model that outputs information about the appearance state of the insulator when an image including the insulator is input. The control unit of the information processing device inputs an infrared image captured by extracting a relatively deep scratch into the learning model (infrared model), and obtains information about the appearance state of the insulator output by the learning model (infrared model). The control unit of the information processing device derives information about the reuse of the insulator based on the obtained information about the appearance state of the insulator, and therefore accurately determines whether the insulator can be reused depending on the presence or absence of a relatively deep scratch, that is, it is possible to efficiently determine whether the insulator can be reused, promote the reuse of the insulator, and provide an insulator suitable for promoting the SDGs.

[0008] In one aspect of the present disclosure, the program derives that information regarding the external condition of the insulator includes the presence or absence of scratches on the outer surface of the insulator, and if there are scratches on the outer surface of the insulator, it derives that the insulator cannot be reused, and if there are no scratches on the outer surface of the insulator, it derives that the insulator can be reused.

[0009] In this embodiment, the learning model (infrared model) outputs information on the presence or absence of scratches on the outer surface or outer periphery of the insulator captured by the infrared camera based on the input infrared image. The learning model (infrared model) may output a determination flag value indicating the presence or absence of scratches, the number of scratches on the outer surface of the insulator, or the area of ​​scratches on the outer surface of the insulator, as the information on the presence or absence of scratches. The control unit of the information processing device determines whether or not the insulator can be reused (recyclability) based on the information on the presence or absence of scratches acquired from the learning model (infrared model). When the information on the presence or absence of scratches (information on the appearance state of the insulator) output by the learning model (infrared model) indicates that there are scratches on the outer surface of the insulator, the control unit of the information processing device determines (derives) that the insulator cannot be reused (cannot be recycled) when the information on the presence or absence of scratches (information on the appearance state of the insulator) output by the learning model (infrared model) indicates that there are no scratches on the outer surface of the insulator, the control unit of the information processing device determines (derives) that the insulator can be reused (recyclable). That is, when it is determined that there is one or more scratches on the outer surface or outer periphery of the insulator based on the information on the presence or absence of scratches (information on the appearance state of the insulator) output by the learning model (infrared model), the control unit of the information processing device determines (derives) that the insulator cannot be reused (cannot be regenerated). In the infrared image of the insulator input to the learning model (infrared model), only relatively deep scratches that can be detected by the learning model (infrared model) are included by using infrared rays with a longer wavelength than visible light. In other words, even if there is a scratch on the outer surface of the insulator, shallow scratches that cannot be clearly imaged by infrared rays will not be included in the infrared image. Or, even if the shallow scratch is included in the infrared image, the learning model (infrared model) is trained not to detect (cannot detect) shallow scratches included in the input infrared image. Therefore, when an infrared image containing shallow scratches that do not affect the reuse of the insulator (damage that is acceptable for reuse) is input, the learning model (infrared model) will not detect scratches in the infrared image (it will output information indicating that no scratches were detected).In this way, the infrared image contains only relatively deep scratches, i.e., scratches that render the product non-recyclable (damage that is unacceptable for reuse), at a level that can be detected by the learning model (infrared model), making it possible to efficiently determine whether or not the product contains scratches that render it non-recyclable.

[0010] In one aspect of the program of the present disclosure, the information regarding the external condition of the insulator includes information regarding scratches on an outer surface of the insulator, and the wavelength of the infrared illumination is determined based on the depth of the scratches in response to a determination of whether the insulator can be reused.

[0011] In this embodiment, the reusability of the insulator is determined according to the depth of the damage present on the outer surface or the outer periphery of the insulator. The damage present on the outer surface of the insulator includes, for example, cracks, chips, lightning strike marks, and scratches. That is, if the depth of the damage is equal to or greater than a predetermined threshold, the reusability of the insulator is confirmed (recyclable), and if the depth of the damage is less than the predetermined threshold, the reusability of the insulator is denied (unrecyclable). For damages of different depths, the longer the wavelength of the infrared light used to capture the infrared image, the greater the depth of the damage extracted (imaged), and therefore, damages with small depths (shallow damages) are not extracted (imaged). The wavelength of the infrared illumination used to capture the infrared image of the insulator is determined based on the depth of the damage according to the judgment of the reusability of the insulator, that is, a wavelength (e.g., 830 nm) corresponding to the threshold depth of damage that is not reusable (damage that is not acceptable for reuse) is set. By setting the wavelength of infrared rays when capturing an infrared image in this manner, an infrared image that efficiently extracts scratches that are not recyclable (scratches that are not acceptable for reuse) can be input to a learning model (infrared model), and the detection accuracy of the learning model (infrared model) can be ensured. When setting the wavelength of infrared rays when capturing an infrared image, the control unit of the information processing device may acquire the product specifications, model, or type of the insulator to be judged for reuse, and determine the wavelength of infrared rays based on the acquired product specifications, etc. Alternatively, the control unit of the information processing device may acquire a threshold value of the depth of scratches that are not recyclable (scratches that are not acceptable for reuse) in the insulator to be judged for reuse (a value indicating that scratches deeper than the threshold value are not recyclable), and determine the wavelength of infrared rays based on the acquired threshold value. In this case, the storage unit of the information processing device may store information associating the product specifications of the insulator with the infrared wavelength, or information associating the threshold value of the flaw depth with the infrared wavelength, for example in a table format (infrared wavelength table), and the control unit of the information processing device may determine (set) a suitable infrared wavelength by referring to the infrared wavelength table. The control unit of the information processing device may set the wavelength of infrared light emitted by the infrared lighting according to the infrared wavelength thus determined.The infrared illuminator irradiates infrared light at a set wavelength in response to an instruction signal (a signal including information related to the wavelength of infrared light) from the information processing device.

[0012] A program according to one embodiment of the present disclosure acquires visible light images of the insulators recovered from the power transmission and distribution equipment using visible light lighting and a visible light camera, and inputs the acquired visible light images including the insulators into a second learning model that is trained to output information regarding the external state of the insulators when a visible light image including the insulators is input, thereby acquiring information regarding the external state of the insulators from the second learning model, and deriving information regarding the reuse of the insulators based on the information acquired from the learning model and the information acquired from the second learning model.

[0013] In this embodiment, the control unit of the information processing device acquires an infrared image of the insulator captured by an infrared camera, as well as a visible light image of the insulator (the same insulator) captured by using a visible light illuminator and a visible light camera. The control unit of the information processing device inputs the infrared image to a learning model (infrared model) and inputs the visible light image to a second learning model (visible light model). The second learning model (visible light model) is trained to output information regarding the external state of the insulator when a visible light image including the insulator is input. The control unit of the information processing device derives information regarding the reuse of the insulator based on information acquired from the learning model (infrared model) (information regarding the external state of the insulator from the infrared image) and information acquired from the second learning model (visible light model) (information regarding the external state of the insulator from the visible light image). The control unit of the information processing device may derive (make a final judgment) that the insulator can be reused (recyclable) only when the judgment result (intermediate judgment result) based on information acquired from the learning model (infrared model) and the judgment result (intermediate judgment result) based on information acquired from the second learning model (visible light model) both affirm the reuse of the insulator. Since the control unit of the information processing device derives information regarding the reuse of the insulator based on these two pieces of information for the same insulator, the accuracy of the judgment regarding the reuse of the insulator can be further improved.

[0014] In the program according to an embodiment of the present disclosure, the information on the appearance state of the insulator obtained from the second learning model includes the presence or absence of discoloration on the outer surface of the insulator.

[0015] In this embodiment, the second learning model (visible light model) extracts color changes such as discoloration or coloring on the outer surface or outer periphery of the insulator based on the input visible light image of the insulator (extraction of different color areas from the surface paint of the insulator). As a result, information on the appearance state of the insulator obtained from the second learning model includes the presence or absence of discoloration on the outer surface of the insulator. In other words, aspects that are relatively easy to detect in a visible light environment are handled (detected) by the second learning model (visible light model), and aspects that are relatively easy to detect in an infrared environment are handled (detected) by the learning model (infrared model). This makes it possible to make a multifaceted judgment regarding the reusability of the insulator.

[0016] An information processing method according to one aspect of the present disclosure causes a computer to execute a process of acquiring an infrared image of an insulator recovered from a power transmission and distribution equipment using infrared lighting and an infrared camera, inputting the acquired infrared image including the insulator into a learning model trained to output information regarding the external condition of the insulator when an infrared image including the insulator is input, acquiring information regarding the external condition of the insulator from the learning model, deriving information regarding the reuse of the insulator based on the acquired information regarding the external condition of the insulator, and outputting the derived information regarding the reuse of the insulator.

[0017] In this aspect, it is possible to provide an information processing method that can efficiently determine whether or not to reuse insulators.

[0018] An information processing device according to one embodiment of the present disclosure includes an image acquisition unit that acquires infrared images of insulators recovered from power transmission and distribution equipment using infrared lighting and an infrared camera, an information acquisition unit that acquires information regarding the external state of the insulators from a learning model trained to output information regarding the external state of the insulators when an infrared image including the insulators is input, by inputting the acquired infrared image including the insulators into the learning model, a derivation unit that derives information regarding the reuse of the insulators based on the acquired information regarding the external state of the insulators, and an output unit that outputs the derived information regarding the reuse of the insulators.

[0019] In this aspect, it is possible to provide an information processing device that can efficiently make a decision regarding the reuse of insulators. Effect of the Invention

[0020] It is possible to provide a program or the like that efficiently determines whether or not insulators can be reused. [Brief description of the drawings]

[0021] [Figure 1] FIG. 1 is a schematic diagram for explaining an overview of an insulator determination system according to a first embodiment. [Diagram 2] FIG. 1 is a block diagram showing a configuration of an information processing device. [Diagram 3] FIG. 1 is an explanatory diagram regarding a generation process of a learning model. [Figure 4] FIG. 1 is an explanatory diagram relating to a comparison between a visible light image and an infrared image. [Diagram 5] 11 is a flowchart illustrating a processing procedure (during learning) by a control unit of the information processing device. [Figure 6] 2 is a functional block diagram illustrating functional units included in a control unit of the information processing device. FIG. [Figure 7] 11 is a flowchart illustrating a processing procedure (during operation) by a control unit of the information processing device. [Figure 8] FIG. 11 is a functional block diagram illustrating functional units included in a control unit of an information processing device according to embodiment 2 (second learning model). [Figure 9] 10 is a flowchart illustrating a processing procedure performed by a control unit of an information processing device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0022] (Embodiment 1) Hereinafter, an embodiment will be described with reference to the drawings. Fig. 1 is a schematic diagram for explaining an overview of an insulator determination system S according to the first embodiment. Fig. 2 is a block diagram showing the configuration of an information processing device 1. The insulator determination system S is configured with the information processing device 1 as a main device, and the information processing device 1 acquires an infrared image of an insulator G captured by an imaging device such as an infrared camera C1. After being used in a power transmission and distribution facility D such as a power transmission line, a power distribution line, or a pole-mounted transformer, the insulator G is periodically taken down from a pole and collected, and the collected insulator G is imaged by the infrared camera C1.

[0023] The infrared camera C1 may be configured as an infrared imaging device together with an infrared light source made of a light emitting diode (infrared LED) that emits near infrared rays from about 700 nm to 1500 nm. The infrared camera C1 may be, for example, a camera built into or externally attached to an information terminal T such as a smartphone held by a maintenance worker of the power transmission and distribution facility D, or may be connected via the input / output IF14 of the information processing device 1.

[0024] An information terminal T of a maintenance worker or the like and the information processing device 1 are communicatively connected via an external network such as the Internet, a LAN, or the like. An infrared image of the insulator G captured by the infrared camera C1 is acquired by the information processing device 1 by transmitting it from the information terminal T of the maintenance worker or the like, acquiring it via the input / output IF 14 of the information processing device 1, or acquiring it offline via a storage medium such as a memory card, or the like. As shown in the figure in this embodiment, in addition to the infrared camera C1, a visible light camera C2 that captures a visible light image may be connected to the information processing device 1. The visible light camera C2 will be described later in embodiment 2.

[0025] The information processing device 1 is a computer capable of various information processing and sending and receiving information, such as a server device, a personal computer, etc. The server device includes not only a single server device, but also a cloud server device or a virtual server device configured by multiple computers. The information processing device 1 includes a control unit 11, a storage unit 12, a communication unit 13, and an input / output IF 14.

[0026] The control unit 11 has one or more central processing units (CPUs), micro-processing units (MPUs), graphics processing units (GPUs), and other processing devices with timing functions, and performs various information processing, control processing, and the like by reading and executing a program P (program product) stored in the memory unit 12.

[0027] The storage unit 12 includes a volatile storage area such as a static random access memory (SRAM), a dynamic random access memory (DRAM), or a flash memory, and a non-volatile storage area such as an EEPROM or a hard disk. The storage unit 12 stores in advance a program P (program product) and data to be referenced during processing, and further stores various data including intermediate data generated during processing. The program P (program product) stored in the storage unit 12 may be a program P (program product) read from a recording medium M readable by the information processing device 1. Alternatively, the program P (program product) may be downloaded from an external computer (not shown) connected to a communication network (not shown) and stored in the storage unit 12.

[0028] The communication unit 13 is a communication module or a communication interface for communicating with an information terminal T, etc., by wired communication such as Ethernet (registered trademark) or wirelessly, and is, for example, a short-range wireless communication module such as Wi-Fi (registered trademark) or Bluetooth (registered trademark), or a wide-area wireless communication module such as 4G or 5G. The control unit 11 communicates with an information terminal T, such as a smartphone held by a maintenance worker of the power transmission and distribution facility D, via the communication unit 13, for example, through an external network such as the Internet or a LAN.

[0029] 3 is an explanatory diagram regarding the generation process of the learning model 101. The control unit 11 of the information processing device 1 uses training data for the learning model 101 (infrared model) to train a neural network such as R-CNN, and generates the learning model 101 that receives an infrared image including an insulator G as an input and outputs information regarding the appearance state of the insulator G.

[0030] The infrared image including the insulator G may be an infrared image in which the background of the insulator G has been keyed and monochromatized or deleted as a pre-processing, an image in which the size has been adjusted (enlarged or reduced to a predetermined size, etc.), etc. The information on the external appearance state of the insulator G output by the learning model 101 may include, for example, the presence or absence of scratches (damaged parts) due to cracks, chips, lightning strike marks, or scratches on the outer surface or outer periphery of the insulator G, and if scratches (damaged parts) are present, the area or size of the scratches (damaged parts).

[0031] Alternatively, the information on the external appearance of the insulator G output by the learning model 101 may be information indicating whether the insulator G can be reused or not depending on the presence or absence of scratches (damage). In this case, the learning model 101 may output that reuse is not possible if there is one or more scratches (damage), and output that reuse is possible if there is no scratch (damage). In this way, the infrared image is captured by an infrared imaging device (infrared lighting, infrared camera C1) with an infrared wavelength set so as to include only relatively deep scratches, i.e., scratches that make the insulator unrecoverable (damage that is not acceptable for reuse), at a level that can be detected by the learning model 101 (infrared model).

[0032] 4 is an explanatory diagram for comparing a visible light image and an infrared image. In the illustration of this embodiment, a visible light image (visible light photography) and an infrared image (infrared photography) of the same part of the same insulator G are shown. The infrared image (infrared photography) was taken by an infrared imaging device including an infrared illuminator and an infrared camera C1 under a condition in which the infrared wavelength was set based on the depth of the flaw corresponding to the judgment of whether the insulator G can be reused. The visible light image (visible light photography) was taken by a visible light imaging device including a visible light illuminator and a visible light camera C2.

[0033] The portion of the insulator G (the portion of the outer surface or outer periphery of the insulator G) to be imaged (subject) has a relatively deep scratch and further includes a discolored area. In the visible light image (visible light photography), only the discolored area is emphasized and extracted, and the deep scratch lacks clarity, resulting in low visibility for maintenance workers of the power transmission and distribution equipment D. In contrast, in the infrared image (infrared photography), the discolored area is not extracted, and the deep scratch is emphasized and extracted, that is, the white part indicating the deep scratch is included in the infrared image. By using the infrared image in which only the deep scratch is thus imaged (white part) for the learning model 101 (infrared model), the deep scratch, that is, the scratch that is not recyclable (the scratch that is not acceptable for reuse), can be efficiently detected.

[0034] The training data includes problem data and answer data, an infrared image including the insulator G corresponds to the problem data, and information on the external appearance of the insulator G corresponds to the answer data. The information on the external appearance of the insulator G, which is the answer data, may be set by annotating (adding) an area of ​​damage (damaged part) due to cracks, chips, lightning strike marks, or scratches on the outer surface of the insulator G to an infrared image including the insulator G. The data set of problem data and answer data included in the training data for learning the learning model 101 and the data set of input data and output data when the learning model 101 is used are synonymous, and if it is defined in either data set, it naturally applies to the other data set.

[0035] The neural network (learning model 101) trained using the training data is expected to be used as a program module that is a part of artificial intelligence software. The learning model 101 is used in the information processing device 1 equipped with the control unit 11 (CPU, etc.) and the storage unit 12 as described above, and a neural network system is configured by being executed in the information processing device 1 having such calculation processing capabilities. That is, the control unit 11 of the information processing device 1 performs calculations to extract features of the infrared image including the insulator G input to the input layer in accordance with instructions from the learning model 101 stored in the storage unit 12, and outputs information on the appearance state of the insulator G from the output layer.

[0036] The learning model 101 is configured, for example, by R-CNN (Region Convolutional Neural Network) or YOLO, and has an input layer that accepts an input of an infrared image including an insulator G, an intermediate layer that extracts features of the infrared image including the insulator G, and an output layer that outputs information on the appearance state of the insulator G. The input layer has a plurality of neurons that accept an input of an infrared image including the insulator G, and passes the input values ​​to the intermediate layer. The intermediate layer is defined using an activation function such as a ReLU function or a sigmoid function, has a plurality of neurons that extract features of each input value, and passes the extracted features to the output layer. Parameters such as the weighting coefficient and bias value of the activation function are optimized using an error backpropagation method. The output layer is configured, for example, by a fully connected layer, and outputs information on the appearance state of the insulator G based on the features output from the intermediate layer.

[0037] In this embodiment, the learning model 101 is R-CNN or the like, but is not limited thereto, and may be a learning model 101 constructed by other machine learning algorithms such as neural networks other than R-CNN, transformers, BERT, GPT, RNN (Recurrent Neural Network), LSTM (Long-short term model), SVM (Support Vector Machine), Bayesian network, linear regression, regression tree, multiple regression, random forest, ensemble, etc. Alternatively, the learning model 101 may be configured using an artificial intelligence chatbot such as ChatGPT. In this case, ChatGPT may be fine-tuned or the like so as to efficiently output information on the appearance state of the insulator G. Alternatively, a question generated using an external database such as WebDB may be input together with an infrared image including the insulator G to a prompt, which is an input interface of ChatGPT.

[0038] 5 is a flowchart illustrating a processing procedure (during learning) by the control unit 11 of the information processing device 1. The control unit 11 of the information processing device 1 receives an operation by an operator using, for example, a keyboard connected to the input / output I / F 14, and performs the following processing based on the received operation.

[0039] The control unit 11 of the information processing device 1 acquires an infrared image including the insulator G (S1). The control unit 11 of the information processing device 1 acquires the infrared image including the insulator G from an infrared camera C1 (infrared imaging device). When capturing the infrared image, the insulator G is captured by the infrared camera C1 to which a polarizing filter is attached. This makes it possible to suppress (prevent) disturbance or halation, and improves the accuracy of inference (estimation) by the learning model 101.

[0040] The control unit 11 of the information processing device 1 may set the wavelength of the infrared light in the infrared illumination to a wavelength (e.g., 830 nm) corresponding to the threshold depth of scratches that are not recyclable (scratches that are not acceptable for reuse). When the threshold depth differs depending on the product specifications, model, or type of the insulator G to be judged for reuse, the control unit 11 of the information processing device 1 may acquire the threshold and determine the wavelength of the infrared light based on the acquired threshold. In this case, the storage unit 12 of the information processing device 1 may store information in which the product specifications of the insulator G and the wavelength of the infrared light are associated with each other, or information in which the threshold depth of scratches and the wavelength of the infrared light are associated with each other, for example, in a table format (infrared wavelength table). The control unit 11 of the information processing device 1 may determine (set) a suitable wavelength of the infrared light by referring to the infrared wavelength table.

[0041] The control unit 11 of the information processing device 1 performs annotation on the infrared image including the insulator G (S2). Areas of damage (damaged parts) due to cracks, chips, lightning strike marks, scratches, etc. on the outer surface or outer periphery of the insulator G are annotated to the infrared image including the insulator G according to the appearance state of the insulator G. In this case, the infrared image including the insulator G corresponds to question data, and the annotated areas and types of damaged parts correspond to answer data.

[0042] The control unit 11 of the information processing device 1 generates a learning dataset (S3). The control unit 11 of the information processing device 1 generates a learning dataset (training data) by combining a plurality of infrared images, which are problem data, with answer data consisting of areas of scratches (damaged parts) annotated on each of these infrared images. The plurality of infrared images, which are problem data, are, for example, infrared images including insulators G that can be reused and insulators G that cannot be reused. A large number of such infrared images of insulators G in various appearance states are stored by equipment companies and the like that are responsible for the maintenance and management of power transmission and distribution equipment D, and by using these, the learning dataset (training data) can be generated.

[0043] The control unit 11 of the information processing device 1 generates a learning model 101 using the generated learning dataset (S4). The control unit 11 of the information processing device 1 applies the learning dataset (training data) to a plurality of different types of neural networks, such as R-CNN and YOLO, and causes them to learn, thereby generating the learning model 101 (infrared model).

[0044] 6 is a functional block diagram illustrating functional units included in the control unit 11 of the information processing device 1. The control unit 11 of the information processing device 1 executes a program P stored in the storage unit 12 to function as an acquisition unit 111, a judgment result derivation unit 112, and an output unit 113. The control unit 11 of the information processing device 1 functions as the learning model 101 by reading an entity file of the learning model 101 stored in the storage unit 12, etc.

[0045] The acquisition unit 111 acquires an infrared image including the insulator G captured by the infrared camera C1, and functions as an image acquisition unit. The infrared image is captured by an infrared imaging device including an infrared light and an infrared camera C1, with the infrared wavelength set so as to include only relatively deep scratches that cannot be restored (scratches that are not acceptable for reuse) at a level that can be detected by the learning model 101 (infrared model). The insulator G collected from the power transmission and distribution equipment D is managed in association with the type of the insulator G (insulator G type) along with various collection information such as the model of the insulator G that is the power transmission and distribution equipment D from which it was collected, the location of the power transmission and distribution equipment D, the period of use, and the collection date. The collection information may be stored in the storage unit 12 of the information processing device 1 by being stored and managed in a collection information table. The insulator G collected from the power transmission and distribution facility D, i.e., the insulator G that has been lowered from the pole, is surrounded by light-shielding objects and placed on a monochromatic floor surface (green cloth) of green or the like under an infrared light source (infrared lighting) composed of light-emitting diodes (infrared LEDs) that emit near-infrared rays of, for example, about 700 nm to 1500 nm, and is then photographed by the infrared camera C1. The infrared camera C1 may be equipped with a polarizing filter.

[0046] The infrared camera C1 is connected to, for example, the input / output IF 14 of the information processing device 1, and an infrared image including the insulator G is acquired from the infrared camera C1 via the input / output IF 14. Alternatively, the infrared camera C1 may be mounted on an information terminal T such as a smartphone carried by a maintenance worker or the like of the power transmission and distribution facility D, and the information processing device 1 may acquire the image including the insulator G by communicating with the information terminal T.

[0047] The acquisition unit 111 inputs the acquired infrared image to the learning model 101. The acquisition unit 111 may perform, as a pre-processing when inputting the infrared image to the learning model 101, a process such as adjusting an image area occupied by the insulator G to a predetermined size.

[0048] The learning model 101 (infrared model) outputs information on the external appearance of the insulator G based on the input infrared image. The information on the external appearance of the insulator G output by the learning model 101 (infrared model) includes, for example, the presence or absence of scratches (damaged parts) due to cracks, chips, lightning strike marks, or scratches on the outer surface or outer periphery of the insulator G, and, if scratches (damaged parts) are present, the area or size of the scratches (damaged parts). The area of ​​the scratches (damaged parts) may be identified, for example, by a bounding box superimposed on the infrared image including the insulator G.

[0049] The judgment result derivation unit 112 functions as an information acquisition unit that acquires information on the appearance state of the insulator G output (estimated) by the learning model 101, and derives a judgment result on whether the insulator G can be reused based on the information on the appearance state of the insulator G. When it is determined that there is one or more scratches on the outer surface or outer periphery of the insulator G based on the information on the presence or absence of scratches (information on the appearance state of the insulator G) output by the learning model 101 (infrared model), the judgment result derivation unit 112 determines (derives) that the insulator G cannot be reused (cannot be recycled). When it is determined that there is no one or more scratches on the outer surface or outer periphery of the insulator G based on the information on the presence or absence of scratches (information on the appearance state of the insulator G) output by the learning model 101 (infrared model), the judgment result derivation unit 112 determines (derives) that the insulator G can be reused (can be recycled).

[0050] The output unit 113 outputs the determination result from the determination result derivation unit 112, i.e., information on whether the insulator G can be reused, to an information terminal T such as a smartphone carried by a maintenance worker of the power transmission and distribution facility D. Alternatively, the output unit 113 may output the determination result to a display device such as a display connected to the input / output IF 14. The output unit 113 may output information on the appearance state of the insulator G that is the basis for the determination (the number of scratches), in addition to the determination result on whether the insulator G can be reused.

[0051] In the present embodiment, the learning model 101 estimates (outputs) information on the appearance state of the insulator G, and the judgment result derivation unit 112 judges whether the insulator G can be reused based on the information on the appearance state of the insulator G, but the present invention is not limited to this, and the learning model 101 may output whether the insulator G can be reused. That is, the information on the appearance state of the insulator G corresponds to intermediate data for judging whether the insulator G can be reused, and the learning model 101 may include the function of the judgment result derivation unit 112 and output whether the insulator G can be reused as information on the appearance state of the insulator G based on the input infrared image including the insulator G. In this case, the answer data included in the training data when learning the learning model 101 corresponds to a flag value indicating whether the infrared image of the insulator G can be reused. At this time, the flag value indicating whether the insulator G can be reused is annotated to the infrared image of the insulator G.

[0052] 7 is a flowchart illustrating a processing procedure (during operation) by the control unit 11 of the information processing device 1. The control unit 11 of the information processing device 1 receives an operation by an operator using, for example, a keyboard connected to the input / output I / F 14, and performs the following processing based on the received operation.

[0053] The control unit 11 of the information processing device 1 acquires an infrared image of the insulator G recovered from the power transmission and distribution equipment D (S101). The control unit 11 of the information processing device 1 acquires an infrared image including the insulator G captured by the infrared camera C1. The infrared image including the insulator G is captured by the infrared camera C1 under an infrared light source such as an infrared LED whose infrared wavelength is set based on the depth of the damage corresponding to the judgment of whether the insulator G can be reused.

[0054] The control unit 11 of the information processing device 1 inputs an infrared image including the insulator G to the learning model 101 (S102). It acquires information on the external appearance state of the insulator G output by the learning model 101 (S103). The control unit 11 of the information processing device 1 inputs the infrared image including the insulator G to the learning model 101 (infrared model) and thereby acquires information on the external appearance state of the insulator G from the learning model 101 (infrared model).

[0055] The learning model 101 (infrared model) outputs (estimates) information on the external appearance of the insulator G (presence or absence of scratches (damaged parts) on the outer surface or outer periphery of the insulator G due to cracks, chips, lightning strike marks, scratches, etc., and if scratches (damaged parts) are present, the area of ​​the scratches (damaged parts), etc.) based on the input infrared image of the insulator G. The control unit 11 of the information processing device 1 obtains from the learning model 101 (infrared model) information on the external appearance of the insulator G, such as the presence or absence of scratches (damaged parts) on the outer surface or outer periphery of the insulator G due to cracks, chips, lightning strike marks, scratches, etc., and if scratches (damaged parts) are present, the area of ​​the scratches (damaged parts).

[0056] The control unit 11 of the information processing device 1 derives information regarding the reuse of the insulator G (performs a judgment as to whether or not reuse is possible) based on the acquired information regarding the external appearance state of the insulator G (S104). When the information regarding the presence or absence of scratches (information regarding the external appearance state of the insulator G) output by the learning model 101 (infrared model) indicates that there are one or more scratches on the outer surface of the insulator G, the control unit 11 of the information processing device 1 determines (derives) that the insulator G cannot be reused (cannot be recycled). When the information regarding the presence or absence of scratches (information regarding the external appearance state of the insulator G) output by the learning model 101 (infrared model) indicates that there are not one or more scratches on the outer surface of the insulator G, the control unit 11 of the information processing device 1 determines (derives) that the insulator G can be reused (can be recycled).

[0057] The infrared image input to the learning model 101 (infrared model) contains only relatively deep scratches, i.e. scratches that are not acceptable for reuse (damage that is not acceptable for reuse), at a level that can be detected by the learning model 101 (infrared model). Therefore, if there is even one scratch detected by the learning model 101 (infrared model), it is determined that the insulator G cannot be reused (cannot be reused), so that a determination regarding the reuse of the insulator G can be made efficiently.

[0058] The control unit 11 of the information processing device 1 outputs the derived information on the reuse of the insulator G (S105). The control unit 11 of the information processing device 1 may generate screen data (judgment result screen) including a table listing information on the reuse of the insulator G, such as the judgment result of whether the reuse is possible and information on the appearance state of the insulator G output by the learning model 101 (infrared model) that is the basis for the judgment. The control unit 11 of the information processing device 1 outputs the derived information on the reuse of the insulator G, such as the generated screen data (judgment result screen), to a display device such as a display or a terminal device such as a smartphone. The display device or terminal device that has acquired the screen data (judgment result screen) from the information processing device 1 displays information on the judgment result based on the screen data (judgment result screen).

[0059] (Embodiment 2) 8 is a functional block diagram illustrating functional units included in the control unit 11 of the information processing device 1 according to the second embodiment (the second learning model 102). The control unit 11 of the information processing device 1 executes a program P stored in the storage unit 12, similarly to the first embodiment, to function as an acquisition unit 111, a judgment result derivation unit 112, and an output unit 113.

[0060] The control unit 11 of the information processing device 1 functions as the learning model 101 (infrared model) and the second learning model 102 (visible light model) by reading the entity files of the learning model 101 and the second learning model 102 stored in the storage unit 12. In this embodiment, the control unit 11 of the information processing device 1 makes a judgment regarding reuse of the insulator G using the learning model 101 (infrared model) corresponding to an infrared image and the second learning model 102 (visible light model) corresponding to a visible light image.

[0061] The second learning model 102 (visible light model) is trained by a neural network such as R-CNN in the same manner as the learning model 101 (infrared model) of the first embodiment, receives a visible light image including an insulator G as an input, and outputs information about the appearance state of the insulator G. The visible light image is captured by a visible light imaging device including a visible light camera C2 and a visible light illumination. The visible light camera C2 is, for example, a CMOS camera, and captures an image of the insulator G placed on a monochromatic floor surface (green cloth) of green or the like, surrounded by a light shielding object, under a visible light source such as a white LED. The visible light camera C2 is connected to the information processing device 1 in the same manner as the infrared camera C1 of the first embodiment. The visible light image may be, for example, an image in which the background of the insulator G is keyed and monochromatized or deleted as a preprocessing before being input to the second learning model 102 (visible light model).

[0062] The training data of the second learning model 102 (visible light model) includes question data and answer data, the visible light image including the insulator G corresponds to the question data, and the information on the external appearance of the insulator G corresponds to the answer data. The information on the external appearance of the insulator G, which is the answer data, may be set by annotating (adding) a discolored area on the outer surface of the insulator G to the visible light image including the insulator G. By learning in this manner, the type of detection target by the second learning model 102 (visible light model) (discolored area on the surface paint of the insulator G) may be made different from the type of detection target by the learning model 101 (infrared model) (scratches).

[0063] The acquisition unit 111 acquires an infrared image including the insulator G captured by the infrared camera C1, and further acquires a visible light image including the insulator G captured by the visible light camera C2. These infrared image and visible light image are images of the same part (same subject) of the insulator G. The acquisition unit 111 inputs the infrared image to the learning model 101 (infrared model), and further inputs the visible light image to a second learning model 102 (visible light model).

[0064] Similar to the first embodiment, the learning model 101 (infrared model) to which the infrared image has been input outputs, as information on the appearance state of the insulator G, the presence or absence of scratches (damaged parts) on the outer surface or outer periphery of the insulator G, and, if scratches (damaged parts) are present, the area of ​​the scratches (damaged parts), etc. The second learning model 102 (visible light model) to which the visible light image has been input outputs, as information on the appearance state of the insulator G, the presence or absence of discolored parts (areas of a different color compared to the surface paint of the insulator G) on the outer surface or outer periphery of the insulator G, and, if discolored parts are present, the area of ​​the discolored parts, etc.

[0065] The judgment result derivation unit 112 functions as an information acquisition unit that acquires information on the external appearance state of the insulator G output (estimated) by the learning model 101 (infrared model) and the second learning model 102 (visible light model), and derives a judgment result regarding whether the insulator G can be reused based on each of the pieces of information on the external appearance state of the insulator G. The judgment result derivation unit 112 makes a judgment (intermediate judgment by the learning model 101) based on information acquired from the learning model 101 (infrared model) as in the first embodiment. Furthermore, the judgment result derivation unit 112 makes a judgment (intermediate judgment by the second learning model 102) based on information acquired from the second learning model 102 (visible light model). The intermediate judgment by the learning model 101 indicates whether the insulator G can be reused depending on the presence or absence of scratches that make it impossible to recycle (damage that is not acceptable for reuse).

[0066] The intermediate judgment by the second learning model 102 indicates whether or not the insulator G can be reused depending on the presence or absence of discoloration (a different color area compared to the surface coating of the insulator G) that makes it impossible to recycle. The judgment result derivation unit 112 may determine that the insulator G cannot be reused (cannot be regenerated) when the size or range of the discoloration detected by the second learning model 102 (if there are multiple discoloration areas, the sum of the sizes of these discoloration areas) is equal to or greater than a predetermined value. In other words, the judgment result derivation unit 112 determines that the insulator G can be reused (cannot be regenerated) when the size, etc. of the discoloration area is less than a predetermined value.

[0067] The judgment result derivation unit 112 derives (makes a final judgment) that the insulator G can be reused (recyclable) only when the judgment result (intermediate judgment result) based on information acquired from the learning model 101 (infrared model) and the judgment result (intermediate judgment result) based on information acquired from the second learning model 102 (visible light model) both affirm the reuse of the insulator G. The judgment result derivation unit 112 derives (makes a final judgment) that the insulator G cannot be reused (recyclable) when at least one of the intermediate judgments by the learning model 101 and the intermediate judgments by the second learning model 102 denies the reuse of the insulator G.

[0068] As in the first embodiment, the output unit 113 outputs the determination result from the determination result derivation unit 112, i.e., information on whether the insulator G can be reused, to an information terminal T such as a smartphone carried by a maintenance worker of the power transmission and distribution equipment D. In addition to the determination result on whether the insulator G can be reused, the output unit 113 may also output information output from the learning model 101 (infrared model) and the second learning model 102 (visible light model), i.e., information on the external appearance of the insulator G that is the basis for the determination (the number of scratches, the size of the discolored area), etc.

[0069] 9 is a flowchart illustrating a processing procedure by the control unit 11 of the information processing device 1. The control unit 11 of the information processing device 1 receives an operation by an operator using, for example, a keyboard connected to the input / output I / F 14, and performs the following processing based on the received operation.

[0070] The control unit 11 of the information processing device 1 acquires an infrared image of an insulator G collected from a power transmission and distribution facility D (S201). The control unit 11 of the information processing device 1 inputs the infrared image including the insulator G to the learning model 101 (S202). The control unit 11 acquires information related to the external appearance state of the insulator G output by the learning model 101 (S203). The control unit 11 of the information processing device 1 performs the processes from S201 to S203, similar to S101 to S103 in the first embodiment.

[0071] The control unit 11 of the information processing device 1 acquires a visible light image of the insulator G collected from the power transmission and distribution facility D (S204). The control unit 11 of the information processing device 1 acquires a visible light image including the insulator G captured by a visible light camera C2 such as a CMOS camera. The visible light image including the insulator G is captured by the visible light camera C2 such as a CMOS camera under a visible light source such as a white LED, which is a light source with high color rendering properties.

[0072] The control unit 11 of the information processing device 1 inputs the visible light image including the insulator G to the second learning model 102 (S205). The control unit 11 of the information processing device 1 acquires information on the appearance state of the insulator G output by the second learning model 102 (S206). The control unit 11 of the information processing device 1 inputs the visible light image including the insulator G to the second learning model 102 (visible light model) to acquire information on the appearance state of the insulator G from the second learning model 102 (visible light model). The information (information on the appearance state of the insulator G) output from the second learning model 102 (visible light model) includes the presence or absence of a discolored area (a different color area compared to the surface paint of the insulator G) on the outer surface or outer periphery of the insulator G, and if a discolored area exists, the area of ​​the discolored area, etc.

[0073] The control unit 11 of the information processing device 1 derives information regarding the reuse of the insulator G (performs a judgment as to whether or not reuse is possible) based on the information acquired from the learning model 101 and the second learning model 102 (S207). The control unit 11 of the information processing device 1 makes a judgment (intermediate judgment by the learning model 101) based on the information acquired from the learning model 101 (infrared model) as in the first embodiment. Furthermore, the control unit 11 of the information processing device 1 makes a judgment (intermediate judgment by the second learning model 102) based on the information acquired from the second learning model 102 (visible light model). The control unit 11 of the information processing device 1 judges that regeneration is not possible (intermediate judgment) when the size or range of the discolored part detected by the second learning model 102 is equal to or greater than a predetermined value, and judges that regeneration is possible (intermediate judgment) when the size or range is less than the predetermined value.

[0074] The control unit 11 of the information processing device 1 derives (makes a final judgment) that the insulator G can be reused (recyclable) only when the intermediate judgment result by the learning model 101 (infrared model) and the intermediate judgment result by the second learning model 102 (visible light model) both affirm the reuse of the insulator G. The control unit 11 of the information processing device 1 judges (makes a final judgment) that the insulator G cannot be reused (cannot be recyclable) when at least one of the intermediate judgment result by the learning model 101 (infrared model) and the intermediate judgment result by the second learning model 102 (visible light model) denies the reuse of the insulator G.

[0075] The control unit 11 of the information processing device 1 outputs the derived information on the reuse of the insulator G (S208). The control unit 11 of the information processing device 1 performs the process of S208 in the same manner as S105 in the first embodiment.

[0076] The embodiments disclosed herein are illustrative in all respects and should not be considered as limiting. The scope of the present invention is defined by the claims, not by the above meaning, and is intended to include all modifications within the scope and meaning equivalent to the claims.

[0077] The claims may be combined with each other regardless of the form of reference. The claims include multiple dependent claims that depend on multiple dependent claims. The claims do not include multiple dependent claims that depend on a multiple dependent claim, but multiple dependent claims that depend on a multiple dependent claim may be included. [Explanation of symbols]

[0078] S insulator judgment system, D power transmission and distribution equipment, G insulator, T information terminal, C1 infrared camera, C2 visible light camera, 1 information processing device, 11 control unit, 111 acquisition unit, 112 judgment result derivation unit, 113 output unit, 12 memory unit, P program (program product), M recording medium, 13 communication unit, 14 input / output IF, 101 learning model (infrared model), 102 second learning model (visible light model)

Claims

1. On the computer, An infrared image is taken of the insulators recovered from the power transmission and distribution equipment using an infrared light and an infrared camera, inputting the acquired infrared image including the insulator into a learning model that has been trained to output information regarding the external appearance state of the insulator when the infrared image including the insulator is input, thereby acquiring information regarding the external appearance state of the insulator from the learning model; deriving information regarding reuse of the insulator based on the acquired information regarding the appearance state of the insulator; Output the derived information regarding reuse of the insulator A program that executes a process.

2. The information regarding the appearance condition of the insulator includes the presence or absence of scratches on the outer surface of the insulator, If there is a scratch on the outer surface of the insulator, it is determined that the insulator cannot be reused; If there is no damage to the outer surface of the insulator, it is determined that the insulator can be reused. The program according to claim 1.

3. The information on the appearance condition of the insulator includes information on scratches on an outer surface of the insulator, The wavelength of the infrared light is determined based on the depth of the flaw depending on whether the insulator can be reused or not. The program according to claim 1.

4. Obtaining a visible light image of the insulator recovered from the power transmission and distribution facility using a visible light illumination and a visible light camera; inputting the acquired visible light image including the insulator into a second learning model that has been trained to output information regarding the appearance state of the insulator when a visible light image including the insulator is input, thereby acquiring information regarding the appearance state of the insulator from the second learning model; Deriving information regarding reuse of the insulator based on information acquired from the learning model and information acquired from the second learning model. The program according to any one of claims 1 to 3.

5. The information on the appearance state of the insulator obtained from the second learning model includes the presence or absence of discoloration on the outer surface of the insulator. The program according to claim 4.

6. An infrared image is taken of the insulators recovered from the power transmission and distribution equipment using an infrared light and an infrared camera, inputting the acquired infrared image including the insulator into a learning model that has been trained to output information regarding the external appearance state of the insulator when the infrared image including the insulator is input, thereby acquiring information regarding the external appearance state of the insulator from the learning model; deriving information regarding reuse of the insulator based on the acquired information regarding the appearance state of the insulator; Output the derived information regarding reuse of the insulator An information processing method for causing a computer to execute processing.

7. an image acquisition unit that acquires an infrared image of the insulator collected from the power transmission and distribution facility by using an infrared light and an infrared camera; an information acquisition unit that inputs the acquired infrared image including an insulator into a learning model that has been trained to output information regarding the external appearance of the insulator when the infrared image including the insulator is input, and acquires information regarding the external appearance of the insulator from the learning model; a derivation unit that derives information regarding reuse of the insulator based on the acquired information regarding the appearance state of the insulator; an output unit that outputs the derived information regarding reuse of the insulator; An information processing device comprising:

Images