Program, information processing method and information processing device

Abstract

To provide a program, an information processing method and an information processing device that efficiently make a determination related to recycling of insulators.SOLUTION: In an insulator determination system, a program causes a computer to execute processing to: acquire an image obtained by capturing an insulator collected from power collection facilities; input the acquired image including the insulator to a plurality of respective learning models learnt to output information related to an appearance state of the insulator on inputting the image including the insulator so as to acquire the information related to the appearance state of the insulator from the plurality of learning models, respectively; derive information related to recycling of the insulator based upon the acquired information related to the appearance state of the insulator; and output the derives information related to the recycling of the insulator.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to a program, an information processing method, and an information processing device. [Background technology]

[0002] For example, Patent Document 1 describes matters related to electrical equipment such as transformers, cables, circuit breakers, and capacitors, and further refers to a method for evaluating deterioration of electrical insulating oil used in these electrical equipment. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2022-65858 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the degradation evaluation method disclosed in Patent Document 1, no consideration is given to efficiently making a decision regarding the reuse of insulators.

[0005] The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a program or the like that can efficiently perform a determination regarding the reuse of insulators. [Means for solving the problem]

[0006] A program according to one aspect of the present disclosure causes a computer to execute a process of acquiring images of insulators recovered from power transmission and distribution equipment, inputting the acquired images including insulators into each of a plurality of learning models that have been trained to output information regarding the external state of the insulators when an image including an insulator is input, acquiring information regarding the external state of the insulators from each of the plurality of learning models, deriving information regarding the reuse of the insulators based on each of the acquired information regarding the external state of the insulators, and outputting the derived information regarding the reuse of the insulators.

[0007] In this embodiment, after the power transmission and distribution equipment such as a power transmission line, a power distribution line, or a pole-mounted transformer is used, the insulators used in the power transmission and distribution equipment are removed and collected. That is, it is assumed that the insulators on the poles are periodically taken down and their deterioration state is determined by performing an appearance inspection. The collected (down) insulators are imaged by an imaging device such as a CMOS camera under a visible light source such as white light. The visible light source such as white light is, for example, a white LED, which is a light source with high color rendering close to natural light (sunlight). By using such a light source with high color rendering, it is possible to clearly photograph the appearance state, and the inference accuracy by the learning model can be improved. In the imaging environment, the insulators may be surrounded by a light shielding object and placed on a monochromatic floor surface (green cloth) of green or the like, and the insulators may be imaged. Furthermore, a polarizing filter may be attached to the imaging device (CMOS camera, etc.) to suppress (prevent) disturbance or halation. The control unit of the information processing device may perform preprocessing on the acquired image including the insulator, for example, by keying the background of the insulator to make the background monochromatic and to make it easier to extract the insulator. In this case, the preprocessed image is input to the learning model. By performing the keying process or the like in this manner, the background of the insulator can be substantially removed, and the inference accuracy by the learning model can be improved. The image including the insulator is input to a plurality of learning models. These multiple learning models are different types of object detection models. For example, when the number of multiple learning models is five, these learning models may be configured by R-CNN (Region CNN), FastR-CNN, FasterR-CNN, YOLO (You Only Look Once), and SSD (Single Shot MultiBox Detector). Alternatively, different types of learning models may be generated by learning (including transfer learning and fine tuning) the same type of object detection model using different training data. Alternatively, for example, an artificial intelligence chatbot such as ChatGPT may be used to perform object detection on an image including an insulator (outputting information regarding the external appearance of the insulator).In this case, the image including the insulator may be stored in a storage server that can be specified by a URL on the Internet that ChatGPT can access, and the image including the insulator may be input to ChatGPT by specifying the URL in a prompt of ChatGPT. In this case, ChatGPT functions as a learning model that outputs information about the appearance state of the insulator when an image including the insulator is input. The control unit of the information processing device inputs the same image (image including the insulator) to each of the different types of learning models and acquires each piece of information about the appearance state of the insulator output by each of these learning models, so that it is possible to acquire multifaceted information about the insulator (information about the appearance state). The control unit of the information processing device derives information about the reuse of the insulator based on each piece of acquired information about the appearance state of the insulator, so that, for example, compared to the case where only a single learning model is used, it is possible to accurately determine whether or not the insulator can be reused, that is, to efficiently determine whether or not the insulator can be reused.

[0008] A program according to one embodiment of the present disclosure acquires judgment criteria for determining whether the insulator can be reused, and for each piece of information regarding the acquired external appearance condition of the insulator, uses the acquired judgment criteria to determine whether the insulator can be reused, and derives the judgment result as information regarding the reuse of the insulator.

[0009] In this embodiment, the storage unit of the information processing device stores information (judgment table) on the judgment criteria for judging whether the insulators can be reused. The control unit of the information processing device judges whether the insulators can be reused by comparing the information on the judgment criteria (judgment table) with each piece of information on the appearance state of the insulators acquired from each of different types of learning models. Since the information on the judgment criteria (judgment table) determined in advance is used to judge whether the insulators can be reused, the judgment can be performed efficiently. The information on the judgment criteria (judgment table) stored in the storage unit of the information processing device may be set to be changeable. That is, the control unit of the information processing device may receive setting parameters input from an external input device such as a keyboard, and set or change the information on the judgment criteria (judgment table) according to the received setting parameters. By configuring the information on the judgment criteria (judgment table) to be changeable in this way, it is possible to ensure flexibility in judging the reusability of the collected insulators according to the type of the collected insulators or the quality standards of the power transmission and distribution equipment in which the insulators are used.

[0010] In a program according to one embodiment of the present disclosure, the judgment criteria include a criterion for judgment based on a majority vote on the output results from each of the multiple learning models, and the possibility of reusing the insulators is judged based on a majority vote on each of the pieces of information regarding the reuse of the insulators obtained from each of the multiple learning models.

[0011] In this embodiment, the judgment criteria for judging whether the insulator can be reused include a criterion for judging by majority vote on the output results from each of the multiple learning models. Therefore, by using the judgment criteria, even if there is variation or difference in the information on the appearance state of the insulator acquired from each of the multiple learning models of different types, the control unit of the information processing device outputs information on the reuse of the insulator (judgment result on whether reuse is possible or not) based on the output results (information on the appearance state of the insulator) from more than half of the multiple learning models. The multiple learning models may output whether reuse is possible or not according to the appearance state as the acquired information on the appearance state of the insulator. In this case, the control unit of the information processing device may acquire the information on whether reuse is possible or not acquired from the multiple learning models, and derive the judgment result on whether reuse is possible or not by majority vote on the information on whether reuse is possible or not.

[0012] In one aspect of the program of the present disclosure, information regarding the external condition of the insulator includes information regarding cracks, information regarding chips, information regarding lightning strike marks, and information regarding scratches, and each of the multiple learning models outputs the information regarding the cracks, information regarding the chips, information regarding the lightning strike marks, and information regarding the scratches.

[0013] In this embodiment, the information on the appearance state of the insulator output by the learning model includes information on cracks, chips, lightning marks, and scratches. That is, the learning model outputs, as information on the appearance state of the insulator, the presence or absence of a damaged part due to a crack, chip, lightning mark, or scratch on the outer surface or outer periphery of the insulator, and if there is a damaged part, the area or size of the damaged part. In this case, the learning model may be configured, for example, by R-FCN (Region-based Fully Convolutional Network), FCIS (fully convolutional instance-aware semantic segmentation), PANet (Path Aggregation Network), or Mask-Scoring-R-CNN having the function of instance segmentation or semantic segmentation. In this way, each learning model outputs the presence or absence of each type of damaged part such as a crack or chip, and if there is, the area of ​​the damaged part (area specified by a bounding box, etc.). At this time, the control unit of the information processing device derives information on the reuse of the insulator (a judgment result of whether or not the insulator can be reused) based on the presence or absence of each type of damaged part output by the multiple learning models. The control unit of the information processing device may aggregate the presence or absence of each type of damaged part output by the multiple learning models, and compare the aggregation result with a judgment criterion (judgment table) to judge whether or not the insulator can be reused (derive a judgment result). For example, the judgment criterion (judgment table) may define that the insulator cannot be reused if there is one or more cracks, lightning strike marks, or chips with a relatively high degree of severity. Furthermore, the fixed criterion (judgment table) may define that the insulator cannot be reused if there are five or more scratches with a relatively low degree of severity. In this way, by judging whether or not the insulator can be reused based on the number of damaged parts by type on the outer surface or outer periphery of the insulator, a judgment regarding the reuse of the insulator can be efficiently performed.

[0014] A program according to one embodiment of the present disclosure outputs, in a tabular format, the information regarding the cracks, the information regarding the chips, the information regarding the lightning strike marks, and the information regarding the scratches output by each of the multiple learning models, in association with the learning model from which the information was output.

[0015] In this embodiment, the control unit of the information processing device outputs, as derived information on reuse of insulators, information on cracks, information on chips, information on lightning strike marks, and information on scratches output by each of the multiple learning models, in a table format that correlates the information with the learning model that is the output source. At this time, the control unit of the information processing device may generate screen data including a table that lists this information, and output the generated screen data to a display device such as a display or a terminal device such as a smartphone. The control unit of the information processing device displays, in a list, each of the information on the appearance state of the insulators output by the multiple learning models together with the derived (determined) information on reuse of insulators, thereby providing useful information to workers who perform appearance inspections of recovered insulators.

[0016] An information processing method according to one aspect of the present disclosure acquires images of insulators recovered from power transmission and distribution equipment, and inputs the acquired images including insulators into each of a plurality of learning models that have been trained to output information regarding the external state of the insulators when an image including an insulator is input, thereby acquiring information regarding the external state of the insulators from each of the plurality of learning models, deriving information regarding the reuse of the insulators based on each of the acquired information regarding the external state of the insulators, and outputting the derived information regarding the reuse of the insulators.

[0017] In this aspect, it is possible to provide an information processing method that can efficiently determine whether or not to reuse insulators.

[0018] An information processing device according to one embodiment of the present disclosure includes an image acquisition unit that acquires images of insulators recovered from power transmission and distribution equipment, an information acquisition unit that acquires information relating to the external state of the insulators from each of a plurality of learning models by inputting the acquired images including insulators into each of the plurality of learning models, the learning models being trained to output information relating to the external state of the insulators when an image including an insulator is input, a derivation unit that derives information relating to the reuse of the insulators based on each of the acquired information relating to the external state of the insulators, and an output unit that outputs the derived information relating to the reuse of the insulators.

[0019] In this aspect, it is possible to provide an information processing device that can efficiently make a decision regarding the reuse of insulators. Effect of the Invention

[0020] It is possible to provide a program or the like that efficiently determines whether or not insulators can be reused. [Brief description of the drawings]

[0021] [Figure 1] FIG. 1 is a schematic diagram for explaining an overview of an insulator determination system according to a first embodiment. [Diagram 2] FIG. 1 is a block diagram showing a configuration of an information processing device. [Diagram 3] FIG. 1 is an explanatory diagram regarding a generation process of a learning model. [Figure 4] 11 is a flowchart illustrating a processing procedure (during learning) by a control unit of the information processing device. [Diagram 5] 2 is a functional block diagram illustrating functional units included in a control unit of the information processing device. FIG. [Figure 6] 11 is a flowchart illustrating a processing procedure (during operation) by a control unit of the information processing device. [Figure 7] FIG. 11 is an explanatory diagram illustrating information (a judgment table) relating to judgment criteria; [Figure 8] FIG. 13 is an explanatory diagram illustrating an example of a determination result screen (screen data including a table listing information). [Figure 9] 13 is an explanatory diagram illustrating a collection information screen (collection information table) of collected insulators; FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0022] (Embodiment 1) Hereinafter, an embodiment will be described with reference to the drawings. Fig. 1 is a schematic diagram for explaining an overview of an insulator determination system S according to the first embodiment. Fig. 2 is a block diagram showing the configuration of an information processing device 1. The insulator determination system S is configured with the information processing device 1 as a main device, and the information processing device 1 acquires an image of an insulator G captured by an imaging device C. After being used in a power transmission and distribution facility D, such as a power transmission line, a power distribution line, or a pole-mounted transformer, the insulator G is periodically taken down from a pole and collected, and the collected insulator G is imaged by the imaging device C.

[0023] The imaging device C is, for example, a CMOS camera, and captures an image of an insulator G placed on a monochromatic floor surface (green cloth) of green or the like, surrounded by a light shielding object under a visible light source such as a white LED. The imaging device C may be, for example, a camera built into an information terminal T such as a smartphone held by a maintenance worker or the like of the power transmission and distribution facility D, or may be connected via an input / output I / F 14 of the information processing device 1. The information terminal T of the maintenance worker or the like and the information processing device 1 are communicatively connected through an external network such as the Internet, a LAN, or the like. The image of the insulator G captured by the imaging device C is acquired by the information processing device 1 by transmitting from the information terminal T of the maintenance worker or the like, acquiring via the input / output I / F 14 of the information processing device 1, or acquiring offline via a storage medium such as a memory card.

[0024] The information processing device 1 is a computer capable of various information processing and sending and receiving information, such as a server device, a personal computer, etc. The server device includes not only a single server device, but also a cloud server device or a virtual server device configured by multiple computers. The information processing device 1 includes a control unit 11, a storage unit 12, a communication unit 13, and an input / output I / F 14.

[0025] The control unit 11 has one or more central processing units (CPUs), micro-processing units (MPUs), graphics processing units (GPUs), and other processing devices with timing functions, and performs various information processing, control processing, and the like by reading and executing a program P (program product) stored in the memory unit 12.

[0026] The storage unit 12 includes a volatile storage area such as a static random access memory (SRAM), a dynamic random access memory (DRAM), or a flash memory, and a non-volatile storage area such as an EEPROM or a hard disk. The storage unit 12 stores in advance a program P (program product) and data to be referenced during processing, and further stores various data including intermediate data generated during processing. The program P (program product) stored in the storage unit 12 may be a program P (program product) read from a recording medium M readable by the information processing device 1. Alternatively, the program P (program product) may be downloaded from an external computer (not shown) connected to a communication network (not shown) and stored in the storage unit 12.

[0027] The communication unit 13 is a communication module or a communication interface for communicating with an information terminal T, etc., by wired communication such as Ethernet (registered trademark) or wirelessly, and is, for example, a short-range wireless communication module such as Wi-Fi (registered trademark) or Bluetooth (registered trademark), or a wide-area wireless communication module such as 4G or 5G. The control unit 11 communicates with an information terminal T, such as a smartphone held by a maintenance worker of the power transmission and distribution facility D, via the communication unit 13, for example, through an external network such as the Internet or a LAN.

[0028] 3 is an explanatory diagram regarding the generation process of the learning model 101. The control unit 11 of the information processing device 1 trains a neural network such as R-CNN using training data for the learning model 101, and generates the learning model 101 that receives an image including an insulator G as input and outputs information regarding the appearance state of the insulator G. The control unit 11 of the information processing device 1 generates a plurality of different types of learning models 101 by training a plurality of different types of object detection models using the same training data.

[0029] The image including the insulator G may be an image in which the background of the insulator G is keyed and monochromatized or deleted. The information on the appearance state of the insulator G output by the learning model 101 may include, for example, the presence or absence of damaged parts due to cracks, chips, lightning strike marks, or scratches on the outer surface or outer periphery of the insulator G, and, if there are damaged parts, the area or size of the damaged parts. Alternatively, the information on the appearance state of the insulator G output by the learning model 101 may be information indicating whether the insulator G can be reused.

[0030] The training data includes problem data and answer data, an image including the insulator G corresponds to the problem data, and information on the external appearance of the insulator G corresponds to the answer data. The information on the external appearance of the insulator G, which is the answer data, may be set by annotating (adding) to the image including the insulator G, the area of ​​the damaged part on the outer surface of the insulator G due to a crack, chip, lightning strike mark, or scratch, and the type of the damaged part. The data set of problem data and answer data included in the training data for learning the learning model 101 and the data set of input data and output data when the learning model 101 is used are synonymous, and if it is defined in either data set, it naturally applies to the other data set.

[0031] The neural network (learning model 101) trained using the training data is expected to be used as a program module that is a part of artificial intelligence software. The learning model 101 is used in the information processing device 1 equipped with the control unit 11 (CPU, etc.) and the storage unit 12 as described above, and a neural network system is configured by being executed in the information processing device 1 having such calculation processing capabilities. That is, the control unit 11 of the information processing device 1 performs calculations to extract features of an image including an insulator G input to an input layer in accordance with instructions from the learning model 101 stored in the storage unit 12, and outputs information on the appearance state of the insulator G from an output layer.

[0032] The learning model 101 is configured, for example, by R-CNN (Region Convolutional Neural Network) or YOLO, and has an input layer that accepts an input of an image including an insulator G, an intermediate layer that extracts features of the image including the insulator G, and an output layer that outputs information on the appearance state of the insulator G. The input layer has a plurality of neurons that accept an input of an image including the insulator G, and passes the input values ​​to the intermediate layer. The intermediate layer is defined using an activation function such as a ReLU function or a sigmoid function, has a plurality of neurons that extract features of each input value, and passes the extracted features to the output layer. Parameters such as the weighting coefficient and bias value of the activation function are optimized using an error backpropagation method. The output layer is configured, for example, by a fully connected layer, and outputs information on the appearance state of the insulator G based on the features output from the intermediate layer.

[0033] In this embodiment, the learning model 101 is R-CNN or the like, but is not limited thereto, and may be a learning model 101 constructed with other machine learning algorithms such as neural networks other than R-CNN, transformers, BERT, GPT, RNN (Recurrent Neural Network), LSTM (Long-short term model), SVM (Support Vector Machine), Bayesian network, linear regression, regression tree, multiple regression, random forest, ensemble, etc. Alternatively, the learning model 101 may be configured using an artificial intelligence chatbot such as ChatGPT. In this case, ChatGPT may be fine-tuned or the like so as to efficiently output information on the appearance state of the insulator G. Alternatively, a question generated using an external database such as WebDB may be input together with an image including the insulator G to a prompt, which is an input interface of ChatGPT.

[0034] 4 is a flowchart illustrating a processing procedure (during learning) by the control unit 11 of the information processing device 1. The control unit 11 of the information processing device 1 accepts an operation by an operator using, for example, a keyboard connected to the input / output I / F 14, and performs the following processing based on the accepted operation.

[0035] The control unit 11 of the information processing device 1 acquires an image including the insulator G (S1). The control unit 11 of the information processing device 1 acquires the image including the insulator G from the imaging device C. When capturing the image, the insulator G is captured by the imaging device C (such as a CMOS camera) to which a polarizing filter is attached. This makes it possible to suppress (prevent) disturbance or halation, and improves the accuracy of inference (estimation) by the learning model 101.

[0036] The control unit 11 of the information processing device 1 keys the background of the acquired image including the insulator G (S2). The control unit 11 of the information processing device 1, for example, keys the background of the image including the insulator G as preprocessing of the image. As a result, the background of the insulator G in the image becomes a single-color image, and the background of the insulator G can be substantially removed, thereby improving the inference (estimation) accuracy by the learning model 101. The preprocessing of the image is not limited to keying, and may include, for example, a process of adjusting the image area occupied by the insulator G to a predetermined size.

[0037] The control unit 11 of the information processing device 1 performs annotation on the image including the keyed insulator G (S3). The image including the keyed insulator G is annotated with the area of ​​the damaged part due to cracks, chips, lightning strike marks, or scratches on the outer surface or outer periphery of the insulator G and the type of the damaged part according to the appearance state of the insulator G. In this case, the image including the insulator G corresponds to the question data, and the annotated area and type of the damaged part correspond to the answer data.

[0038] The control unit 11 of the information processing device 1 generates a learning dataset (S4). The control unit 11 of the information processing device 1 generates a learning dataset (training data) by combining a plurality of images, which are problem data, with answer data consisting of the areas and types of damaged parts annotated in each of these images. The plurality of images, which are problem data, are, for example, images including insulators G that can be reused and insulators G that cannot be reused. A large number of images of such insulators G in various appearance states are stored by equipment companies and the like that are responsible for the maintenance and management of power transmission and distribution equipment D, and by using these images, the learning dataset (training data) can be generated.

[0039] The control unit 11 of the information processing device 1 generates a learning model 101 using the generated learning dataset (S5). The control unit 11 of the information processing device 1 applies the learning dataset (training data) to a plurality of different types of neural networks, such as R-CNN and YOLO, and causes them to learn, thereby generating a plurality of different types of learning models 101.

[0040] 5 is a functional block diagram illustrating functional units included in the control unit 11 of the information processing device 1. The control unit 11 of the information processing device 1 executes a program P stored in the storage unit 12 to function as an acquisition unit 111, a preprocessing unit 112, a judgment result derivation unit 113, and an output unit 114. The control unit 11 of the information processing device 1 functions as the learning model 101 by reading an entity file of the learning model 101 stored in the storage unit 12, etc.

[0041] The acquisition unit 111 acquires an image including the insulator G captured by an imaging device C such as a CMOS camera, and functions as an image acquisition unit. The insulator G collected from the power transmission and distribution equipment D is managed in association with the type (insulator type) of the insulator G along with various collection information such as the model of the insulator G mounted in the power transmission and distribution equipment D from which the insulator G was collected, the location of the power transmission and distribution equipment D, the period of use, and the collection date. The collection information may be stored in the storage unit 12 of the information processing device 1 by being saved and managed in a collection information table. The insulator G collected from the power transmission and distribution equipment D, i.e., the insulator G that has been lowered from the pole, is surrounded by a light shielding object under a visible light source such as white light, placed on a monochromatic floor surface (green cloth) such as green, and is imaged by the imaging device C such as a CMOS camera. A polarizing filter may be attached to the imaging device C (CMOS camera, etc.).

[0042] The imaging device C is connected to, for example, an input / output I / F 14 of the information processing device 1, and acquires an image including the insulator G from the imaging device C via the input / output I / F 14. Alternatively, the imaging device C may be implemented in an information terminal T such as a smartphone carried by a maintenance worker or the like of the power transmission and distribution facility D, and the information processing device 1 may acquire an image including the insulator G by communicating with the information terminal T.

[0043] The pre-processing unit 112 performs pre-processing, such as keying, on the image acquired by the acquisition unit 111. The pre-processing unit 112 performs processing on an image including an insulator G, such as keying the background of the insulator G to monochromatize or delete the background. The pre-processing unit 112 may further perform processing such as adjusting the image area occupied by the insulator G to a predetermined size. The pre-processing unit 112 inputs the pre-processed image (image including the insulator G), such as keying the background, to each of the multiple learning models 101.

[0044] These multiple learning models 101 (three in the figure) are different types of object detection models. Alternatively, these multiple learning models 101 may be the same type of object detection models that have been trained using different training data. Each of the multiple learning models 101 outputs information about the appearance state of the insulator G based on an input image (an image including the insulator G with the background keyed). The information about the appearance state of the insulator G output by the learning model 101 includes, for example, the presence or absence of a damaged portion due to a crack, chip, lightning strike mark, or scratch on the outer surface or outer periphery of the insulator G, and, if there is a damaged portion, the area or size of the damaged portion. The area of ​​the damaged portion may be specified, for example, by a bounding box superimposed on the image including the insulator G.

[0045] The judgment result derivation unit 113 functions as an information acquisition unit that acquires information on the external appearance state of the insulator G output (estimated) by the learning model 101, and derives a judgment result on whether or not the insulator G can be reused based on the information on the external appearance state of the insulator G. Each of the multiple learning models 101 outputs, as information on the external appearance state of the insulator G, the presence or absence of damaged parts due to cracks, chips, lightning strike marks, or scratches on the outer surface or outer periphery of the insulator G, and if there are damaged parts, the area of ​​the damaged parts, etc.

[0046] The determination result derivation unit acquires information including the type and presence or absence of damaged parts in the insulator G (information regarding the external appearance state of the insulator G) from each of the multiple learning models 101, and consolidates the acquired information. In consolidating this information, the determination result derivation unit may calculate the number of damaged parts of each type output by each of the multiple learning models 101. Alternatively, in consolidating this information, the determination result derivation unit may specify the identity of the areas of damaged parts output by each of the multiple learning models 101.

[0047] The determination result derivation unit 113 may derive the number of damaged parts for each type by majority vote from the information on the appearance state of the insulator G (the number of damaged parts for each type) acquired from each of the multiple learning models 101 in this way. The determination result derivation unit 113 may compare the number of damaged parts for each type derived in this way with a determination criterion (determination table) stored in the storage unit 12 in a table format, for example, to determine whether or not the insulator can be reused (derive a determination result). The details of the determination table will be described later, but the number allowed for reuse (determination threshold) is predetermined in the determination criterion (determination table) according to the type of damage. The determination result derivation unit 113 may determine that the insulator G is reusable when the number of damaged parts for each type derived by majority vote is less than the determination threshold, and determine that the insulator G is not reusable when the number is equal to or greater than the determination threshold.

[0048] The output unit 114 outputs the determination result from the determination result derivation unit 113, i.e., information on whether the insulator G can be reused, to an information terminal T such as a smartphone held by a maintenance worker of the power transmission and distribution facility D. Alternatively, the output unit 114 may output the determination result to a display device such as a display connected to the input / output I / F 14. The output unit 114 may output information on the appearance condition of the insulator G that is the basis for the determination (the number of damaged parts by type), etc., in addition to the determination result on whether the insulator G can be reused.

[0049] In the present embodiment, the learning model 101 estimates (outputs) information on the appearance state of the insulator G, and the judgment result derivation unit 113 judges whether the insulator G can be reused based on the information on the appearance state of the insulator G, but the present invention is not limited to this, and the learning model 101 may output whether the insulator G can be reused. That is, the information on the appearance state of the insulator G corresponds to intermediate data for judging whether the insulator G can be reused, and the learning model 101 may include the function of the judgment result derivation unit 113 and output whether the insulator G can be reused as information on the appearance state of the insulator G based on the input image including the insulator G. In this case, the answer data included in the training data when learning the learning model 101 corresponds to a flag value indicating whether the image of the insulator G can be reused. At this time, the flag value indicating whether the image of the insulator G can be reused is annotated to the image of the insulator G.

[0050] 6 is a flowchart illustrating a processing procedure (during operation) by the control unit 11 of the information processing device 1. The control unit 11 of the information processing device 1 accepts an operation by an operator using, for example, a keyboard connected to the input / output I / F 14, and performs the following processing based on the accepted operation.

[0051] The control unit 11 of the information processing device 1 acquires an image of an insulator G collected from a power transmission and distribution facility D (S101). The control unit 11 of the information processing device 1 acquires an image including the insulator G captured by an imaging device C such as a CMOS camera. The image including the insulator G is captured by the imaging device C such as a CMOS camera under a visible light source such as a white LED, which is a light source with high color rendering properties.

[0052] The control unit 11 of the information processing device 1 performs keying on the background of the acquired image including the insulator G (S102). The control unit 11 of the information processing device 1 performs keying on the background of the insulator G in the image including the insulator G to monochromatize or delete the background.

[0053] The control unit 11 of the information processing device 1 inputs an image including the keyed insulator G to each of the multiple learning models 101 (S103). The control unit 11 of the information processing device 1 acquires each of the pieces of information related to the appearance state of the insulator G output from each of the learning models 101 (S104). The control unit 11 of the information processing device 1 inputs the image including the keyed insulator G (an image with the background of the insulator G deleted) to each of the multiple learning models 101 consisting of different types. In other words, the same image (an image with the background of the insulator G deleted) is input to each of the multiple learning models 101.

[0054] Each of the multiple learning models 101 outputs (estimates) information about the external appearance of the insulator G (presence or absence of damaged areas due to cracks, chips, lightning strike marks, or scratches on the outer surface or outer periphery of the insulator G, and if there are damaged areas, the area of ​​the damaged areas, etc.) based on the input image of the insulator G. The control unit 11 of the information processing device 1 obtains information about the external appearance of the insulator G (presence or absence of damaged areas due to cracks, chips, lightning strike marks, or scratches on the outer surface or outer periphery of the insulator G, and if there are damaged areas, the area of ​​the damaged areas, etc.) from each of the multiple learning models 101.

[0055] The control unit 11 of the information processing device 1 derives information regarding the reuse of the insulator G (performs a judgment as to whether or not reuse is possible) based on each piece of acquired information regarding the external appearance state of the insulator G (S105). The control unit 11 of the information processing device 1 aggregates each piece of information regarding the external appearance state of the insulator G acquired from each of the multiple learning models 101, and derives information regarding the reuse of the insulator G (performs a judgment as to whether or not reuse is possible) by referring to the judgment criteria (judgment table) stored in the memory unit 12.

[0056] 7 is an explanatory diagram illustrating information (a judgment table) relating to judgment criteria. The judgment criteria for judging whether the insulator G can be reused include a criterion for judging by majority vote on the output results from each of the multiple learning models 101. The control unit 11 of the information processing device 1 judges whether the insulator G can be reused based on the output results (information relating to the external appearance state of the insulator G) from more than half of the multiple learning models 101 in accordance with the majority vote determined by the judgment criteria.

[0057] Furthermore, the judgment criteria (judgment table) defines judgment thresholds for the number of damaged parts by type (the number derived by majority vote). In the illustration of this embodiment, the judgment table defines a judgment threshold (e.g., 1) for cracks, lightning strike marks, or chips, which are relatively serious, and a judgment threshold (e.g., 5) for scratches, which are relatively less serious. These judgment thresholds are merely examples, and the judgment table may be configured so that the judgment thresholds can be changed.

[0058] The control unit 11 of the information processing device 1 compares the number of damaged parts for each type derived by majority vote with a judgment threshold determined for each type of damage, and judges that the insulator G can be reused if the number is less than the judgment threshold. That is, in the example of the judgment table in this embodiment, the control unit 11 of the information processing device 1 judges that the insulator G cannot be reused if there is one or more cracks, lightning marks, or chips, or if there are five or more scratches. The control unit 11 of the information processing device 1 judges that the insulator G can be reused if there is less than one crack, lightning mark, or chip (i.e., there are no cracks, lightning marks, or chips) and there are less than five scratches. The control unit 11 of the information processing device 1 derives the judgment result of whether the insulator G can be reused as information regarding the reuse of the insulator G.

[0059] The control unit 11 of the information processing device 1 outputs the derived information on the reuse of the insulator G (S106). The control unit 11 of the information processing device 1 generates screen data (determination result screen) including a table listing information on the reuse of the insulator G, such as the determination result of whether the reuse is possible and information on the appearance state of the insulator G output by each of the learning models 101 that is the basis for the determination, and outputs the generated screen data (determination result screen) to a display device such as a display or a terminal device such as a smartphone.

[0060] The display device or terminal device that has acquired the screen data (determination result screen) from the information processing device 1 displays the determination result screen based on the screen data (determination result screen). Furthermore, the control unit 11 of the information processing device 1 may generate screen data (recycling information screen) including a table that lists collection information associated with the collected insulators G in addition to the determination result screen, and output the generated screen data (recycling information screen) to a display device such as a display or a terminal device such as a smartphone.

[0061] 8 is an explanatory diagram illustrating a judgment result screen (screen data including a table listing information). The judgment result screen includes a management item display area, a result display area for each model, and a result display area based on majority vote.

[0062] The management item display area includes, as display items, insulator type, installed model, installation location, period of use, and recovery date. The insulator type display item displays the model of the insulator G that is the subject of judgment as to whether it can be reused. The installed model displays the model name of the power transmission and distribution equipment D in which the insulator G was installed. The installation location displays the installation location of the power transmission and distribution equipment D in which the insulator G was installed. The period of use displays the period of use of the power transmission and distribution equipment D in which the insulator G was installed. The recovery date displays the date on which the insulator G was taken down from the pole and recovered. By displaying a list of information (recovery information) related to the recovery of the insulator G that is the subject of judgment as to whether it can be reused in this manner, visibility can be improved for maintenance workers of the power transmission and distribution equipment D.

[0063] The result display area for each model includes, as display items, the learning model 101, the number of each type of damaged part (cracks, chips, lightning marks, scratches), and an image link. In the learning model 101, a model name for uniquely identifying each of the multiple learning models 101 is displayed. In the number of each type of damaged part (cracks, chips, lightning marks, scratches), the number of cracks, the number of chips, the number of lightning marks, and the number of scratches are displayed. These numbers are values ​​counted by the control unit 11 of the information processing device 1 based on the output results from each of the multiple learning models 101. In the image link, a character string (click here for the output image) with a hyperlink set to an image output by the corresponding learning model 101 (an image of an insulator G with a bounding box or the like indicating the area of ​​the damaged part superimposed thereon) is displayed. In this way, by listing the output results from each of the multiple learning models 101, the visibility for maintenance workers of the power transmission and distribution facility D can be improved.

[0064] The majority vote result display area includes, as display items, the number of each type of damage (cracks, chips, lightning strike marks, scratches) and the reuse judgment result. The number of each type of damage (cracks, chips, lightning strike marks, scratches) is displayed as the number of each type of damage derived by majority vote in the information on the appearance state of the insulator G (the number of damaged parts by type) acquired from each of the multiple learning models 101. The reuse judgment result is displayed as the result of judgment (judgment result of whether or not reuse is possible) using a judgment threshold defined in a judgment table for the number of each type of damage derived by majority vote. In this way, by displaying not only the reuse judgment result but also the intermediate data that is the basis for the judgment, it is possible to improve visibility for maintenance workers of the power transmission and distribution facility D.

[0065] 9 is an explanatory diagram illustrating a collection information screen (collection information table) of collected insulators G. The collection information screen is configured based on collection information (collection information table) stored in table format in the storage unit 12, and displays the collection information in a list. The collection information displayed in a list includes display items such as insulator type, mounted model, placement location, period of use, collection date, cracks, chips, lightning strike marks, scratches, and reuse.

[0066] The display items of insulator type, installed model, installation location, period of use, and collection date are the same as the display items in the management item display area on the judgment result screen. The display items of period of use, collection date, cracks, chips, lightning strike marks, scratches, and reuse are the same as the display items in the majority vote result display area on the judgment result screen. The character strings (possible, not possible) displayed in the reuse display item may be set with a hyperlink that pops up the judgment result screen for the insulator G corresponding to the character string (possible, not possible).

[0067] These various display items correspond to the management items of the collection information table. The control unit 11 of the information processing device 1 can analyze the distribution or tendency of the collected insulators G by searching the collection information stored in each management item of the collection information table using a predetermined search condition.

[0068] The embodiments disclosed herein are illustrative in all respects and should not be considered as limiting. The scope of the present invention is defined by the claims, not by the above meaning, and is intended to include all modifications within the scope and meaning equivalent to the claims.

[0069] The claims may be combined with each other regardless of the form of reference. The claims include multiple dependent claims that depend on multiple dependent claims. The claims do not include multiple dependent claims that depend on a multiple dependent claim, but multiple dependent claims that depend on a multiple dependent claim may be included. [Explanation of symbols]

[0070] S insulator judgment system, D power transmission and distribution equipment, G insulator, T information terminal, C imaging device, 1 information processing device, 11 control unit, 111 acquisition unit, 112 preprocessing unit, 113 judgment result derivation unit, 114 output unit, 12 memory unit, P program (program product), M recording medium, 13 communication unit, 14 input / output I / F, 101 learning model

Claims

1. On the computer, Images of the insulators collected from the power transmission and distribution equipment are acquired, inputting the acquired image including the insulator into each of a plurality of learning models that have been trained to output information regarding the external appearance state of the insulator when an image including the insulator is input, thereby acquiring information regarding the external appearance state of the insulator from each of the plurality of learning models; deriving information regarding reuse of the insulators based on each of the pieces of information regarding the appearance states of the insulators obtained; Output the derived information regarding reuse of the insulator A program that executes a process.

2. obtaining a criterion for determining whether the insulator can be reused; determining whether or not the insulator can be reused using the obtained judgment criteria for each piece of information on the appearance state of the insulator; The determination result is derived as information regarding reuse of the insulator. The program according to claim 1.

3. The judgment criteria include a criterion for judgment by majority vote on output results from each of the plurality of learning models, A decision is made as to whether or not the insulators can be reused based on a majority vote for each of the pieces of information regarding the reuse of the insulators acquired from each of the plurality of learning models. The program according to claim 2.

4. The information on the appearance state of the insulator includes information on cracks, information on chips, information on lightning strike marks, and information on scratches, Each of the plurality of learning models outputs information about the crack, information about the chip, information about the lightning strike mark, and information about the scratch. The program according to any one of claims 1 to 3.

5. The information on the cracks, the chips, the lightning strike marks, and the scratches output from each of the plurality of learning models are output in a table format in association with the learning models that are the output sources. The program according to claim 4.

6. Images of the insulators collected from the power transmission and distribution equipment are acquired, inputting the acquired image including the insulator into each of a plurality of learning models that have been trained to output information regarding the external appearance state of the insulator when an image including the insulator is input, thereby acquiring information regarding the external appearance state of the insulator from each of the plurality of learning models; deriving information regarding reuse of the insulators based on each of the pieces of information regarding the appearance states of the insulators obtained; Output the derived information regarding reuse of the insulator An information processing method for causing a computer to execute processing.

7. an image acquisition unit that acquires an image of the insulator collected from the power transmission and distribution facility; an information acquisition unit that inputs the acquired image including an insulator into each of a plurality of learning models that have been trained to output information regarding the appearance state of an insulator when an image including an insulator is input, and acquires information regarding the appearance state of the insulator from each of the plurality of learning models; a derivation unit that derives information regarding reuse of the insulators based on each of the acquired information regarding the appearance states of the insulators; an output unit that outputs the derived information regarding reuse of the insulator; An information processing device comprising:

Images