Program, information processing method, information processing device, and model generation method

Abstract

To provide a program, an information processing method, an information processing device, and a model generation method for improving accuracy of determining whether a connection state between an electric wire and a device to which the electric wire is connected is appropriate.SOLUTION: A program causes an information processing device to acquire a captured image of a connection state between a device having multiple terminals to which multiple electric wires are connected and electric wires, input the acquired captured image into a first learning model that has been trained to detect a terminal in the captured image when the captured image is input to detect each terminal of the device in the captured image, input the acquired captured image into a second learning model that has been trained to detect areas of each color in the captured image when the captured image is input to detect each color area in the captured image, identify a color of the electric wire connected to each terminal on the basis of the position of each terminal in the captured image and the area of each color in the captured image, and determine appropriateness of the connection state between each terminal and each electric wire on the basis of the identified color of each electric wire.SELECTED DRAWING: Figure 5

Description

Technical Field

[0001] The present disclosure relates to a program, an information processing method, an information processing apparatus, and a model generation method.

Background Art

[0002] In facilities that use electricity, an electricity meter for measuring the amount of electricity used is installed, and the electricity meter is required to be replaced regularly to maintain accurate measurement. In the replacement work of the electricity meter, the wires connected to the electricity meter are removed from the electricity meter and reconnected to a new electricity meter. After the replacement work, the person in charge submits a photo taken with a camera of the connection state between the electricity meter and the wires, and the person in charge of inspection visually checks this photo to determine whether the wiring state of the wires is appropriate. In the replacement work of the electricity meter, although accurate wiring is performed in most of the work, there are rare cases where incorrect wiring is performed due to misrecognition or illusion of the person in charge of the replacement work. Therefore, in Patent Document 1, an apparatus that uses a learning model to determine whether the connection between an instrument (electricity meter) and wiring (wires) is correct or incorrect from an image including the instrument and the wiring has been proposed. The technology disclosed in Patent Document 1 alerts the person in charge of the work and greatly contributes to preventing the occurrence of incorrect wiring from the time of the work. Also, by using the technology of Patent Document 1, there is an effect that the burden on the person in charge of inspection, who needs to carefully perform the inspection work for all photos, is reduced.

Prior Art Documents

Patent Documents

[0003] ]>

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Patent Document 1 describes a learning model that recognizes the feature quantities of the connection points between the instrument's terminals and the wiring cable, as well as the color of the wiring cable, from images of the instrument and the wiring cable, and determines whether the connection between the instrument and the wiring cable is correct or incorrect. To realize such a learning model, it is necessary to train it with a huge amount of training data. Therefore, there is a need for a technology that can accurately determine whether the connection between an energy meter and an electric wire is correct or incorrect using a learning model without requiring a huge amount of training data.

[0005] This disclosure aims to provide a program or the like that can improve the accuracy of determining whether the connection between the equipment to which the wire is connected and the wire is appropriate. [Means for solving the problem]

[0006] A program according to one aspect of the present disclosure acquires a captured image of the connection state between a device having multiple terminals to which multiple wires are connected and the wires, inputs the acquired captured image to a first learning model that has been trained to detect the terminals in the captured image when the captured image is input, detects each terminal of the device in the captured image, inputs the acquired captured image to a second learning model that has been trained to detect regions of each color in the captured image when the captured image is input, detects regions of each color in the captured image, identifies the color of the wire connected to each terminal based on the position of each terminal in the captured image and the regions of each color in the captured image, and determines whether the connection state between each terminal and each wire is appropriate based on the identified color of each wire. [Effects of the Invention]

[0007] This disclosure makes it possible to improve the accuracy of determining whether the connection between the equipment to which the wire is connected and the wire is appropriate. [Brief explanation of the drawing]

[0008] [Figure 1] This is an explanatory diagram showing an example of the installation of an electricity meter for high-voltage power. [Figure 2]This is a block diagram showing an example of the configuration of an information processing device. [Figure 3] This is an explanatory diagram of the terminal detection model and the color detection model. [Figure 4] This is a flowchart showing an example of the process for generating a terminal detection model. [Figure 5] This flowchart shows an example of a procedure for determining the wiring status. [Figure 6] This is an explanatory diagram of the judgment process. [Figure 7] This is an explanatory diagram showing an example screen. [Figure 8] This flowchart shows an example of the procedure for determining the wiring status in Embodiment 2. [Figure 9] This is an explanatory diagram of the judgment process. [Figure 10] This is an explanatory diagram showing examples of the installation of electricity meters for purchased electricity and electricity meters for sold electricity. [Figure 11] This is an explanatory diagram showing an example of a record layout in the wiring information database. [Figure 12] This flowchart shows an example of the procedure for determining the wiring status in Embodiment 3. [Figure 13] This is an explanatory diagram showing an example of the configuration of an information processing system. [Modes for carrying out the invention]

[0009] The following describes in detail the program, information processing method, information processing device, and model generation method of this disclosure, based on drawings illustrating their embodiments. In each of the following embodiments, an example is described of a device that determines the suitability of the wiring status of wires connected to an electricity meter that measures the amount of electricity used. However, the configuration of this disclosure is applicable to devices that determine the suitability of the wiring status of wires in various devices that have multiple terminals, each to which a wire with a different colored outer protective coating is connected. Note that the wires connected to the devices may be cables, but hereinafter they will be collectively referred to as "wires".

[0010] (Embodiment 1) This embodiment describes an information processing device for determining the appropriateness (correctness) of the wiring status (connection status) of the power lines connected to a power meter for high-voltage power. Figure 1 is an explanatory diagram showing an example of the installation of a power meter for high-voltage power. The power meter 21 for high-voltage power is used in connection with a transformer 23 installed in a high-voltage circuit. The transformer 23 has a voltage transformer and a current transformer. The voltage transformer converts the high voltage in the high-voltage circuit to a low voltage (e.g., 110V), and the current transformer converts the large current to a small current (e.g., 5A). The transformer 23 is connected to power lines connected to the power source side and power lines connected to the load side. The power meter 21 measures the amount of power supplied from the power source side to the load side in the high-voltage circuit based on the voltage and current supplied via the transformer 23. The power meter 21 is connected to the transformer 23 via a terminal block 22, and the voltage and current supplied from the transformer 23 are supplied to the power meter 21 via the terminal block 22.

[0011] The transformer 23 and the energy meter 21 are configured to be connected by seven wires, and the transformer 23 and the energy meter 21 are provided with seven terminals P1, P2, P3, 1S, 3S, 1L, and 3L to which each wire is connected. In the example in Figure 1, the transformer 23 is provided with terminals P1, P2, P3, 1S, 3S, 1L, and 3L from left to right, and the energy meter 21 is provided with terminals 1S, P1, P3, 3S, 3L, P2, and 1L from left to right. In the transformer 23 and the energy meter 21, the corresponding terminals are connected by wires with outer protective sheaths of different colors, for example, red, white, black, green, blue, yellow, and brown wires connect the corresponding terminals P1, P2, P3, 1S, 3S, 1L, and 3L, respectively. The terminal block 22 has a terminal section 22a having terminals corresponding to each terminal of the transformer 23, and a terminal section 22b having terminals corresponding to each terminal of the energy meter 21. Seven wires connected to each terminal of the transformer 23 are connected to the terminals of terminal section 22a, and seven wires connected to each terminal of the energy meter 21 are connected to the terminals of terminal section 22b. In the terminal block 22, the corresponding terminals of terminal sections 22a and 22b are connected, and the voltage and current supplied from the transformer 23 to terminal section 22a of the terminal block 22 are supplied to the energy meter 21 via terminal section 22b of the terminal block 22. When, for example, the energy meter 21 is replaced, the power supply to the energy meter 21 can be stopped by attaching a predetermined jig (e.g., a shorting plug) to the terminal block 22, thereby enabling the replacement of the energy meter 21 without a power outage.

[0012] FIG. 2 is a block diagram showing a configuration example of an information processing apparatus. The information processing apparatus 10 is an apparatus capable of performing various information processes and information transmission and reception, and is, for example, a server computer, a personal computer, or the like. The information processing apparatus 10 includes a control unit 11, a storage unit 12, a communication unit 13, an input unit 14, a display unit 15, a reading unit 16, etc., and these units are mutually connected via a bus. The control unit 11 includes one or a plurality of processors such as a CPU (Central Processing Unit), MPU (Micro-Processing Unit), GPU (Graphics Processing Unit), or AI chip (semiconductor for AI). The control unit 11 executes various information processes and control processes that the information processing apparatus 10 should perform by appropriately executing the program 12P stored in the storage unit 12.

[0013] The storage unit 12 includes a RAM (Random Access Memory), a flash memory, a hard disk, an SSD (Solid State Drive), etc. The storage unit 12 stores in advance the program 12P (program product) executed by the control unit 11 and various data necessary for the execution of the program 12P. Further, the storage unit 12 temporarily stores data and the like generated when the control unit 11 executes the program 12P. Further, the storage unit 12 stores, for example, a terminal detection model M1 and a color detection model M2 that have learned training data by machine learning. The terminal detection model M1 and the color detection model M2 are assumed to be used as program modules constituting artificial intelligence software. The terminal detection model M1 and the color detection model M2 perform a predetermined operation on an input value and output an operation result, and data such as coefficients and threshold values of a function defining this operation are stored in the storage unit 12 as the terminal detection model M1 and the color detection model M2.

[0014] The communication unit 13 is an interface for connecting to the network N by wired communication or wireless communication, and transmits and receives information to and from other devices via the network N. The network N may be the Internet or a public switched telephone network, or may be a LAN constructed within the facility where the information processing device 10 is installed. The input unit 14 receives operation inputs from the user and sends a control signal corresponding to the operation content to the control unit 11. The display unit 15 is a liquid crystal display, an organic EL display, or the like, and displays various types of information according to instructions from the control unit 11. The input unit 14 and the display unit 15 may be a touch panel configured integrally.

[0015] The reading unit 16 reads information stored in a portable storage medium 10a such as a CD (Compact Disc), DVD (Digital Versatile Disc), USB (Universal Serial Bus) memory, SD card, micro SD card, CompactFlash (registered trademark), etc. The program 12P and various types of data stored in the storage unit 12 may be read by the control unit 11 from the portable storage medium 10a via the reading unit 16 and stored in the storage unit 12. Also, the program 12P and various types of data may be written into the storage unit 12 at the manufacturing stage of the information processing device 10, or may be downloaded by the control unit 11 from other devices via the communication unit 13 and stored in the storage unit 12.

[0016] The information processing device 10 may be a multi-computer including a plurality of computers, may be a virtual machine virtually constructed by software within a single device, or may be a cloud server. Also, the program 12P may be executed on a single computer, or may be executed on a plurality of computers interconnected via the network N. Furthermore, for the information processing device 10, the input unit 14 and the display unit 15 are not essential, and it may be configured to receive operations through a connected computer, or may be configured to output information to be displayed to an external display device.

[0017] Figure 3 is an explanatory diagram of the terminal detection model M1 and the color detection model M2. Figure 3A shows an example of the configuration of the terminal detection model M1. The terminal detection model M1 (first learning model) is trained to take a captured image of the connection state between the power meter 21 and the power wire as input, and to perform a calculation to recognize each terminal of the power meter 21 in the captured image based on the input image, and to output the recognized result. The captured image input to the terminal detection model M1 is, for example, as shown in Figure 3C, an image (hereinafter referred to as the connection image) extracted from a captured image that includes the power meter 21 and the power wire connected to the power meter 21, showing the region of the connection between the terminals of the power meter 21 and the power wire. The terminal detection model M1 can be constructed using object detection algorithms such as YOLO (You Only Look Once), R-CNN (Regions with Convolution Neural Network), SSD (Single Shot Multibook Detector), or a combination of multiple algorithms.

[0018] The terminal detection model M1 has an input layer that receives a wiring image, an intermediate layer that extracts features from the input wiring image, and an output layer that outputs an image in which the terminals of the power meter 21 in the wiring image are detected based on the calculation results of the intermediate layer. The intermediate layer calculates output values ​​based on the wiring image input from the input layer using various functions and thresholds. The output layer outputs an image (hereinafter referred to as a label image) in which bounding boxes surrounding the detected terminals are added to the input wiring image. With this configuration, when a wiring image is input, the terminal detection model M1 outputs a label image in which bounding boxes are added to the terminals of the power meter in the wiring image.

[0019] The terminal detection model M1 can be generated by machine learning using training data (first training data) that includes training wiring images and images (training label images) in which the terminals of the electricity meter in the wiring images have been marked (bounding boxes). The training data is generated by associating the wiring images with images (correct label images) in which an annotation person has marked (bounding boxes) the terminals in the images.

[0020] The terminal detection model M1 learns to output the correct label image included in the training data when a wiring image included in the training data is input. During the learning process, the terminal detection model M1 performs calculations in the hidden layer and output layer based on the input wiring image and calculates the label image to be output from the output layer. The terminal detection model M1 compares the calculated label image with the correct label image and optimizes the parameters used in the calculations in the hidden layer and output layer so that the two are approximate. These parameters are the weights (coupling coefficients) between nodes in the hidden layer and output layer. The method of parameter optimization is not particularly limited, but methods such as backpropagation and steepest descent can be used. As a result, a terminal detection model M1 is obtained that outputs a label image with bounding boxes added to the terminals of the power meter in the wiring image when a wiring image is input.

[0021] Figure 3B shows an example of the configuration of the color detection model M2. The color detection model M2 is a model that recognizes predetermined objects contained in an input image, and can classify objects in an image on a pixel-by-pixel basis, for example, by semantic segmentation. Specifically, the color detection model M2 (second learning model) is trained to take a captured image of the connection state between the power meter 21 and the power lines as input, perform calculations to recognize each color in the captured image based on the input image, and output the recognized result. The captured image input to the color detection model M2 is also a connection image obtained by extracting the region of the connection between the terminals of the power meter 21 and the power lines from a captured image including the power meter 21 and the power lines, as shown in Figure 3C. The color detection model M2 can be constructed using algorithms such as U-Net, FCN (Fully Convolutional Network), SegNet, Transformer, etc., and may be constructed by combining multiple algorithms.

[0022] The color detection model M2 has an input layer that receives a line drawing image, an intermediate layer that extracts features from the input line drawing image, and an output layer that, based on the calculation results of the intermediate layer, classifies each pixel of the line drawing image according to a learned color and outputs a classified image (hereinafter referred to as the labeled image) in which each pixel is associated with a color label. In the example shown in Figure 3B, the color detection model M2 indicates the pixels classified into each color with hatching corresponding to each color. With this configuration, when a line drawing image is input, the color detection model M2 outputs a labeled image in which each pixel in the line drawing image is classified according to color.

[0023] The color detection model M2 can be generated by machine learning using training data (second training data) that includes a training line drawing image and a ground truth label image in which data indicating the color to be distinguished is labeled for each pixel in the line drawing image. The training data is generated by associating the line drawing image with an image (ground truth label image) in which an annotation person has labeled the regions of each color in the image according to the color.

[0024] The color detection model M2 learns to output the correct label image from the training data when a line drawing image from the training data is input. During the learning process, the color detection model M2 performs calculations in the hidden layer and output layer based on the input line drawing image, and calculates the label image to be output from the output layer. Specifically, the color detection model M2 calculates a label image in which a value indicating the determined color is labeled for each pixel in the line drawing image. Then, the color detection model M2 compares the calculated label image with the regions of each color in the correct label image and optimizes the parameters used in the calculations in the hidden layer and output layer so that the two approximate each other. Here again, parameters such as the weights (coupling coefficients) between nodes in the hidden layer and output layer are optimized using methods such as backpropagation and steepest descent. As a result, a color detection model M2 is obtained that outputs a label image in which each pixel in the line drawing image is classified by color when a line drawing image is input.

[0025] The information processing device 10 prepares the terminal detection model M1 and color detection model M2 as described above in advance and uses them to determine whether the wiring state is appropriate based on the captured image of the wiring state of the power meter 21 and the wires. Specifically, the information processing device 10 uses the terminal detection model M1 to acquire label images in which the terminals of the power meter are detected in the wiring image, and uses the color detection model M2 to acquire label images in which each pixel in the wiring image is classified by color. Then, based on the label images in which the terminals are detected and the label images classified by color, the information processing device 10 identifies (predicts) the color of the wires connected to each terminal of the power meter and determines whether the color of each identified wire is appropriate. The terminal detection model M1 and color detection model M2 may be trained by another learning device. The trained terminal detection model M1 and color detection model M2 generated by training by another learning device are downloaded from the learning device to the information processing device 10, for example, via the network N or the portable storage medium 10a, and stored in the storage unit 12.

[0026] The following describes the generation process for terminal detection model M1 and color detection model M2. Figure 4 is a flowchart showing an example of the generation process for terminal detection model M1. The following process is performed by the control unit 11 of the information processing device 10 according to the program 12P stored in the memory unit 12, but it may also be performed by another learning device. In the following process, the control unit 11 first generates training data based on the captured image including the power meter 21 and the power lines, and then uses the generated training data to train terminal detection model M1.

[0027] The control unit 11 of the information processing device 10 acquires a captured image of the power meter 21 and the wiring status of the power lines (S11), and displays the acquired captured image on the display unit 15 (S12). The captured image used for generating training data may be stored in the storage unit 12 in advance, acquired from another device via the communication unit 13, or read from the portable storage medium 10a by the reading unit 16. The captured image is an image like the one shown on the left side of Figure 3C.

[0028] The control unit 11 extracts a region (connection image) from the captured image that includes each terminal of the energy meter 21 and the wires connected to each terminal (S13). The region to be extracted may be specified by the user (e.g., the person in charge of annotation) via the input unit 14, or it may be a region pre-set for the captured image. Alternatively, the control unit 11 may perform object detection processing on the captured image to detect the energy meter 21 in the image and extract a region (including the terminals) at a predetermined position for the detected energy meter 21 as a connection image. Object detection in the image can be performed using a learning model such as YOLO, R-CNN, or SSD, or it can be performed using template matching technology. In template matching, a template showing the image features of the device to be detected (here, the energy meter 21) is stored in the storage unit 12 in advance, and the control unit 11 can detect the presence or absence of the device to be detected depending on whether or not a region matching the template exists in the captured image. When using a learning model or template matching, the clipping range to be extracted as a connection image from the captured image can be automatically determined. The wiring diagram is shown on the right side of Figure 3C.

[0029] The control unit 11 performs annotation on the extracted wiring image by marking the terminals in the image (S14). Here, the control unit 11 receives the area of ​​the terminals in the image from the user via the input unit 14, and generates a correct label image by adding a bounding box to the received area. The control unit 11 generates training data by associating the wiring image extracted in step S13 with the correct label image generated in step S14, and stores it in the storage unit 12 (S15). The control unit 11 stores the training data in a training DB (not shown) prepared in the storage unit 12, for example.

[0030] The control unit 11 determines whether or not there are any unprocessed images among the captured images to be processed as training data (S16). If it determines that there are unprocessed images (S16: YES), the control unit 11 returns to the process of step S11 and performs the processing of steps S11 to S15 on the unprocessed images. The control unit 11 repeats the processing of steps S11 to S16 until it determines that there are no unprocessed images. As a result, training data to be used to train the terminal detection model M1 is generated based on the captured images prepared for processing and stored in the training DB.

[0031] If the control unit 11 determines that there are no unprocessed captured images (S16: NO), it trains the terminal detection model M1 using the training data stored in the training DB as described above. The control unit 11 reads one of the training data stored in the training DB through the process described above (S17). Then, the control unit 11 performs the training process for the terminal detection model M1 based on the read-out training data (S18). Here, the control unit 11 inputs the wiring image included in the training data to the terminal detection model M1 and obtains the output value (label image) output from the terminal detection model M1 as a result of the input of the wiring image. The control unit 11 compares the label image output from the terminal detection model M1 with the correct label image included in the training data and trains the terminal detection model M1 so that the two approximate each other. In the training process, the terminal detection model M1 optimizes the parameters used for calculations in the hidden layer and output layer by using, for example, a backpropagation method that sequentially updates from the output layer to the input layer.

[0032] The control unit 11 determines whether there is any unprocessed training data stored in the training DB that has not undergone learning processing (S19). If it determines that there is unprocessed training data (S19: YES), the control unit 11 returns to the process in step S17 and performs the processing in steps S17 to S18 on the unprocessed training data. If it determines that there is no unprocessed training data (S19: NO), the control unit 11 terminates the series of processes.

[0033] The learning process described above generates a terminal detection model M1 that, when a wiring image is input, outputs a label image in which the terminals in the image are surrounded by a bounding box. In the process described above, the training data generation process in steps S11 to S16 and the terminal detection model M1 generation process in steps S17 to S19 may be performed on separate devices. The terminal detection model M1 can be further optimized by repeatedly performing the learning process using the training data described above. In addition, even an already trained terminal detection model M1 can be retrained using the learning process described above to generate a model with further improved discrimination accuracy.

[0034] The color detection model M2 can also be generated by the same process as in Figure 4. In the generation process of the color detection model M2, in step S14, annotation is performed on the line drawing image by adding marks corresponding to each color region. Here, the control unit 11 receives regions of each color in the image from the user via the input unit 14, and generates a correct label image by adding marks of each color to each received region. Then, the control unit 11 generates training data by associating the line drawing image extracted in step S13 with the correct label image generated in step S14, and stores it in the training DB (not shown) for the color detection model M2 prepared in the storage unit 12. Subsequently, the control unit 11 trains the color detection model M2 using the training data stored in the training DB for the color detection model M2, thereby generating a color detection model M2 that outputs a label image in which each pixel in the image is classified by color when a line drawing image is input.

[0035] Note that the generation of training data for terminal detection model M1 and the generation of training data for color detection model M2 may be performed simultaneously. For example, in step S14 in Figure 4, annotation for the training data of terminal detection model M1 and annotation for the training data of color detection model M2 may be performed on the connection image. Then, in step S18 in Figure 4, the learning process for terminal detection model M1 and the learning process for color detection model M2 are performed based on the training data for terminal detection model M1 and color detection model M2 that have been generated in this way. This enables the generation of terminal detection model M1 and color detection model M2.

[0036] The following describes the process of determining whether the wiring status of the wires connected to each terminal of the electricity meter 21 is appropriate, based on images taken of the wiring status between the electricity meter 21 and the wires. Figure 5 is a flowchart showing an example of the wiring status determination process, Figure 6 is an explanatory diagram of the determination process, and Figure 7 is an explanatory diagram showing an example screen.

[0037] The control unit 11 (acquisition unit) of the information processing device 10 acquires a wiring image (S21) of the connection points between each terminal of the energy meter 21 and the electric wires. The wiring image is an image taken by a worker after the energy meter 21 has been newly installed or after the energy meter 21 has been replaced. In this embodiment, the determination of whether the wiring state is correct is made using a wiring image extracted from a captured image including the energy meter 21 and the electric wires, showing the connection points between each terminal of the energy meter 21 and the electric wires, but the captured image may be used as is. The control unit 11 may display the captured image and extract a region specified by the user (e.g., an acceptance inspector) via the input unit 14 to be used as the wiring image, or it may extract a region that has been pre-set on the captured image as the wiring image. The wiring image or captured image to be processed may be stored in the storage unit 12, acquired from another device via the communication unit 13, or read from a portable storage medium 10a by the reading unit 16.

[0038] The control unit 11 (first detection unit) detects the terminals of the energy meter in the image based on the wiring image (S22). Specifically, the control unit 11 inputs the wiring image to the terminal detection model M1 and acquires the label image output from the terminal detection model M1. For example, the control unit 11 acquires a label image in which each terminal is shown as a bounding box, as shown in Figure 6A. Next, the control unit 11 (second detection unit) detects the regions of each color in the image based on the wiring image (S23). Specifically, the control unit 11 inputs the wiring image to the color detection model M2 and acquires the label image output from the color detection model M2. For example, the control unit 11 acquires a label image in which each pixel is classified into each color region, as shown in Figure 6B.

[0039] Next, as shown in Figure 6C, the control unit 11 identifies the region of each terminal based on the terminal label image in the label image classified into color regions (S24). The terminal label image and the color region label image are the same size, and the positions of the terminals and wires coincide in the two label images. Furthermore, each pixel in the two label images is represented by coordinate values ​​of an XY coordinate system, for example, with the upper left corner of the image as the origin, the X-axis pointing to the right, and the Y-axis pointing downwards. The region of each terminal can be identified by the coordinate values ​​of the upper left and lower right pixels of each terminal region. Therefore, the control unit 11 identifies the coordinate values ​​of the upper left and lower right pixels of each terminal region in the terminal label image, and identifies the region of each terminal in the color region label image based on the identified coordinate values.

[0040] Next, the control unit 11 identifies the regions adjacent to the identified terminal regions in the color region label image (S25). For example, if the wiring image to be processed is an image in which the terminals are at the top and the wires connected to the terminals are at the bottom, the control unit 11 identifies the regions adjacent to the bottom of each terminal region, as shown in Figure 6D. Specifically, the lower edge of each terminal region is taken as the upper edge, and the region having the width of this upper edge and extending to the lower edge of the wiring image is identified as the region adjacent to the bottom of each terminal. The regions adjacent to each terminal identified here correspond to the regions of the wires connected to each terminal. Note that the wiring image to be processed is not limited to an image in which the terminals are at the top and the wires are at the bottom, but the region of the wires will be either the region on one side or the region on the other side in the direction that intersects (orthogonal to) the parallel direction of each terminal with respect to each terminal. Therefore, the control unit 11 may also be configured to identify one of the two regions adjacent in the intersecting direction in the parallel direction (the lower region in Figure 6D) as the region of the wires. In this case, the control unit 11 may identify the larger of the two regions as the region of the wires, or it may identify the region of the wires based on the arrangement order of the colors in the two regions. Furthermore, the control unit 11 may assign weights to each region to identify one of the two regions as the region of the wires. For example, if there is a rule that the wiring image to be processed should be an image in which the terminals are on the upper side and the wires are on the lower side, the control unit 11 can be configured to assign a larger weight to the lower region.

[0041] The control unit 11 (identification unit) then identifies the color of each region connected to each terminal from the label image of the color regions, thereby identifying the color of the wire connected to each terminal (S26). Here, the control unit 11 calculates the number or proportion of pixels classified by each color in the region connected to each terminal, i.e., the region of each wire, and identifies the color with the largest number or proportion of pixels as the color of the wire. The control unit 11 (determination unit) determines whether the wire connected to each terminal is appropriate based on the identified color of the wire connected to each terminal (S27). For example, as information indicating the correct (appropriate) wiring state, information in which the colors of the wires connected to each terminal are arranged from left to right when the terminal is at the top is stored in the storage unit 12. For example, as correct wiring information for terminals 1S, P1, P3, 3S, 3L, P2, 1L of the power meter 21, "green, red, black, blue, brown, white, yellow" is stored in the storage unit 12. The control unit 11 then determines whether the wires connected to each terminal are appropriate by comparing the arrangement order of the colors of the identified wires with the arrangement order of the colors in the correct wiring state. For example, the control unit 11 determines the color of a wire whose wiring state is not appropriate by identifying a color among the identified wires that is in a different arrangement order than the correct arrangement order. The information indicating the correct wiring state may be changed according to operations performed via the input unit 14.

[0042] The control unit 11 stores the determination result in the storage unit 12, associating it with the image ID assigned to the wiring image (S28). The control unit 11 may also generate a screen to notify the determination result and store it in the storage unit 12. For example, the control unit 11 generates a screen as shown in Figure 7A. The screen shown in Figure 7A displays the date and location of the installation (replacement) of the power meter 21, the meter number assigned to the power meter 21, the date and time of shooting, the captured image, and the determination result. If it is determined that there are wires with improper wiring, a message indicating a high possibility of incorrect wiring is displayed as the determination result, as shown in Figure 7A. In addition, on the screen in Figure 7A, wires that are determined to have improper wiring (brown and white wires) are enclosed in solid rectangles to indicate wires that are likely to be incorrectly wired. If it is determined that the wiring status of all wires is appropriate, a message indicating a low possibility of incorrect wiring may be displayed on the screen shown in Figure 7A, as shown in Figure 7B. Furthermore, if a wire color with an improper connection is identified, a message notifying the potentially incorrectly connected wire color may be displayed, as shown in Figure 7C. If the control unit 11 determines that there is a wire with an improper connection, it may display the generated screen on the display unit 15 to warn the acceptance inspector or operator.

[0043] The above-described process allows for the determination of the correctness of the wiring status at each terminal of the electricity meter 21 based on the captured images of the connection points between the electricity meter 21 and the power lines. Therefore, after the installation or replacement of the electricity meter 21, the acceptance inspector can consider the determination result from the information processing device 10 when checking the wiring status of the power lines to the electricity meter 21. For example, the acceptance inspector can reduce the chances of overlooking a wiring error by performing a more careful inspection of electricity meters 21 that the information processing device 10 has determined to have a high probability of being incorrectly wired. In this way, the level of caution in the acceptance inspection can be adjusted according to the likelihood (presence or absence) of incorrect wiring, thereby reducing the burden on the acceptance inspector.

[0044] In this embodiment, if the information processing device 10 determines that the wiring state is incorrect, it may be configured to notify the user of the color of the incorrectly connected wire or the terminal to which the incorrect wire is connected, in addition to the determination result. For example, a message such as "The wire of color ○ is highly likely to be incorrectly connected" may be displayed on the screen in Figure 7A. Also, for example, if a white wire is connected to a terminal to which a brown wire should be connected, and a brown wire is connected to a terminal to which a white wire should be connected, a message such as "The brown and white wires are highly likely to be swapped" may be displayed on the screen in Figure 7A. Furthermore, the information processing device 10 may be configured to provide advice on reconnecting wires that are incorrectly connected. For example, the storage unit 12 may store advice to be notified in order to correct the wiring state of each wire when the wiring state of each wire is incorrect, corresponding to each color of wire. The control unit 11 may then read the advice corresponding to the color of the wire that has been determined to be incorrectly connected from the storage unit 12 and display it on the screen shown in Figure 7A, for example. This allows us to provide not only the results of determining whether the wiring is appropriate, but also advice regarding wiring methods and other related matters.

[0045] The wires connected to the electricity meter 21 may experience deterioration of the outer protective coating due to aging. Therefore, it is preferable to include training data from images of electricity meters 21 connected to wires that have deteriorated in color in the training data used to train the color detection model M2. For example, if the years of use of the wires are associated with the images of the electricity meter 21 and the wires, the control unit 11 can extract images of electricity meters 21 connected to wires that have been used for a predetermined number of years (10 years, 15 years, etc.) or longer. Also, if, for example, the information processing device 10 manages the years of use of the wires connected to each electricity meter 21, the control unit 11 can extract images of electricity meters 21 connected to wires that have been used for a predetermined number of years or longer by identifying the years of use of the wires connected to the electricity meter 21 in each image. Using the captured images extracted in this way, the control unit 11 executes steps S12 to S19 in the color detection model M2 generation process shown in Figure 4, thereby realizing a color detection model M2 that can appropriately distinguish the color of each wire, even if the wires have deteriorated over time.

[0046] Specifically, the control unit 11 extracts a wiring image from the extracted captured image (an image showing deteriorated wires), including each terminal of the power meter 21 and the wires connected to each terminal (S13), annotates the extracted wiring image (S14), and generates a correct label image. Using this training data, the control unit 11 performs training on the color detection model M2 (S18). Specifically, the control unit 11 inputs the wiring image included in the training data into the color detection model M2 and obtains the output value (label image) output from the color detection model M2. Then, the control unit 11 compares the label image output from the color detection model M2 with the correct label image included in the training data and trains the color detection model M2 so that the two approximate each other. In the training process, the color detection model M2 optimizes the parameters used for calculations in the hidden layer and output layer, for example, by using a backpropagation method that sequentially updates from the output layer to the input layer.

[0047] In this embodiment, a configuration for determining the suitability of the wire connections to a high-voltage power meter 21 has been described, but the system is not limited to this configuration. For example, as shown in Figure 1, since the terminals of the power meter 21 and the terminals of the terminal section 22a of the terminal block 22 are arranged in the same order, the connection area of ​​the terminals of the power meter 21 and the connection area of ​​the terminal section 22a of the terminal block 22 will have the same image of the wire connections. Therefore, by processing similar to that in Figure 5, the suitability of the wire connections to each terminal of the terminal section 22a of the terminal block 22 can be determined based on the connection image of the terminal section 22a of the terminal block 22. Furthermore, the configuration of this embodiment can also be applied to a device for determining the suitability of the wire connections to a low-voltage power meter.

[0048] (Embodiment 2) This section describes an information processing device that, after detecting the terminals of the power meter 21 in the wiring image, extracts the region connected to the terminals from the wiring image, identifies the color of the wires connected to each terminal of the power meter 21 based on the extracted region, and determines whether the wiring information is correct. The information processing device of this embodiment has the same configuration as the information processing device 10 of Embodiment 1 shown in Figure 2, so a description of the configuration will be omitted.

[0049] Figure 8 is a flowchart showing an example of the wiring status determination process procedure in Embodiment 2, and Figure 9 is an explanatory diagram of the determination process. The process shown in Figure 8 is the same as the process shown in Figure 5, but with steps S31 to S33 added instead of steps S23 to S25. The same steps as in Figure 5 will not be explained.

[0050] In the information processing device 10 of this embodiment, the control unit 11 performs the same processing as in steps S21 to S22 in Figure 5. As a result, the control unit 11 acquires a wiring image as shown in Figure 9A and a wiring image (label image) after the terminals of the power meter have been detected, as shown in Figure 9B. In this embodiment, the control unit 11 identifies the region of each terminal in the terminal label image (S31). Here again, the control unit 11 identifies the coordinate values ​​of the upper left and lower right pixels of the region of each terminal using the coordinate values ​​of the XY coordinates. The control unit 11 identifies and extracts the region adjacent to each identified terminal region in the terminal label image (S32). The process of identifying the region adjacent to each terminal region here is the same as the process performed in step S25 in Figure 5. For example, as shown in Figure 9C, the control unit 11 identifies and extracts the region below each terminal region. Alternatively, the control unit 11 may extract the region adjacent to each terminal region from the wiring image acquired in step S21.

[0051] The control unit 11 detects regions of each color within the extracted region (region adjacent to the terminal region) based on the extracted region (S33). Specifically, the control unit 11 inputs the extracted region to the color detection model M2 and obtains a label image output from the color detection model M2. Here, the control unit 11 obtains a label image as shown in Figure 9D. Then, the control unit 11 identifies the color of each region adjacent to each terminal from the label image of the color regions, thereby identifying the color of the wire connected to each terminal (S26). Here, the control unit 11 calculates the number or proportion of pixels classified by each color in the region adjacent to each terminal extracted in step S32, and identifies the color with the largest number or proportion of pixels as the color of the wire connected to that terminal. After that, the control unit 11 executes the processing from step S27 onwards.

[0052] Through the process described above, in this embodiment as well, the appropriateness of the wiring status of the wires at each terminal of the power meter 21 can be determined based on the captured image of the connection point between the power meter 21 and the wires. In this embodiment, the color discrimination process is performed only on the region adjacent to each terminal in the wiring image, so that the color components can be distinguished with higher accuracy for the region of the wires connected to each terminal. The configuration of this embodiment is applicable to the information processing device 10 of Embodiment 1 described above, and when applied to the information processing device 10 of Embodiment 1, the same processes as in Embodiment 1 can be executed, except for the processes described in this embodiment, and the same effects can be obtained. Furthermore, in this embodiment as well, the modifications described as appropriate in Embodiment 1 described above can be applied.

[0053] (Embodiment 3) In this embodiment, an information processing device for determining the appropriateness of the wiring status of power lines for two power meters used for high-voltage power: a power purchase meter that measures the amount of electricity purchased from the power company, and a power sales meter that measures the amount of electricity sold to the power company. Figure 10 is an explanatory diagram showing an example of the installation of the power purchase meter and the power sales meter. The power purchase meter 21 and the power sales meter 24 are the same as the power meter 21 in Figure 1, and the wiring status of the power lines at each terminal of the power purchase meter 21 and each terminal of the terminal section 22b of the terminal block 22, and the wiring status of the power lines at each terminal of the power sales meter 24 and each terminal of the terminal section 25b of the terminal block 25 are the same as the power meter 21 and terminal block 22 in Figure 1.

[0054] For example, in a customer's facility with a solar power generation system, in addition to a transformer 23 and a power meter 21 for measuring the amount of electricity purchased from the power company, a power meter 24 for selling electricity generated by the solar power generation system to the power company is installed. The power meter 24 for selling electricity is connected to the transformer 23 via a terminal block 25, and the power meter 21 for purchasing electricity is connected to the transformer 23 via terminal blocks 22 and 25. Specifically, the wires connected to each terminal of the transformer 23 are connected to the corresponding terminals of the terminal section 25a of the terminal block 25. The corresponding terminals of the terminal section 25a of the terminal block 25 and the terminal section 22a of the terminal block 22 are connected via wires. As a result, the power meter 21 for purchasing electricity measures the amount of electricity supplied from the power source side to the load side in a high-voltage circuit based on the voltage and current supplied from the transformer 23 via terminal blocks 25 and 22. Furthermore, the electricity meter 24 for selling electricity measures the amount of electricity supplied from the solar power generation system to the power company's high-voltage circuit based on the voltage and current supplied from the transformer 23 via the terminal block 25.

[0055] In this embodiment, it is possible to determine whether the terminals and wires of the electricity meter 21 for purchasing electricity (dashed line A1 in Figure 10), the terminals and wires of the terminal section 22a of the terminal block 22 (dashed line A2 in Figure 10), the terminals and wires of the electricity meter 24 for selling electricity (dashed line A3 in Figure 10), the terminals and wires of the terminal section 25a of the terminal block 25 (dashed line A4 in Figure 10), and the terminals and wires of the transformer 23 (dashed line A5 in Figure 10) are suitable or unsuitable.

[0056] The information processing device of this embodiment has the same configuration as the information processing device 10 of Embodiment 1 shown in Figure 2, so a description of the configuration will be omitted. The information processing device 10 of this embodiment stores a connection information DB in the storage unit 12. Figure 11 is an explanatory diagram showing an example of the record layout of the connection information DB. Note that the connection information DB may be stored in other storage devices connected to the information processing device 10, or in other storage devices that the information processing device 10 can communicate with.

[0057] The wiring information database shown in Figure 11 includes an instrument ID column, an instrument information column, and a wiring information column, and stores information about instruments associated with the instrument ID. In this embodiment, the instruments include energy meters 21 and 24, a transformer 23, and terminal blocks 22 and 25. The instrument ID column stores identification information assigned to instruments and equipment that should be used to determine the wiring status of the wires. The instrument information column stores information about instruments and equipment that should be used to determine the wiring status of the wires. In the example in Figure 11, the instrument information column stores information indicating whether it is an energy meter 21 or 24, a transformer 23, or the terminal block 25 of the energy meter for selling electricity 24. The wiring information column stores the color of the wires to be connected to each terminal P1, P2, P3, 1S, 3S, 1L, 3L, and E of each instrument, and associates them with each terminal. For example, if the terminal is on the top and the wire is on the bottom, the sequence of wire colors to be connected may be stored, starting from the leftmost terminal. The contents of the wiring information DB are not limited to the example shown in Figure 11.

[0058] Figure 12 is a flowchart showing an example of the wiring status determination process procedure in Embodiment 3. The process shown in Figure 12 is the same as the process shown in Figure 5, but with steps S41 to S42 added between steps S26 and S27. The steps that are the same as in Figure 5 will not be explained.

[0059] In the information processing device 10 of this embodiment, the control unit 11 performs the same processing as in steps S21 to S26 in Figure 5. As a result, the control unit 11 identifies the color of the wire connected to each terminal in the wiring image. Next, the control unit 11 identifies the type of equipment in the wiring image (S41). The type of equipment may be specified by the user (e.g., an acceptance inspector) via the input unit 14, or it may be identified from the number of wires connected to it. Alternatively, the control unit 11 may perform object detection processing on the captured image before extracting the wiring image, and identify the type of equipment in the image from the shape of the equipment in the image, the number of wires connected to it, etc.

[0060] The control unit 11 reads the correct wiring information corresponding to the identified device from the wiring information DB (S42). Then, the control unit 11 determines whether the wires connected to each terminal are correct based on the color of the wires connected to each terminal identified in step S26 and the correct wiring information (S27). This allows the system to determine whether the wiring status of the wires in the wiring image is correct, even when there are multiple devices whose wiring status needs to be determined, based on the correct wiring information for each device. Note that the processing in steps S41 to S42 only needs to be completed before the processing in step S27 is executed.

[0061] The configuration of this embodiment is applicable not only to equipment to which power supply wires are connected (transformer 23, energy meters 21, 24), but also to various types of equipment having multiple terminals to which multiple wires are connected, for determining the appropriateness of the wire connection state. The configuration of this embodiment is applicable to the information processing device 10 of Embodiments 1 and 2 described above. When applied to the information processing device 10 of Embodiments 1 and 2, it is possible to perform the same processing as in Embodiments 1 and 2, except for the processing described in this embodiment, and the same effects can be obtained. Furthermore, in this embodiment as well, the modifications described as appropriate in Embodiments 1 and 2 described above can be applied.

[0062] (Embodiment 4) In the embodiments 1 to 3 described above, the process of determining the suitability of the wiring status of electrical wires from captured images of equipment such as the power meter 21 is not limited to a configuration in which the information processing device 10 performs the process locally. In this embodiment, an information processing system in which a server performs the process of determining the suitability of the wiring status of electrical wires is described. Figure 13 is an explanatory diagram showing an example of the configuration of the information processing system. The information processing system of this embodiment includes a server 10 and a worker terminal 30, and these devices are connected by communication via a network N.

[0063] Server 10 has the same configuration as the information processing device 10 in Embodiments 1 to 3 and is capable of performing the same processing. The worker terminal 30 is a terminal used by a worker who has performed installation or replacement work on, for example, an electricity meter 21, and can be a smartphone, tablet terminal, personal computer, etc. The worker terminal 30 has a control unit 31, a storage unit 32, a communication unit 33, an input unit 34, a display unit 35, a camera 36, ​​etc., and each of these units is connected via a bus. Since each of the units 31 to 35 of the worker terminal 30 has the same configuration as each of the units 11 to 15 of the information processing device 10 in Embodiment 1, a description of the configuration will be omitted.

[0064] The camera 36 is an imaging device having a lens and an image sensor, and performs imaging processing according to instructions from the control unit 31 and stores the obtained image data in the storage unit 32. The camera 36 may be built into the worker terminal 30 or it may be configured to be externally attached to the worker terminal 30.

[0065] In the information processing system of this embodiment, after completing the installation or replacement of the power meter 21, the worker takes a photograph of the power meter 21 using the worker terminal 30, and obtains a photographic image including the power meter 21 and the wires. The worker then sends the photographic image to the server 10 if they wish to determine whether the wiring status of the power meter 21 is appropriate. The server 10 determines whether the wiring status of the wires of the power meter 21 in the photographic image is appropriate by executing the process shown in Figure 5. If the information processing device 10 of Embodiment 2 is applied to the server 10 of this embodiment, the server 10 can determine whether the wiring status of the wires of the power meter 21 in the photographic image is appropriate by executing the process shown in Figure 8 based on the photographic image received from the worker terminal 30. Alternatively, if the information processing device 10 of Embodiment 3 is applied to the server 10 of this embodiment, the server 10 may execute the process shown in Figure 12 based on the photographic image received from the worker terminal 30 to determine whether the wiring status of the wires of the equipment in the photographic image is appropriate.

[0066] In this embodiment, the server 10 may, based on the captured image received from the worker terminal 30, determine whether the wiring status of the equipment in the image is appropriate, and then send a screen as shown in Figure 7A to the worker terminal 30 to provide feedback on the determination result to the worker. This allows the server 10 to determine whether the wiring status of the wires to equipment such as the power meter 21 is appropriate, and also allows the worker to be notified of the determination result while they are at the work site, enabling them to redo the wire connections early if the wiring status is not appropriate. In this case, the system may be configured to provide advice on how to redo the wires with incorrect wiring status. Thus, it is possible to avoid a time lag between the completion of equipment installation and the completion of rewiring. If the rewiring work is to be done at a later date, or if the work is to be done under a power outage, then process adjustments such as adjusting the power outage time and derivative tasks such as compensating for measurement differences due to incorrect wiring will occur. However, in this embodiment, since rewiring can be done during equipment installation or replacement, derivative tasks that occur when incorrect wiring occurs are avoided, and the burden on the worker can be reduced.

[0067] In this embodiment, the same processing as that performed by the information processing device 10 in embodiments 1 to 3 described above can be performed, and the same effects can be obtained. Furthermore, in this embodiment as well, the modifications described as appropriate in embodiments 1 to 3 described above can be applied.

[0068] The matters described in each embodiment can be combined with each other. Furthermore, the independent and dependent claims described in the claims can be combined with each other in any combination, regardless of the form of reference. In addition, the claims use a form in which claims referencing two or more other claims (multi-claim form), but are not limited to this. A form in which multi-claims referencing at least one multi-claim (multi-multi-claim) may also be used.

[0069] The embodiments disclosed herein should be considered in all respects to be illustrative and not restrictive. The scope of this disclosure is indicated by the claims, not in the sense described above, and all modifications within the meaning and scope equivalent to the claims are intended. [Explanation of symbols]

[0070] 10 Information Processing Devices 11 Control Unit 12 Storage section 13 Communications Department 21 Energy meter 22 Terminal block M1 Terminal Detection Model M2 color detection model

Claims

1. A device having multiple terminals to which multiple wires are connected is captured in an image showing the connection status between the device and the wires. When the aforementioned captured image is input to a first learning model that has been trained to detect the terminals in the captured image, the acquired captured image is input to detect each terminal of the device in the captured image. The acquired image is input to a second learning model that has been trained to detect regions of each color in the captured image when the captured image is input, and the regions of each color in the captured image are detected. Based on the position of each terminal in the captured image and the regions of each color in the captured image, the color of the wire connected to each terminal is identified. Based on the color of each identified wire, the appropriateness of the connection between each terminal and each wire is determined. A program that instructs a computer to perform a process.

2. From the acquired captured image, the region of the connection between the terminal and the electric wire is extracted. The extracted region is input to the first learning model to detect each terminal in the region. The extracted region is input to the second learning model to detect regions of each color within the region. The program according to claim 1, which causes the computer to perform the processing.

3. For each terminal in the captured image that was detected, a region is extracted that is connected in a direction that intersects the parallel direction of each terminal. The extracted region is input to the second learning model, and by detecting the regions of each color within the extracted region, the color of the wire connected to each terminal is identified. The program according to claim 1, which causes the computer to perform the processing.

4. Outputs the results of determining whether the connection status between each terminal and each wire is appropriate. The program according to claim 1 or 2, which causes the computer to perform the processing.

5. If it is determined that the connection between any terminal and the wire is not proper, advice will be output to ensure that the connection between the terminal and the wire is proper. The program according to claim 1 or 2, which causes the computer to perform the processing.

6. The aforementioned equipment consists of an electricity meter and a transformer. Determine whether the connection status of the wires connected to each terminal of the aforementioned electricity meter and transformer is appropriate. The program according to claim 1 or 2, which causes the computer to perform the processing.

7. A device having multiple terminals to which multiple wires are connected is captured in an image showing the connection status between the device and the wires. When the aforementioned captured image is input to a first learning model that has been trained to detect the terminals in the captured image, the acquired captured image is input to detect each terminal of the device in the captured image. The acquired image is input to a second learning model that has been trained to detect regions of each color in the captured image when the captured image is input, and the regions of each color in the captured image are detected. Based on the position of each terminal in the captured image and the regions of each color in the captured image, the color of the wire connected to each terminal is identified. Based on the color of each identified wire, the appropriateness of the connection between each terminal and each wire is determined. An information processing method in which a computer performs the processing.

8. An acquisition unit that acquires images of the connection state between a device having multiple terminals to which multiple wires are connected and the wires, A first learning model, which has been trained to detect the terminals in the captured image when the captured image is input, is provided with a first detection unit that inputs the acquired captured image and detects each terminal of the device in the captured image. A second learning model, which has been trained to detect regions of each color in the captured image when the captured image is input, is provided with a second detection unit that inputs the acquired captured image and detects regions of each color in the captured image. A identifying unit that identifies the color of the wire connected to each terminal based on the position of each terminal in the captured image and the regions of each color in the captured image, A determination unit that determines whether the connection status between each terminal and each wire is appropriate based on the color of each identified wire. An information processing device equipped with the following features.

9. First training data is acquired, which includes an image of the connection state between a device having multiple terminals to which multiple wires are connected and the wires, and information indicating each terminal of the device in the image. Using the acquired first training data, a first learning model is generated that detects each terminal of the device in the captured image when the captured image is input. Second training data is acquired, which includes the aforementioned captured image and information indicating the regions of each color in the captured image. Using the acquired second training data, a second learning model is generated that detects regions of each color in the captured image when the captured image is input. A model generation method in which a computer performs the processing.

Images