Inference device, wiring connection matching device, learning device, and wiring connection matching system

The use of an inference device to infer terminal block positions and dimensions from image data allows for efficient verification of electric wire connections in large-scale power distribution boards, addressing the inefficiencies of existing methods by reducing processing time and data load.

JP7584711B2Active Publication Date: 2024-11-15MITSUBISHI ELECTRIC CORP
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
JP2024528783
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2022-06-14
Filing Date
2023-06-08
Publication Date
2024-11-15
Estimated Expiration
2043-06-08

AI Technical Summary

Technical Problem

Existing methods for verifying the connection state of electric wires in large-scale power distribution boards are inefficient due to the large amount of image data that needs to be processed, leading to prolonged processing times and difficulty in completing the verification within a reasonable timeframe.

Method used

An inference device that uses a trained model to infer the installation position, rotation angle, and dimensions of terminal blocks from image data, allowing for the extraction of a partial image area containing wire identification information, thereby reducing the amount of image data that needs processing.

Benefits of technology

This approach effectively verifies the connection state of electric wires while minimizing processing time and reducing the load on image data processing, enabling efficient verification using commonly available office personal computers and image processing units.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007584711000001
    Figure 0007584711000001
  • Figure 0007584711000002
    Figure 0007584711000002
  • Figure 0007584711000003
    Figure 0007584711000003
Patent Text Reader

Abstract

The technology disclosed in the present specification is for effectively verifying the connection state of a power line, while suppressing a lengthening of processing time. An inference device relating to the technology disclosed in the present specification comprises: an image data acquisition unit that acquires captured image data; and an inference unit that uses a first trained model, which infers, from the image data acquired by the image data acquisition unit, the installation position, the rotation angle, the length and breadth dimensions, and model information of a terminal block, to infer and outputs the installation position, the rotation angle, the length and breadth dimensions, and the model information of the terminal block from the image data.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The technology disclosed in this specification relates to a technology for checking the connection state of electric wires. [Background technology]

[0002] In the assembly or inspection process at the manufacturing site of switchboards, workers may be required to perform wiring work in which multiple electric wires are connected to a terminal block. This wiring work poses the problem that incorrect connections due to human error by the worker can lead to manufacturing defects.

[0003] However, it is not easy and takes a lot of time to check whether the multiple electric wires of a switchboard are connected correctly. Therefore, reducing the time required to check whether the multiple electric wires are connected correctly is effective for efficiently manufacturing switchboards.

[0004] To address this problem, for example, Patent Document 1 discloses a configuration in which image data captured inside a distribution board using a camera built into a mobile terminal is sent to a data processing server, where electric wire identification information is read from the image data using image processing, and the electric wire identification information is compared with electric wire connection data that has been prepared in advance from circuit design information and stored in the server to determine whether the electric wires are connected correctly. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent Publication No. 2021-129474 Summary of the Invention [Problem to be solved by the invention]

[0006] However, with the configuration described in Patent Document 1, when a large-scale distribution board is the target, the range to be imaged becomes wide, and when the number of connected electric wires increases, the amount of image data to be processed becomes enormous, making it difficult to perform image processing by combining a normally used office personal computer, a commercially available image processing unit, and image processing software. Alternatively, when an attempt is made to perform image processing by combining a normally used office personal computer, a commercially available image processing unit, and software, the processing time becomes very long, and the desired processing cannot be completed within a reasonable processing time.

[0007] The technology disclosed in the present specification has been made in consideration of the problems described above, and is a technology for effectively verifying the connection status of electric wires while preventing processing times from becoming too long. [Means for solving the problem]

[0008] An inference device that is a first aspect of the technology disclosed in the present specification is a wiring connection matching device that matches design data based on a wiring connection diagram that shows the installation location of at least one terminal block in a distribution board and the connection locations of electric wires to the terminal block with image data captured inside the distribution board to match the connection state of the electric wires, and when reading electric wire identification information assigned to the electric wire connected to the terminal block in the distribution board from the image data, the inference device infers the installation position, rotation angle, length and width dimensions and model information of the terminal block to which the electric wire is connected in order to extract a partial image area including the electric wire identification information, and includes an image data acquisition unit that acquires the captured image data, and an inference unit that infers and outputs the installation position, rotation angle, length and width dimensions and model information of the terminal block from the image data using a first learned model for inferring the installation position, rotation angle, length and width dimensions and model information of the terminal block from the image data acquired by the image data acquisition unit. Effect of the Invention

[0009] According to at least the first aspect of the technique disclosed in the present specification, it is possible to effectively verify the connection state of electric wires while preventing the processing time from becoming long.

[0010] Furthermore, objects, features, aspects and advantages associated with the technology disclosed herein will become more apparent from the detailed description set forth below and the accompanying drawings. [Brief description of the drawings]

[0011] [Figure 1] FIG. 2 is a plan view showing an example of a terminal block in a switchboard according to an embodiment. [Diagram 2] FIG. 13 is a diagram showing an example of a band mark according to an embodiment. [Diagram 3] FIG. 13 is a diagram showing an example of a band mark according to an embodiment. [Figure 4] 1 is a diagram conceptually illustrating an example of the configuration of a wiring connection matching device according to an embodiment; [Diagram 5] FIG. 13 is a diagram illustrating an example of design data. [Figure 6] 13 is a flowchart showing an example of processing by the inference device. [Figure 7] 13 is a flowchart showing an example of processing in an image data processing unit. [Figure 8] 11A and 11B are diagrams illustrating an example of a method for determining a partial image region. [Figure 9] 11A and 11B are diagrams illustrating an example of a method for determining a partial image region. [Figure 10] FIG. 13 is a diagram showing an example of a matching result. [Figure 11] FIG. 13 is a diagram showing an example of a matching result. [Figure 12] FIG. 13 is a diagram showing an example of a mismatched portion. [Figure 13] FIG. 11 is a diagram illustrating an example of the configuration of a learning device for providing a learned model to a learned model storage unit in a wiring connection matching device. [Figure 14] FIG. 1 is a diagram illustrating an example of a three-layer neural network. [Figure 15] 13 is a flowchart relating to a learning process of the learning device. [Figure 16] 1 is a diagram conceptually illustrating an example of the configuration of a wiring connection matching system according to an embodiment; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, the embodiments will be described with reference to the accompanying drawings. In the following embodiments, detailed features are shown for the purpose of explaining the technology, but they are merely examples and are not necessarily all essential features for enabling the embodiments to be implemented.

[0013] The drawings are schematic, and for the convenience of explanation, the configuration may be omitted or simplified as appropriate. The size and positional relationship of the configurations shown in different drawings are not necessarily described accurately, and may be changed as appropriate. Hatching may be added to drawings such as plan views that are not cross-sectional views, in order to make it easier to understand the contents of the embodiment.

[0014] In the following description, the same components are denoted by the same reference numerals, and their names and functions are also the same. Therefore, detailed descriptions thereof may be omitted to avoid duplication.

[0015] Furthermore, in the description in this specification, when a certain component is described as "comprising," "including," or "having," unless otherwise specified, this is not an exclusive expression that excludes the presence of other components.

[0016] Furthermore, even if ordinal numbers such as "first" or "second" are used in the description of this specification, these terms are used for convenience to facilitate understanding of the contents of the embodiments, and the contents of the embodiments are not limited to the orders that may result from these ordinal numbers.

[0017] <First embodiment> The inference device and the wiring connection verification device according to this embodiment will be described below.

[0018] Fig. 1 is a plan view showing an example of a terminal block in a switchboard according to the present embodiment. As shown in Fig. 1, a terminal block 1 is housed in the switchboard, and a worker performs wiring work to connect multiple electric wires 2 to one terminal block 1.

[0019] The position, angle, and type of the terminal block 1, and the number of electric wires 2 connected to one terminal block 1 may be any desired embodiment.

[0020] 2 and 3 are diagrams showing examples of band marks according to this embodiment. As shown in the example in Fig. 2, a crimp terminal 4 for bonding to the terminal block 1 is attached to the end of an electric wire 2a connected to the terminal block 1, and a band mark 3a is attached in a position close to the crimp terminal 4. The band mark 3a carries electric wire identification information consisting of text information. As shown in the example in Fig. 3, a crimp terminal 4 for bonding to the terminal block 1 is attached to the end of an electric wire 2b connected to the terminal block 1, and a band mark 3b is attached in a position close to the crimp terminal 4. The band mark 3b carries electric wire identification information consisting of text information and a QR code (registered trademark).

[0021] The wire identification information, examples of which are shown in Figures 2 and 3, is information that uniquely identifies the wire 2a (wire 2b), and is at least one of a character string in which a wire number, which is a number specific to the wire, is recorded, a QR code (registered trademark), or a barcode.

[0022] Although the band mark 3a (or band mark 3b) on which the electric wire identification information is written has a cylindrical shape in Figs. 2 and 3, the shape of the band mark can be changed arbitrarily as long as it is visible.

[0023] When connecting an electric wire having a band mark 3a (or band mark 3b) attached to the terminal block 1, the worker connects the electric wire to the terminal block 1 so that the electric wire identification information is visible to the imaging unit 51 of the wiring connection matching device 5 (both described later).

[0024] At this time, a part of the electric wire identification information may be hidden as viewed from the imaging unit 51 to the extent that a recognition data creation unit 53d of the wiring connection matching device 5 described later can perform appropriate correction processing. When correcting the electric wire identification information, for example, artificial intelligence (AI) technology can be utilized to learn the correction results, thereby improving the identification accuracy, including for electric wire identification information in which part of a character string is hidden.

[0025] The combination of QR codes, bar codes, and character strings, and the number of each of them, may be arbitrary. That is, only a QR code, only a bar code, or only a character string may be printed as the electric wire identification information, or all of a QR code, a bar code, and a character string may be printed as the electric wire identification information. Also, a mixture of multiple QR codes, multiple bar codes, and multiple character strings may be printed as the electric wire identification information.

[0026] <Configuration of the wiring connection verification device> Fig. 4 is a diagram conceptually illustrating an example of the configuration of a wiring connection matching device according to this embodiment. As illustrated in Fig. 4, the wiring connection matching device 5 includes an imaging unit 51, a display 52, a control unit 53, a design data storage unit 54, a terminal block model information storage unit 55, a recognition data storage unit 56, a matching result storage unit 57, and a trained model storage unit 58.

[0027] The imaging unit 51 is, for example, a camera, and captures an image of the switchboard housing.

[0028] The display 52 has a function of accepting touch operations on the screen and a function of displaying various information.

[0029] The design data stored in the design data storage unit 54 is created corresponding to each terminal block 1 installed in the switchboard. The design data is created based on a wiring connection diagram showing the installation location of at least one terminal block 1 in the switchboard and the connection location of the electric wire 2 to the terminal block 1.

[0030] Fig. 5 is a diagram showing an example of design data. As shown in the example of Fig. 5, the design data of the terminal block 1 includes a manufacturing number, a drawing number, a board number, a terminal block device number, a number of the terminal block 1 (terminal block number), a line number (an internal terminal number and an external terminal number), and whether or not the terminal block needs to be inspected.

[0031] The serial number is a number indicating the serial number of the product in which the distribution board is mounted. The drawing number is a number indicating the manufacturing drawing of the product. The board number is a number indicating the distribution board. The terminal block device number is a number including position information of the terminal block 1 that is uniquely assigned to each terminal block 1 arranged in the distribution board. The terminal block number is a number indicating the terminal block 1. The necessity of terminal block inspection is information for distinguishing between terminal blocks 1 to be inspected and terminal blocks 1 that are not to be inspected when automatic inspection is performed in the wiring connection matching device 5.

[0032] When the design data acquisition unit 53a acquires the design data, the design data acquisition unit 53a stores the design data in the design data storage unit 54 in a database.

[0033] The control unit 53 is a calculator mainly composed of a microcomputer having a central processing unit (CPU), a read only memory (ROM), a random access memory (RAM), an input unit, an output unit, etc., and, depending on the functions of the software, is provided with a design data acquisition unit 53a, an inference device 53b having an image data acquisition unit 53b1 and an inference unit 53b2, an image data processing unit 53c, a recognition data creation unit 53d, a data matching unit 53e, and a matching result output unit 53f.

[0034] The worker operates the display 52 to input the serial number and the panel number into the control unit 53. The control unit 53 stores the input serial number and panel number. In this case, the date and time when the worker operates the wiring connection verification device 5 may be stored as an operation record linked to the above information.

[0035] When the imaging unit 51 captures an image of the switchboard housing, the control unit 53 associates the captured image data with the serial number and the board number.

[0036] The trained model storage unit 58 stores an inference model trained to output the installation position, rotation angle, length and width dimensions, and model information of the terminal block 1. The installation position is coordinate information from the image origin of the inferred terminal block 1. The rotation angle is the rotation angle from the reference axis of the inferred terminal block 1. The length and width dimensions are the vertical and horizontal lengths of the detected terminal block 1. The model information is information indicating the model of the terminal block 1.

[0037] The inference device 53b includes an image data acquisition unit 53b1 and an inference unit 53b2. The image data acquisition unit 53b1 acquires the image of the switchboard housing captured by the imaging unit 51. The inference unit 53b2 infers the installation position, rotation angle, vertical and horizontal dimensions, and model information of the terminal block 1 obtained from the image data acquired by the image data acquisition unit 53b1 using the trained model stored in the trained model storage unit 58. Then, the inference unit 53b2 outputs the inference result to the image data processing unit 53c. That is, by inputting the switchboard housing image acquired by the image data acquisition unit 53b1 to this trained model, the installation position, rotation angle, vertical and horizontal dimensions, and model information of the terminal block 1 inferred from the switchboard housing image can be output.

[0038] Next, a process for obtaining the installation position, rotation angle, length and width dimensions, and model information of the terminal block 1 using the inference device 53b will be described with reference to Fig. 6. Here, Fig. 6 is a flowchart showing an example of the process of the inference device.

[0039] First, in step ST1, the image data acquisition unit 53b1 acquires an image of the switchboard housing captured by the imaging unit 51.

[0040] Next, in step ST2, the inference unit 53b2 inputs the distribution board casing image stored in the learned model memory unit 58 into the learned model, and obtains the installation position, rotation angle, length and width dimensions, and model information of the terminal block 1.

[0041] Next, in step ST3, the inference unit 53b2 outputs the installation position, rotation angle, length and width dimensions, and model information of the terminal block 1 obtained from the trained model to the image data processing unit 53c of the wiring connection matching device 5.

[0042] Next, based on the information output from the inference device 53b, the image data processing unit 53c cuts out a partial image area including the electric wire identification information. Fig. 7 is a flowchart showing an example of the process in the image data processing unit 53c.

[0043] First, in step ST4, the image data processing unit 53c obtains, from the terminal block model information memory unit 55, positional relationship information indicating at what position the area containing the identification information is located based on the reference point of the terminal block 1, from the model information of the terminal block 1 output by the inference device 53b.

[0044] Next, in step ST5, the image data processing unit 53c specifies a partial image area based on the positional relationship information acquired from the terminal block model information storage unit 55 and the installation position, rotation angle, and vertical and horizontal dimensions of the terminal block 1 output by the inference device 53b. That is, the image data processing unit 53c determines the range of image processing (corresponding to the partial image area) based on the installation position, rotation angle, vertical and horizontal dimensions, and model information of the terminal block 1.

[0045] 8 and 9 are diagrams showing examples of methods for determining partial image regions.

[0046] The dimensions h and w in Fig. 8 are the length and width dimensions of the terminal block 1, respectively. The dimensions h and w are the dimensions of the area 100 corresponding to the terminal block 1 inferred by the inference device 53b, and are an area that does not include the multiple electric wires 2 connected to the terminal block 1. The angle θ in Fig. 8 is the rotation angle of the terminal block 1 from the reference axis (x-axis or y-axis). In Fig. 8, the y-axis is the reference axis. The coordinate O1 in Fig. 8 is the coordinate corresponding to the reference point of the terminal block 1, and is set to (0,0) here.

[0047] Magnifications αh and αw in Fig. 9 are the magnifications for the dimensions h and w in Fig. 8, respectively. Coordinate O2 in Fig. 9 is the coordinate corresponding to the reference point of partial image area 102, and is assumed to be (dx, dy) here. dx corresponds to the amount of movement in the x-axis direction from coordinate O1 to coordinate O2. Similarly, dy corresponds to the amount of movement in the y-axis direction from coordinate O1 to coordinate O2. Magnifications αh, αw, dx, and dy are stored in terminal block model information storage unit 55 for each model of terminal block 1, and correspond to positional relationship information.

[0048] First, as shown in step ST4 of FIG. 7, the image data processing unit 53c obtains the magnifications αh, αw, dx and dy corresponding to the model from the terminal block model information storage unit 55 based on the model information output from the inference device 53b.

[0049] Next, the image data processing unit 53c performs appropriate correction using the angle θ output from the inference device 53b, and determines the coordinate O2, which is the reference point of the partial image area 102, by moving the coordinate O1 by dx in the x-axis direction and dy in the y-axis direction.

[0050] Thereafter, the image data processing unit 53c determines the size of the partial image area 102 by multiplying the dimension h output from the inference device 53b by a magnification αh and the dimension w by a magnification αw.

[0051] Next, in step ST6, the image data processing unit 53c extracts the partial image area 102 specified in step ST5 as partial image data (image data).

[0052] Then, the recognition data creation unit 53d creates recognition data in a format that can be collated with the design data, based on the information output from the inference unit 53b2 and information obtained by image analysis of the partial image data (image data) extracted by the image data processing unit 53c. That is, if the captured image data includes the terminal block 1 position information (terminal block position information) for deriving the terminal block device number and the terminal block number, or the electric wire identification information for deriving the line number, the recognition data creation unit 53d creates recognition data including the terminal block device number, the terminal block number, and the line number by image analysis of the image data.

[0053] When the recognition data creating unit 53d creates the recognition data, the recognition data creating unit 53d stores the recognition data in a database in the recognition data storage unit 56. At this time, if the electric wire identification information is a character string, the character string may be recognized using, for example, a character recognition inference device.

[0054] When a collation instruction signal is sent to the control unit 53 by the worker operating the display 52, the data collation unit 53e reads out design data corresponding to the serial number and board number associated with the image data from the design data storage unit 54, and also reads out recognition data from the recognition data storage unit 56. The data collation unit 53e then collates the design data with the recognition data. That is, the data collation unit 53e collates the design data including the terminal block device number, the terminal block number, and the line number with the recognition data. Note that in the collation in the data collation unit 53e, at least one of the position information of the terminal block 1, the number of the terminal block 1, and the line number is used as the collation target to collate the design data with the recognition data.

[0055] In this case, if the electric wire 2 is properly connected to the terminal block 1, the design data and the recognition data will match. On the other hand, if the electric wire 2 is not properly connected to the terminal block 1, the design data and the recognition data will not match. That is, the data collating unit 53e collates the design data with the recognition data to determine whether the connection state of the electric wire 2 is normal or not.

[0056] The collation result output unit 53f stores the collation result of the data collation unit 53e in a database in the collation result storage unit 57, and also displays the collation result on the display 52, for example.

[0057] Fig. 10 is a diagram showing an example of the collation result. If the design data and the recognition data match and the collation result is positive, the control unit 53 causes the display 52 to display, as shown in the example of Fig. 10, the serial number and panel number input by the worker through the display 52, as well as the word "Passed", for example, indicating that the collation result is positive.

[0058] On the other hand, if the design data and the recognition data do not match and the comparison result is negative, the control unit 53 causes the display 52 to display, as shown in Fig. 11, the serial number and panel number input by the worker through the display 52, as well as the word "Failed" and the mismatched parts, indicating that the comparison result is negative. Note that Fig. 11 is a diagram showing an example of the comparison result.

[0059] FIG. 12 is a diagram showing an example of a mismatched portion. For example, if an electric wire 2 to be connected to the wire number "15A" and an electric wire 2 to be connected to the wire number "16A" are mistakenly connected in the wrong order in the terminal block 1, the corresponding design data and recognition data will not match. In such a case, as shown in FIG. 12, the control unit 53 causes the display 52 to display information indicating that the electric wire 2 to be connected to the wire number "15A" and the electric wire 2 to be connected to the wire number "16A" are incorrect (including the terminal block device number, terminal block number, and design data and recognition data of the wire numbers of those electric wires 2). At this time, an area corresponding to the mismatched portion may be displayed from the image data, for example, to make it easier for the worker to grasp the error.

[0060] The timing for the comparison may be when the recognition data and the corresponding design data are prepared, or may be when instructed by the operator after the recognition data and the corresponding design data are prepared.

[0061] The wiring connection matching device 5 including the inference device 53b according to this embodiment can infer the installation position, rotation angle, length and width dimensions, and model information of the terminal block 1 in the switchboard housing image. Then, the image data processing unit 53c determines the partial image area 102 including the optimal electric wire identification information based on the inference result.

[0062] Since only image data corresponding to partial image region 102 is image-processed, the image data processing load can be reduced. Therefore, even when a worker who connects electric wires 2 to terminal block 1 of a switchboard proceeds with the wiring work using wiring connection checking device 5 and checks the wiring connections of a large-scale switchboard that could not be processed by conventional methods, according to this embodiment, a system for checking the wiring connections can be constructed by combining a commonly used office personal computer, a commercially available image processing unit, and software.

[0063] Furthermore, if the date and time of operation are recorded at the same time that the worker operates the display 52 to input information, the collation results are stored in a database, so that manufacturing history information such as inspection results or inspection dates and times can be searched for and viewed later using the serial number as a key.

[0064] In addition, the inference device 53b in this embodiment is configured to store the learned model in a learned model memory unit 58 provided outside the inference device 53b, but the learned model memory unit 58 may be provided inside the inference device 53b.

[0065] In the wiring connection checking device 5 according to this embodiment, after the inference device 53b infers the installation position, rotation angle, length and width dimensions, and model information of the terminal block 1, the inferred terminal block 1 may be displayed on the display 52 so that the worker can select the terminal block 1 not to be inspected. This allows the worker to exclude the terminal block 1 that has passed the reinspection and continue the subsequent processing, thereby shortening the checking time.

[0066] Furthermore, the data collation unit 53e may read the terminal block inspection necessity of the design data at the time of collation, and may exclude the parts not requiring collation from the objects of collation in advance. This allows, for example, the electric wire 2 that is difficult to inspect because it is in a blind spot from the photographing angle to be set as not requiring collation in advance, thereby shortening the collation time.

[0067] Furthermore, in the inference device 53b according to this embodiment, when the number of connection points of the electric wires 2 in the switchboard to be imaged exceeds 100, a central processing unit capable of realizing a processing speed of about 90 gigaflops can be used to operate the inference unit 53b2 of the inference device 53b. In this case, the hardware constituting the inference device 53b according to this embodiment does not require a dedicated central processing unit capable of realizing a high processing speed, and the inference device 53b according to this embodiment can be constituted by using a commercially available personal computer that is widely used.

[0068] Furthermore, inference unit 53b2 of inference device 53b according to this embodiment may output partial image area 102 including electric wire identification information derived from the installation position, rotation angle, length and width dimensions, and model information of terminal block 1, instead of outputting the installation position, rotation angle, length and width dimensions, and model information of terminal block 1. In other words, partial image area 102 may be output directly from the switchboard image without passing through the installation position, rotation angle, length and width dimensions, and model information of terminal block 1.

[0069] In this way, the trained model obtained from trained model storage unit 58 functions as an inference device that infers and outputs partial image area 102 including electric wire identification information from the switchboard housing image captured by imaging unit 51. Then, according to the trained model, it becomes possible to output partial image area 102 that has been weighted and optimized by a neural network through supervised learning, rather than partial image area 102 calculated based on an analytical formula with reference to the installation position, rotation angle, vertical and horizontal dimensions, and model information of terminal block 1.

[0070] <Second embodiment> A learning device according to the present embodiment will be described. In the following description, components similar to those described in the above embodiment are illustrated with the same reference numerals, and detailed descriptions thereof will be omitted as appropriate.

[0071] <About the configuration of the learning device> 13 is a diagram showing an example of the configuration of a learning device 6 for providing a learned model to the learned model storage unit 58 in the wiring connection matching device 5. As shown in the example in FIG. 13, the learning device 6 includes a data acquisition unit 61 and a model generation unit 62 connected to the learned model storage unit 58.

[0072] The data acquisition unit 61 acquires the image of the switchboard housing and the terminal block label (correct answer) as learning data. Here, the terminal block label (correct answer) is information indicating the correct installation position, the correct rotation angle, the correct length and width dimensions, and the correct model information of the terminal block 1.

[0073] The model generation unit 62 learns the installation position, rotation angle, length and width dimensions, and model information of the terminal block 1 to be detected, based on learning data created based on a combination of the switchboard housing image and terminal block label (correct answer) output from the data acquisition unit 61. That is, a learned model is generated that infers the optimal installation position, rotation angle, length and width dimensions, and model information of the terminal block 1 to be detected from the switchboard housing image and terminal block label (correct answer) of the wiring connection matching device 5. Here, the learning data is data in which the switchboard housing image and the terminal block label (correct answer) are associated with each other.

[0074] The learning device 6 and the inference device 53b are used to learn the installation position, rotation angle, length and width dimensions, and model information of the terminal block 1 to be detected by the wiring connection matching device 5, but may be devices connected to the wiring connection matching device 5 via a network and separate from the wiring connection matching device 5. The learning device 6 and the inference device 53b may also be built into the wiring connection matching device 5. Furthermore, the learning device 6 and the inference device 53b may exist on a cloud server.

[0075] A known supervised learning algorithm may be used as the learning algorithm used by the model generating unit 62. As an example, a case where a neural network is applied will be described.

[0076] The model generation unit 62 learns the installation position, rotation angle, length and width dimensions, and model information of the terminal block 1 to be detected, for example, by so-called supervised learning according to a neural network model. Here, supervised learning refers to a method in which a set of data of input and result (label) is given to the learning device 6, and the features of the learning data are learned and the result is inferred from the input.

[0077] A neural network is composed of an input layer consisting of multiple neurons, an intermediate layer (hidden layer) consisting of multiple neurons, and an output layer consisting of multiple neurons. The intermediate layer may be one layer or two or more layers.

[0078] Fig. 14 is a diagram showing an example of a three-layered neural network. For example, in the case of a three-layered neural network as shown in Fig. 14, when multiple inputs are input to the input layer (X1, X2, and X3), the values ​​are multiplied by weights W1 (W11, W12, W13, W14, W15, and W16) and input to the intermediate layer (Y1 and Y2), and the results are further multiplied by weights W2 (W21, W22, W23, W24, W25, and W26) and output from the output layer (Z1, Z2, and Z3). This output result changes depending on the values ​​of weights W1 and W2.

[0079] In this embodiment, the neural network learns the installation position, rotation angle, length and width dimensions and model information of the terminal block 1 to be detected by so-called supervised learning, according to learning data created based on a combination of the distribution board housing image and terminal block label (correct answer) acquired by the data acquisition unit 61.

[0080] In other words, the neural network learns by inputting an image of a switchboard casing to the input layer and adjusting the weights W1 and W2 so that the result output from the output layer approaches the terminal block label (correct answer).

[0081] The model generation unit 62 executes the above-described learning to generate and output a trained model.

[0082] The trained model storage unit 58 stores the trained model output from the model generation unit 62.

[0083] Next, the learning process of the learning device 6 will be described with reference to Fig. 15. Here, Fig. 15 is a flowchart relating to the learning process of the learning device 6.

[0084] First, in step ST7, the data acquisition unit 61 acquires a switchboard housing image and a terminal block label (correct answer). In Fig. 15, the switchboard housing image and the terminal block label (correct answer) are acquired simultaneously, but as long as the switchboard housing image and the terminal block label (correct answer) can be input in association with each other, the data of the switchboard housing image and the terminal block label (correct answer) may be acquired at different times.

[0085] Next, in step ST8, the model generation unit 62 learns the installation position, rotation angle, length and width dimensions and model information of the terminal block 1 to be detected by so-called supervised learning according to the learning data created based on the combination of the distribution board housing image and the terminal block label (correct answer) acquired by the data acquisition unit 61, and generates a learned model.

[0086] Next, in step ST9, the trained model storage unit 58 stores the trained model generated by the model generation unit 62.

[0087] The learning device 6, which is a device that performs machine learning according to this embodiment, autonomously learns the image of the switchboard housing and the installation position, rotation angle, vertical and horizontal dimensions, and model information of the terminal block 1 that constitutes the switchboard. Then, based on the learning result, the image data processing unit 53c determines the partial image area 102 that includes the optimal electric wire identification information and can perform image processing on only the partial image area 102, thereby reducing the processing load of the image data.

[0088] Therefore, even when verifying the wiring connections of a large-scale distribution board that could not be processed by conventional methods, according to this embodiment, the wiring connections can be verified by combining a commonly used office personal computer, a commercially available image processing unit, and software.

[0089] Alternatively, the electric wire identification information may be a character string consisting of alphanumeric characters and symbols, and image processing may be used to read the electric wire identification information from the character string. In this case, the functions of a personal computer or an image processing unit and commercially available software, which are hardware components constituting the learning device 6 according to this embodiment, can be diverted as a function for reading the electric wire identification information, thereby saving the cost of constructing additional hardware.

[0090] Furthermore, in a wiring identification and matching device in which the electric wire identification information is a character string consisting of alphanumeric characters and symbols, the capacity of the storage device implemented in the image processing unit used for image processing may be, for example, about 8 gigabytes. The reason for this is that when recognizing the electric wire identification information, all image data of the image of the switchboard housing is not processed, but only the optimal partial image area 102 including the electric wire identification information is processed. For this reason, a dedicated image processing unit equipped with a large-capacity storage device is not required, and the learning device 6 according to this embodiment can be configured using a commercially available image processing unit that is widely used.

[0091] Furthermore, in the learning device 6 according to this embodiment, instead of the terminal block label (correct answer), a partial image area label (correct answer) including electric wire identification information derived from the installation position, rotation angle, length and width dimensions, and model information of the terminal block 1 may be used. In other words, the partial image area 102 may be output directly from the switchboard image without passing through the installation position, rotation angle, length and width dimensions, and model information of the terminal block 1.

[0092] In this case, the trained model output from the model generation unit 62 constitutes an inference device 53b that infers and outputs a partial image area 102 including electric wire identification information from the switchboard housing image. According to the trained model, it becomes possible to output a partial image area 102 that has been weighted and optimized by a neural network through supervised learning, rather than a partial image area 102 calculated based on an analytical formula with reference to the installation position, rotation angle, vertical and horizontal dimensions, and model information of the terminal block 1.

[0093] <Third embodiment> A wiring connection matching system according to the present embodiment will be described. In the following description, components similar to those described in the above embodiment are illustrated with the same reference numerals, and detailed description thereof will be omitted as appropriate.

[0094] <About the configuration of the wiring connection matching system> Fig. 16 is a diagram conceptually showing an example of the configuration of a wiring connection matching system according to this embodiment. As shown in Fig. 16, in the wiring connection matching system, a mobile terminal 7 carried by an operator and a server 8 are configured to be able to communicate data with each other. Also, in the wiring connection matching system, a design support system 9 (CAD) used when constructing design data based on a wiring connection diagram showing a method of connecting an electric wire 2 to a terminal block 1 of a switchboard and the server 8 are configured to be able to communicate data with each other.

[0095] Furthermore, there may be a PC terminal for remotely inputting data or monitoring the results, and the PC terminal and the server 8 may be configured to be capable of data communication. In this case, the mobile terminal 7, the design support system 9, the PC terminal, and the server 8 perform data communication by wireless connection, for example, wireless LAN or Bluetooth (registered trademark). Note that the mobile terminal 7, the design support system 9, the PC terminal, and the server 8 may perform data communication by wireless connection via a public communication line including the Internet or a virtual private network (VPN), or may perform data communication by wired connection via a connection cable without being limited to wireless connection.

[0096] The mobile terminal 7 is a terminal operated by a worker, and is, for example, a smartphone or a tablet. The mobile terminal 7 includes a control unit 71 that controls the overall operation of the mobile terminal 7, a data communication unit 72 that controls data communication with the server 8, a touch panel display 73 having a function of accepting touch operations on the screen or a function of displaying various information, a digital camera 74 including an imaging element such as a CMOS sensor or a CCD sensor, a microphone 75 that inputs the voice to be transmitted when making a phone call, and a speaker 76 that outputs the voice to be received when making a phone call. The digital camera 74 corresponds to the imaging unit.

[0097] The control unit 71 is mainly composed of a microcomputer having a CPU, ROM, RAM, I / O, etc., and executes a stored program to control the operation of the mobile terminal 7. When the worker operates the display 73 to input a serial number and a panel number, the control unit 71 stores the input serial number and panel number. At that time, information linked to the date and time when the worker operated the mobile terminal 7 may be stored as an operation record. When the control unit 71 captures an image of the band mark 3 attached to the electric wire 2 with the digital camera 74 while the electric wire 2 is connected to the terminal block 1, the control unit 71 creates image data of the captured image in association with the serial number and panel number, and causes the data communication unit 72 to transmit an image data signal including the created image data to the server 8. When the worker operates the display 73 to instruct data matching, the control unit 71 causes the data communication unit 72 to transmit a matching instruction signal to the server 8.

[0098] The server 8 comprises a control unit 81 that controls the overall operation of the server 8, a data communication unit 82 that controls data communication with the mobile terminal 7, the design support system 9, and the PC terminal, a design data memory unit 83, a terminal block type information memory unit 84, a recognition data memory unit 85, a matching result memory unit 86, and a learned model memory unit 87.

[0099] The control unit 81 is mainly composed of a microcomputer having a CPU, ROM, RAM, I / O, etc., and, depending on the functions of the software, is equipped with a design data acquisition unit 81a, an inference device 81b consisting of an image data acquisition unit 81b1 and an inference unit 81b2, an image data processing unit 81c, a recognition data creation unit 81d, a data matching unit 81e, and a matching result output unit 81f.

[0100] In addition, if the wiring connection matching system is to have an AI learning function for creating an independent trained model, it may be equipped with a learning device 81g consisting of a data acquisition unit 81g1 and a model generation unit 81g2.

[0101] The model generating unit 81g2 learns the installation position, rotation angle, length and width dimensions, and model information of the terminal block 1 to be detected based on learning data created by combining the switchboard housing image and the terminal block label (correct answer) output from the data acquiring unit 81g1 in the learning device 81g. That is, in the wiring connection matching system, a trained model is generated that infers the optimal installation position, rotation angle, length and width dimensions, and model information of the terminal block 1 to be detected from the switchboard housing image and the terminal block label (correct answer). Here, the learning data is data in which the switchboard housing image and the terminal block label (correct answer) are associated with each other.

[0102] The learning device 81g and the inference device 81b are used in the wiring connection matching system to learn the installation position, rotation angle, length and width dimensions, and model information of the terminal block 1 to be detected, but may be devices connected to the wiring connection matching system via a network and separate from the wiring connection matching system. The learning device 81g and the inference device 81b may be built into the wiring connection matching system. Furthermore, the learning device 81g and the inference device 81b may exist on a cloud server.

[0103] The design data acquisition unit 81a acquires design data based on a wiring connection diagram showing the connection of the electric wire 2 to the terminal block 1 of the switchboard. That is, for example, when the wiring connection is designed by the design support system 9 (CAD), design data is created from the wiring connection diagram, and a design data signal including the created design data is transmitted from the data communication unit 91 provided in the design support system 9, so that the design data acquisition unit 81a acquires the design data by receiving the design data signal transmitted from the design support system 9. The wiring connection diagram includes information such as the serial number, drawing number, board number, terminal block device number, whether or not the terminal block needs to be inspected, terminal block number, and line number, so that the design data created from the wiring connection diagram also includes the information.

[0104] The design data is created for each terminal block 1 installed in the switchboard, and includes a serial number, a drawing number, a panel number, a terminal block device number, a terminal block number, a line number, and whether or not the terminal block needs to be inspected. The serial number is a number indicating the serial number of the product in which the switchboard is mounted. The drawing number is a number indicating the manufacturing drawing of the product. The panel number is a number indicating the switchboard. The terminal block device number is a number including the position information of the terminal block 1 that is uniquely assigned to each terminal block 1 arranged in the switchboard. The terminal block number is a number indicating the terminal block 1. The line number is a number unique to the electric wire 2 that is assigned by a band mark 3 or the like to identify the electric wire 2. The terminal block inspection need is information for distinguishing between the terminal block 1 to be inspected and the terminal block 1 that is not inspected when performing automatic inspection in the system. When the design data acquisition unit 81a acquires the design data, the acquired design data is stored in the design data storage unit 83 in a database.

[0105] When an image data signal transmitted from the portable terminal 7 is received by the data communication unit 82, the image data acquisition unit 81b1 extracts and acquires image data from the received image data signal, and inputs the acquired image data to the inference unit 81b2 within the inference device 81b.

[0106] The inference unit 81b2 infers the installation position, rotation angle, length and width dimensions, and model information of the terminal block 1 arranged inside the switchboard housing obtained by using the trained model. That is, by inputting image data of the inside of the switchboard housing acquired by the data acquisition unit 81g1 to this trained model, it is possible to output the installation position, rotation angle, length and width dimensions, and model information of the terminal block 1 inferred from the image data of the inside of the switchboard housing.

[0107] In this embodiment, the installation position, rotation angle, length and width dimensions, and model information of the terminal block 1 are output using a learned model learned by the model generation unit 81g2 of the wiring connection matching system. However, it is also possible to obtain a learned model from an external source, such as another wiring connection matching system, and output the installation position, rotation angle, length and width dimensions, and model information of the terminal block 1 based on this learned model.

[0108] Based on the installation position, rotation angle, length and width dimensions, and model information of the terminal block 1 output from the inference unit 81b2, the image data processing unit 81c determines a partial image area 102 containing optimal electric wire identification information. Once the model information of the terminal block 1 is determined, this partial image area 102 is determined to have a certain positional relationship relative to the position of the terminal block 1. Therefore, by analytically processing the image data based on the installation position, rotation angle, length and width dimensions, and model information of the terminal block 1 output from the inference unit 81b2 and the positional relationship information according to the model of the terminal block 1 stored in the terminal block model information storage unit 84, it is possible to extract partial image data (image data) in the partial image area 102 to contribute to reading the electric wire identification information.

[0109] The recognition data creation unit 81d creates recognition data in a format that can be collated with the design data, based on the information output from the inference unit 81b2 and information obtained by image analysis of the partial image data (image data) extracted by the image data processing unit 81c. That is, if the captured image data includes a terminal block device number, terminal block position information for deriving the terminal block number, or electric wire identification information for deriving the line number, the recognition data creation unit 81d performs image analysis on the image data to create recognition data including the terminal block device number, the terminal block number, and the line number. After creating the recognition data, the recognition data creation unit 81d stores the created recognition data in the recognition data storage unit 85 as a database.

[0110] When the data communication unit 82 receives a comparison instruction signal transmitted from the mobile terminal 7, the data comparison unit 81e reads out design data corresponding to the serial number and board number associated with the image data from the design data storage unit 83, reads out recognition data from the recognition data storage unit 85, and compares the read design data with the recognition data. That is, the data comparison unit 81e compares the design data including the terminal block device number, terminal block number, and line number with the recognition data.

[0111] In this case, if the electric wire 2 is properly connected to the terminal block 1, the design data and the recognition data will match. On the other hand, if the electric wire 2 is not properly connected to the terminal block 1, the design data and the recognition data will not match. The matching result output unit 81f stores the matching result of the data matching unit 81e in a database in the matching result storage unit 86, and transmits a matching result signal including the matching result from the data communication unit 82 to the mobile terminal 7. In the mobile terminal 7, when the matching result signal transmitted from the server 8 is received by the data communication unit 72, the control unit 81 extracts and acquires the matching result from the received matching result signal, and causes the display 73 to display the acquired matching result.

[0112] According to the wiring connection verification system including the learning device 81g and the inference device 81b of this embodiment, the image of the housing of the switchboard and the installation position, rotation angle, vertical and horizontal dimensions, and model information of the terminal block 1 constituting the switchboard are autonomously learned. Then, based on the learning result, the image data processing unit 81c determines the partial image area 102 including the optimum electric wire identification information, and performs image processing on only the partial image data (image data), so that the processing load of the image data can be reduced. Therefore, even in an environment in which the design support system 9 is used when designing a wiring connection diagram, and a worker who connects the electric wire 2 to the terminal block 1 of the switchboard uses a mobile terminal 7 while proceeding with the wiring work, it is possible to construct a system for verifying the wiring connections of a large-scale switchboard that could not be processed by the conventional method, by using a combination of a commonly used office personal computer, a commercially available image processing unit, and software as the server 8 according to this embodiment. Furthermore, if the date and time of operation is recorded at the same time that the worker operates the display 73 to input information, the collation results are stored in a database, so that manufacturing history information such as inspection results or inspection dates and times can be searched and viewed later using the serial number as a key.

[0113] Although the above describes a configuration in which the matching result is displayed on the mobile terminal 7, the matching result may be displayed on another terminal, such as a PC terminal connected to a network installed in a management room or the like. The matching result may be displayed on the mobile terminal 7 or a plurality of other terminals at the same time. When the matching result is no, the mismatched portion may be displayed on the display 73 of the mobile terminal 7 and an alarm sound may be output. The display mode or the alarm sound output mode may change depending on the degree of mismatch.

[0114] Furthermore, in the above description, a configuration has been described in which the digital camera 74 built into the mobile terminal 7 is used, but a configuration in which the digital camera 74 is used, image data captured by the digital camera 74 is extracted by the image data acquisition unit 81b1 of the inference device 81b in the server 8, and input to the inference unit 81b2 in the inference device 81b may also be used. In that case, the worker must, for example, operate a PC terminal connected via a network to input the serial number and panel number, and the image data captured by the digital camera 74 may be associated with the serial number and panel number input by the worker, for example, from the PC terminal connected via the network.

[0115] In the wiring connection matching system according to the present embodiment, the inference device 81b may have a function of displaying the inferred terminal block 1 on the display 73 of the mobile terminal 7 after inferring the installation position, rotation angle, length and width dimensions, and model information of the terminal block 1, and allowing the worker to select the terminal block 1 not to be inspected. This allows the worker to exclude the terminal block 1 that has passed the reinspection and continue the subsequent processing, thereby shortening the matching time.

[0116] Furthermore, the data collation unit 81e may read the terminal block inspection necessity of the design data at the time of collation, and may exclude the parts not requiring collation in advance from the collation. This allows the collation time to be shortened by setting the electric wire 2 that is difficult to inspect because it is in a blind spot from the shooting angle as not requiring collation.

[0117] The above embodiment is presented as an example. In addition, the above embodiment can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the gist of the present technology. The above embodiment or its modifications are included in the scope or gist of the present technology, and are included in the scope of equivalents.

[0118] <Effects of the above-described embodiments> Next, examples of effects produced by the above-described embodiments are shown. In the following description, the effects are described based on the specific configurations shown as examples in the above-described embodiments, but may be replaced with other specific configurations shown as examples in this specification as long as the same effects are produced. In other words, for convenience, only one of the corresponding specific configurations may be described as a representative below, but the representatively described specific configuration may be replaced with another corresponding specific configuration.

[0119] Furthermore, the replacement may be made across a number of embodiments, that is, configurations shown as examples in different embodiments may be combined to produce the same effect.

[0120] According to the embodiment described above, when reading the electric wire identification information given to the electric wire 2 connected to the terminal block 1 in the switchboard from the image data, the inference device infers the installation position, rotation angle, length and width dimensions, and model information of the terminal block 1 to which the electric wire 2 is connected, in order to extract the partial image area 102 including the electric wire identification information, based on the installation position, rotation angle, length and width dimensions, and model information of the terminal block 1 to which the electric wire 2 is connected. The inference device is provided in the wiring connection matching device. The wiring connection matching device matches design data based on a wiring connection diagram showing the installation position of at least one terminal block 1 in the switchboard and the connection position of the electric wire 2 to the terminal block 1 with image data captured inside the switchboard, and matches the connection state of the electric wire 2. The inference device includes an image data acquisition unit 53b1 (or an image data acquisition unit 81b1) that acquires the captured image data, and an inference unit 53b2 (or an inference unit 81b2). The inference unit 53b2 infers and outputs the installation position, rotation angle, length and width dimensions and model information of the terminal block 1 from the image data acquired by the image data acquisition unit 53b1 using a first learned model for inferring the installation position, rotation angle, length and width dimensions and model information of the terminal block 1 from the image data acquired by the image data acquisition unit 53b1.

[0121] According to this configuration, the partial image area 102 including the electric wire identification information can be extracted based on the installation position, rotation angle, vertical and horizontal dimensions, and model information of the terminal block inferred using the trained model, so that the electric wire identification information can be read efficiently from the image data. Therefore, the processing load of the image data for reading the electric wire identification information can be reduced, and the connection state of the electric wire can be effectively verified while suppressing an increase in processing time.

[0122] Furthermore, the same effect can be achieved even if other configurations, examples of which are shown in this specification, are appropriately added to the above configuration, i.e., even if other configurations in this specification that were not mentioned as the above configuration are appropriately added.

[0123] Moreover, according to the embodiment described above, the electric wire identification information is a character string consisting of alphanumeric characters and symbols. Moreover, image processing is used when reading the character string from the image data. Then, the inference unit 53b2 (or the inference unit 81b2) infers and outputs the installation position, rotation angle, length and width dimensions, and model information of the terminal block 1, which are used when determining the range of image processing. With such a configuration, since the electric wire identification information is a character string, the worker can visually confirm the information regarding the identification of the electric wire.

[0124] Furthermore, according to the embodiment described above, when the number of connection points of the electric wires 2 in the switchboard exceeds 100, the inference unit 53b2 (or the inference unit 81b2) is configured with an electronic computer equipped with a central processing unit having a processing speed of 90 gigaflops or less. With such a configuration, a dedicated central processing unit capable of realizing a high processing speed is not required, and the wiring connection check of the switchboard can be completed in a reasonable work time using a commercially available personal computer that is widely used.

[0125] Furthermore, according to the embodiment described above, the inference unit 53b2 (or the inference unit 81b2) infers and outputs the partial image region 102 including the electric wire identification information from the image data using a second trained model for inferring the partial image region 102 including the electric wire identification information from the image data. With such a configuration, the second trained model can infer and output the partial image region 102 that has been weighted and optimized by a neural network through supervised learning, rather than the partial image region 102 calculated based on an analytical formula with reference to the installation position, rotation angle, vertical and horizontal dimensions, and model information of the terminal block 1.

[0126] According to the embodiment described above, the wiring connection checking device includes the inference device, the image data processing unit 53c (or the image data processing unit 81c), the recognition data creation unit 53d (or the recognition data creation unit 81d), the design data acquisition unit 53a (or the design data acquisition unit 81a), the data matching unit 53e (or the data matching unit 81e), and the matching result output unit 53f (or the matching result output unit 81f). The image data processing unit 53c extracts the partial image area 102 including the electric wire identification information from the image data acquired by the image data acquisition unit 53b1 based on the output result of the inference unit 53b2. The recognition data creation unit 53d creates recognition data in a format that can be matched with the design data from the image data in the partial image area 102. The design data acquisition unit 53a acquires the design data. The data matching unit 53e matches the design data with the recognition data to determine whether the connection state of the electric wire 2 is normal or not. The collation result output unit 53f outputs the collation result of the data collation unit 53e.

[0127] According to this configuration, the partial image area 102 including the electric wire identification information can be extracted based on the inferred installation position, rotation angle, length and width dimensions, and model information of the terminal block, so that the electric wire identification information can be efficiently read from the image data. Also, by creating recognition data from the image data in the partial image area 102 and comparing it with the design data, the connection state of the electric wire can be automatically determined.

[0128] Furthermore, according to the embodiment described above, a plurality of terminal blocks 1 are provided in the switchboard. Then, the data matching unit 53e (or the data matching unit 81e) selects whether or not to extract the partial image area 102 for each terminal block 1. With this configuration, the terminal blocks 1 that do not need to be matched are excluded from the matching targets, thereby shortening the matching time.

[0129] According to the embodiment described above, the design data includes at least the position information of the terminal block 1, the number of the terminal block 1, and the line number. The data comparison unit 53e (or the data comparison unit 81e) compares the design data with the recognition data using at least one of the position information of the terminal block 1, the number of the terminal block 1, and the line number as a comparison target. According to this configuration, the recognition data is created from the image data in the partial image area 102, and the recognition data is compared with the design data using at least one of the position information of the terminal block, the number of the terminal block, and the line number as a comparison target, thereby making it possible to automatically determine the connection state of the electric wires.

[0130] According to the embodiment described above, the learning device learns the installation position, rotation angle, length and width dimensions, and model information of the terminal block 1 to which the electric wire 2 is connected, for comparing design data based on a wiring connection diagram showing the installation location of the terminal block 1 in the switchboard and the connection location of the electric wire 2 to the terminal block 1 with image data of the inside of the switchboard. The learning device includes a data acquisition unit 81g1 and a model generation unit 81g2. The data acquisition unit 81g1 acquires first learning data including the image data and the installation position, rotation angle, length and width dimensions, and model information of the terminal block 1. The model generation unit 81g2 uses the first learning data to generate a first learned model for inferring the installation position, rotation angle, length and width dimensions, and model information of the terminal block 1 from the image data.

[0131] According to this configuration, the installation position, rotation angle, length and width dimensions, and model information of the terminal block can be inferred using the trained model, and the partial image area 102 including the electric wire identification information can be extracted based on the installation position, rotation angle, length and width dimensions, and model information of the terminal block. Therefore, the electric wire identification information can be efficiently read from the image data while reducing the processing load of the image data.

[0132] According to the embodiment described above, the electric wire identification information given to the electric wire 2 connected to the terminal block 1 in the switchboard is a character string consisting of alphanumeric characters and symbols. Image processing is used to read the character string from the image data. Then, the model generation unit 81g2 generates a first trained model for inferring the installation position, rotation angle, vertical and horizontal dimensions, and model information of the terminal block 1, which are used to determine the range of image processing. According to this configuration, since the electric wire identification information is a character string, the worker can visually confirm information regarding the identification of the electric wire. Furthermore, the image recognition function for learning of the learning device can be diverted as a function for reading the electric wire identification information, so that the cost of constructing additional hardware can be saved.

[0133] According to the embodiment described above, the model generating unit 81g2 is configured with an electric computer having a storage device with a storage capacity of 8 gigabytes or less. With such a configuration, a wiring identification and matching device can be configured with an inexpensive and easily available image processing unit and commercially available optical character recognition software.

[0134] According to the embodiment described above, the data acquisition unit 81g1 acquires second learning data including image data and a partial image area 102 including electric wire identification information assigned to an electric wire 2 connected to a terminal block 1 in a switchboard. Then, the model generation unit 81g2 uses the second learning data to generate a second trained model for inferring the partial image area 102 including the electric wire identification information from the image data. According to such a configuration, the second trained model can infer and output the partial image area 102 that has been weighted and optimized by a neural network through supervised learning, rather than the partial image area 102 calculated based on an analytical formula with reference to the installation position, rotation angle, length and width dimensions, and model information of the terminal block 1.

[0135] According to the embodiment described above, the wiring connection checking system includes the inference device, the image data processing unit 53c (or the image data processing unit 81c), the recognition data creation unit 53d (or the recognition data creation unit 81d), the design data acquisition unit 53a (or the design data acquisition unit 81a), the data matching unit 53e (or the data matching unit 81e), and the matching result output unit 53f (or the matching result output unit 81f). The image data processing unit 53c extracts the partial image area 102 including the electric wire identification information from the image data acquired by the image data acquisition unit 53b1 based on the output result of the inference unit 53b2. The recognition data creation unit 53d creates recognition data in a format that can be matched with the design data from the image data in the partial image area 102. The design data acquisition unit 53a acquires the design data. The data matching unit 53e matches the design data with the recognition data to determine whether the connection state of the electric wire 2 is normal or not. The collation result output unit 53f outputs the collation result of the data collation unit 53e.

[0136] According to this configuration, the partial image area 102 including the electric wire identification information can be extracted based on the inferred installation position, rotation angle, length and width dimensions, and model information of the terminal block, so that the electric wire identification information can be efficiently read from the image data. Also, by creating recognition data from the image data in the partial image area 102 and comparing it with the design data, the connection state of the electric wire can be automatically determined.

[0137] According to the embodiment described above, the wiring connection matching system includes a data acquisition unit 81g1 and a model generation unit 81g2. The data acquisition unit 81g1 acquires first learning data including image data and the installation position, rotation angle, length and width dimensions, and model information of the terminal block 1. The model generation unit 81g2 uses the first learning data to generate a first trained model for inferring the installation position, rotation angle, length and width dimensions, and model information of the terminal block 1 from the image data. According to this configuration, the system includes a learning device 81g, so that the user (location of use) can change or enhance the trained model as necessary.

[0138] <Modifications of the above-described embodiments> In the multiple embodiments described above, the dimensions, shapes, relative positional relationships, and implementation conditions of each component may be described, but these are merely examples in all aspects and are not limiting.

[0139] Therefore, countless modifications and equivalents not shown as examples are assumed within the scope of the technology disclosed in the present specification, including, for example, modifying, adding, or omitting at least one component, and further, extracting at least one component in at least one embodiment and combining it with a component in another embodiment.

[0140] Furthermore, unless a contradiction arises, when it is stated in the above-described embodiments that "one" component is provided, it is understood that "one or more" of that component may be provided.

[0141] Furthermore, each component in the embodiments described above is a conceptual unit, and the scope of the technology disclosed in this specification includes cases where one component is made up of multiple structures, where one component corresponds to a part of a structure, and even where multiple components are provided in one structure.

[0142] Furthermore, each of the components in the embodiments described above includes structures having other structures or shapes as long as they perform the same function.

[0143] Furthermore, the descriptions in this specification are incorporated by reference for all purposes related to the present technology, and none of them are admitted to be prior art.

[0144] Various aspects of the present disclosure are summarized below as appendices.

[0145] (Appendix 1) A wiring connection verification device that verifies a connection state of electric wires by verifying design data based on a wiring connection diagram that indicates an installation location of at least one terminal block in a distribution board and a connection location of electric wires to the terminal block against image data obtained by capturing an image of the inside of the distribution board, an inference device that, when reading electric wire identification information given to the electric wire connected to the terminal block in the switchboard from the image data, infers an installation position, a rotation angle, length and width dimensions and model information of the terminal block to which the electric wire is connected, in order to extract a partial image area including the electric wire identification information, based on the installation position, rotation angle, length and width dimensions and model information of the terminal block to which the electric wire is connected; An image data acquisition unit that acquires the captured image data; and an inference unit that infers and outputs the installation position, rotation angle, length and width dimensions, and model information of the terminal block from the image data acquired by the image data acquisition unit using a first trained model for inferring the installation position, rotation angle, length and width dimensions, and model information of the terminal block from the image data acquired by the image data acquisition unit. Reasoning device.

[0146] (Appendix 2) 2. An inference device according to claim 1, The electric wire identification information is a character string consisting of alphanumeric characters and symbols, image processing is used when reading the character string from the image data; the inference unit infers and outputs the installation position, rotation angle, length and width dimensions, and model information of the terminal block, which are used when determining the range of the image processing; Reasoning device.

[0147] (Appendix 3) 3. An inference device according to claim 1 or 2, When the number of connection points of the electric wire in the switchboard exceeds 100, The inference unit is composed of an electronic computer equipped with a central processing unit having a processing speed of 90 gigaflops or less. Reasoning device.

[0148] (Appendix 4) An inference device according to any one of claims 1 to 3, The inference unit infers the partial image area including the electric wire identification information from the image data by using a second trained model for inferring the partial image area including the electric wire identification information from the image data, and outputs the inference. Reasoning device.

[0149] (Appendix 5) An inference device according to any one of claims 1 to 4; an image data processing unit that extracts the partial image area including the electric wire identification information from the image data acquired by the image data acquisition unit based on an output result of the inference unit; a recognition data creation unit that creates recognition data in a format that can be compared with the design data from the image data in the partial image area; A design data acquisition unit that acquires the design data; a data collating unit that compares the design data with the recognition data and determines whether or not the connection state of the electric wire is normal; A matching result output unit outputs a matching result of the data matching unit. Wiring connection verification device.

[0150] (Appendix 6) A wiring connection verification device according to claim 5, A plurality of the terminal blocks are provided in the power distribution board, the data collation unit selects whether or not to extract the partial image area for each of the terminal blocks. Wiring connection verification device.

[0151] (Appendix 7) A wiring connection verification device according to claim 5 or 6, The design data includes at least the position information of the terminal block, the number of the terminal block, and the line number, the data collating unit collates the design data with the recognition data by collating at least one of the position information of the terminal block, the number of the terminal block, and the line number; Wiring connection verification device.

[0152] (Appendix 8) a learning device that learns the installation position, rotation angle, length and width dimensions, and model information of the terminal block to which the electric wire is connected, for comparing design data based on a wiring connection diagram showing the installation positions of the terminal block in the switchboard and the connection positions of the electric wires to the terminal block with image data obtained by capturing an image of the inside of the switchboard; A data acquisition unit that acquires the image data and first learning data including an installation position, a rotation angle, a length and width dimension, and model information of the terminal block; A model generation unit that generates a first trained model for inferring an installation position, a rotation angle, a length and width dimension, and a model information of the terminal block from the image data using the first training data, Learning device.

[0153] (Appendix 9) 9. The learning device according to claim 8, The electric wire identification information given to the electric wire connected to the terminal block in the switchboard is a character string consisting of alphanumeric characters and symbols, image processing is used when reading the character string from the image data; The model generation unit generates the first trained model for inferring the installation position, rotation angle, length and width dimensions, and model information of the terminal block, which are used when determining the range of the image processing. Learning device.

[0154] (Appendix 10) 10. The learning device according to claim 8 or 9, The model generation unit is configured with an electric computer having a storage device with a storage capacity of 8 gigabytes or less. Learning device.

[0155] (Appendix 11) A learning device according to any one of claims 8 to 10, the data acquisition unit acquires second learning data including the image data and a partial image area including electric wire identification information assigned to the electric wire connected to the terminal block in the switchboard; the model generation unit generates a second trained model for inferring the partial image region including the electric wire identification information from the image data, using the second training data; Learning device.

[0156] (Appendix 12) An inference device according to any one of claims 1 to 4; an image data processing unit that extracts the partial image area including the electric wire identification information from the image data acquired by the image data acquisition unit based on an output result of the inference unit; a recognition data creation unit that creates recognition data in a format that can be compared with the design data from the image data in the partial image area; A design data acquisition unit that acquires the design data; a data collating unit that compares the design data with the recognition data and determines whether or not the connection state of the electric wire is normal; A matching result output unit outputs a matching result of the data matching unit. Wiring connection matching system.

[0157] (Appendix 13) A wiring connection matching system according to claim 12, A data acquisition unit that acquires the image data and first learning data including an installation position, a rotation angle, a length and width dimension, and model information of the terminal block; A model generation unit that generates a first trained model for inferring an installation position, a rotation angle, a length and width dimension, and a model information of the terminal block from the image data using the first training data, Wiring connection matching system. [Explanation of symbols]

[0158] 1 terminal block, 2 electric wire, 2a electric wire, 2b electric wire, 5 wiring connection matching device, 6 learning device, 53a design data acquisition unit, 53b inference device, 53b1 image data acquisition unit, 53b2 inference unit, 53c image data processing unit, 53d recognition data creation unit, 53e data matching unit, 53f matching result output unit, 61 data acquisition unit, 62 model generation unit, 81a design data acquisition unit, 81b inference device, 81b1 image data acquisition unit, 81b2 inference unit, 81c image data processing unit, 81d recognition data creation unit, 81e data matching unit, 81f matching result output unit, 81g learning device, 81g1 data acquisition unit, 81g2 model generation unit, 100 area, 102 partial image area, 1103 storage device.

Claims

1. A wiring connection verification device that verifies a connection state of electric wires by verifying design data based on a wiring connection diagram that indicates an installation location of at least one terminal block in a distribution board and a connection location of electric wires to the terminal block against image data obtained by capturing an image of the inside of the distribution board, an inference device that, when reading electric wire identification information given to the electric wire connected to the terminal block in the switchboard from the image data, infers an installation position, a rotation angle, length and width dimensions and model information of the terminal block to which the electric wire is connected, in order to extract a partial image area including the electric wire identification information, based on the installation position, rotation angle, length and width dimensions and model information of the terminal block to which the electric wire is connected; An image data acquisition unit that acquires the captured image data; and an inference unit that infers and outputs the installation position, rotation angle, length and width dimensions, and model information of the terminal block from the image data acquired by the image data acquisition unit using a first trained model for inferring the installation position, rotation angle, length and width dimensions, and model information of the terminal block from the image data acquired by the image data acquisition unit. Reasoning device.

2. 2. The inference device according to claim 1, The electric wire identification information is a character string consisting of alphanumeric characters and symbols, image processing is used when reading the character string from the image data; the inference unit infers and outputs the installation position, rotation angle, length and width dimensions, and model information of the terminal block, which are used when determining the range of the image processing; Reasoning device.

3. 3. The inference device according to claim 1, When the number of connection points of the electric wire in the switchboard exceeds 100, The inference unit is composed of an electronic computer equipped with a central processing unit having a processing speed of 90 gigaflops or less. Reasoning device.

4. 3. The inference device according to claim 1, The inference unit infers the partial image area including the electric wire identification information from the image data by using a second trained model for inferring the partial image area including the electric wire identification information from the image data, and outputs the inference. Reasoning device.

5. An inference device according to claim 1 or 2; an image data processing unit that extracts the partial image area including the electric wire identification information from the image data acquired by the image data acquisition unit based on an output result of the inference unit; a recognition data creation unit that creates recognition data in a format that can be compared with the design data from the image data in the partial image area; A design data acquisition unit that acquires the design data; a data collating unit that compares the design data with the recognition data and determines whether or not the connection state of the electric wire is normal; A matching result output unit outputs a matching result of the data matching unit. Wiring connection verification device.

6. 6. The wiring connection verification device according to claim 5, A plurality of the terminal blocks are provided in the power distribution board, the data collation unit selects whether or not to extract the partial image area for each of the terminal blocks. Wiring connection verification device.

7. 6. The wiring connection verification device according to claim 5, The design data includes at least the position information of the terminal block, the number of the terminal block, and the line number, the data collating unit collates the design data with the recognition data by collating at least one of the position information of the terminal block, the number of the terminal block, and the line number; Wiring connection verification device.

8. a learning device that learns the installation position, rotation angle, length and width dimensions, and model information of the terminal block to which the electric wire is connected, for comparing design data based on a wiring connection diagram showing the installation positions of the terminal block in the switchboard and the connection positions of the electric wires to the terminal block with image data obtained by capturing an image of the inside of the switchboard; a data acquisition unit that acquires the image data and first learning data including an installation position, a rotation angle, a length and width dimension, and model information of the terminal block; A model generation unit that generates a first trained model for inferring an installation position, a rotation angle, a length and width dimension, and model information of the terminal block from the image data using the first training data, Learning device.

9. The learning device according to claim 8, The electric wire identification information given to the electric wire connected to the terminal block in the switchboard is a character string consisting of alphanumeric characters and symbols, image processing is used when reading the character string from the image data; The model generation unit generates the first trained model for inferring the installation position, rotation angle, length and width dimensions, and model information of the terminal block used when determining the range of the image processing. Learning device.

10. A learning device according to claim 8 or 9, the data acquisition unit acquires second learning data including the image data and a partial image area including electric wire identification information assigned to the electric wire connected to the terminal block in the switchboard; the model generation unit generates a second trained model for inferring the partial image region including the electric wire identification information from the image data, using the second training data; Learning device.

11. An inference device according to claim 1 or 2; an image data processing unit that extracts the partial image area including the electric wire identification information from the image data acquired by the image data acquisition unit based on an output result of the inference unit; a recognition data creation unit that creates recognition data in a format that can be compared with the design data from the image data in the partial image area; A design data acquisition unit that acquires the design data; a data collating unit that compares the design data with the recognition data and determines whether or not the connection state of the electric wire is normal; A matching result output unit outputs a matching result of the data matching unit. Wiring connection matching system.

12. The wiring connection matching system according to claim 11, a data acquisition unit that acquires the image data and first learning data including an installation position, a rotation angle, a length and width dimension, and model information of the terminal block; A model generation unit that generates a first trained model for inferring an installation position, a rotation angle, a length and width dimension, and a model information of the terminal block from the image data using the first training data, Wiring connection matching system.

Citation Information

Patent Citations

  • Method for checking cable of component of electrical cabinet

    CN110019323A

  • Distribution board cable connection confirmation device, and image processing system

    JP2014093870A

  • Wiring work support system

    JP2021129474A