A matching method and apparatus for terminal blocks and terminal block wiring.

Abstract

This invention discloses a matching method and apparatus for terminal blocks and terminal block wiring. The method includes: acquiring and preprocessing images of terminal blocks in the field; inputting the terminal block images into a pre-trained image recognition neural network model and a text recognition model; identifying the terminal blocks and terminal block wiring pipes in the terminal block images using the image recognition neural network model; identifying the text in the terminal block images using the text recognition model; matching the recognition results of the image recognition neural network model and the recognition results of the text recognition model to obtain the correspondence between the terminal blocks, terminal block wiring pipes, and text, and generating a text file. This method can effectively match and correspond terminal blocks and terminal block wiring, providing a valid basis for checking whether the wiring is correct and reducing labor costs.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence, and in particular to a matching method and apparatus for terminal blocks and terminal block wiring. Background Technology

[0002] The power station's distribution cabinets and terminal boxes contain a large number of terminal blocks, and the wiring of these terminal blocks on-site is complex. Ensuring the proper functioning of the terminal blocks and their wiring is an important part of daily inspections. In the early days, the YOLO target detection algorithm could only mark the terminal blocks with rectangular boxes, and it could not effectively detect the overlapping areas of the terminal blocks and their wiring, thus failing to meet the detection requirements for the corresponding areas of the terminal blocks and their wiring.

[0003] Patent document CN112001238A provides a method for identifying the wiring status of a terminal block. The method involves acquiring an image of the terminal block as the image to be identified, determining the wiring area in the image to be identified, and comparing the wiring area in the image to be identified with the wiring area in a standard wiring status diagram to determine the wiring status of the terminal block.

[0004] The aforementioned patent documents can only identify the wiring status of the terminal blocks, but cannot detect whether the terminal blocks and their wiring correspond. Summary of the Invention

[0005] This invention provides a matching method and apparatus for terminal blocks and terminal block wiring, which can effectively match and correspond terminal blocks and terminal block wiring.

[0006] A matching method for terminal blocks and terminal block wiring, comprising:

[0007] Acquire images of the terminal blocks at the site and perform preprocessing;

[0008] The terminal block images are respectively input into a pre-trained image recognition neural network model and a text recognition model;

[0009] The image recognition neural network model is used to identify the terminal block and terminal block wiring pipe in the terminal block image;

[0010] The text recognition model is used to identify the text in the terminal block image;

[0011] The recognition results of the image recognition neural network model and the recognition results of the text recognition model are matched to obtain the correspondence between the terminal block, the terminal block wiring pipe and the text, and a text file is generated.

[0012] Further, training the image recognition neural network model includes:

[0013] Collect training images and label the terminal block area and terminal block wiring pipe area in the training images to form a training set;

[0014] Establish an image recognition neural network model containing learnable parameters;

[0015] The training set is input into the neural network model, and the parameters of the image recognition neural network model are updated based on the gradient descent method until the image recognition neural network model converges.

[0016] Further, the image recognition neural network model is used to identify the terminal block and terminal block wiring pipe in the terminal block image, including:

[0017] The edge contours of the terminal blocks and terminal block conduits in the terminal block image are extracted, and the minimum bounding rectangle is used to replace the edge contour point set. The endpoint coordinates and area of ​​the minimum bounding rectangle of the terminal block area, as well as the endpoint coordinates and area of ​​the minimum bounding rectangle of the terminal block conduit, are calculated.

[0018] Furthermore, recognizing the text in the terminal block image using the text recognition model includes:

[0019] Extract feature sequences from the terminal block image;

[0020] Predict the feature sequence and output the probability distribution of all characters;

[0021] Select the string with the highest output probability as the final predicted string to obtain the text recognition result;

[0022] The text recognition result includes a first string and a second string;

[0023] The method further includes:

[0024] Select the first string and the second string by drawing a box to obtain a first string box and a second string box;

[0025] Calculate the coordinates and area of ​​the first and second string boxes.

[0026] Further, matching the recognition results of the image recognition neural network model with the recognition results of the text recognition model includes:

[0027] Calculate the first intersection-to-union ratio of the area of ​​the first string frame to the area of ​​the minimum outer rectangle of the terminal block region, and the second intersection-to-union ratio of the area of ​​the first string frame to the area of ​​the minimum outer rectangle of the terminal block wiring pipe. If the first intersection-to-union ratio is greater than the first preset threshold, then the first string is determined to be the terminal sequence number; if the second intersection-to-union ratio is greater than the second preset threshold, then the first string is determined to be the wiring pipe number.

[0028] After determining that the first string is the terminal sequence number, calculate the third intersection ratio of the area of ​​the second string with the area of ​​the smallest outer rectangle of the terminal block wiring pipe. If the third intersection ratio is greater than the third threshold, then determine that the second string is the wiring pipe number; or, after determining that the second string is the wiring pipe number, calculate the fourth intersection ratio of the area of ​​the second string with the area of ​​the smallest outer rectangle of the terminal block area. If the fourth intersection ratio is greater than the fourth threshold, then determine that the second string is the terminal sequence number.

[0029] A matching device for terminal blocks and terminal block wiring, comprising:

[0030] The preprocessing module is used to acquire and preprocess images of the on-site terminal blocks;

[0031] The input module is used to input the terminal block image into a pre-trained image recognition neural network model and a text recognition model, respectively;

[0032] The first identification module is used to identify the terminal block and terminal block wiring pipe in the terminal block image through the image recognition neural network model;

[0033] The second recognition module is used to recognize the text in the terminal block image through the text recognition model;

[0034] The matching module is used to match the recognition results of the image recognition neural network model with the recognition results of the text recognition model to obtain the correspondence between the terminal block and the terminal block wiring pipe and their text, and generate a text file.

[0035] Furthermore, the device also includes a training module for collecting training images and labeling the terminal block area and terminal block wiring pipe area in the training images to form a training set; establishing an image recognition neural network model containing learnable parameters; inputting the training set into the neural network model, and updating the parameters of the image recognition neural network model based on the gradient descent method until the image recognition neural network model converges.

[0036] Furthermore, the first recognition module is also used to extract the edge contours of the terminal block and the terminal block wiring pipe in the terminal block image through the image recognition neural network model, and to use the minimum bounding rectangle to replace the edge contour point set, calculate the endpoint coordinates and area of ​​the minimum bounding rectangle of the terminal block area, and the endpoint coordinates and area of ​​the minimum bounding rectangle of the terminal block wiring pipe.

[0037] Further, the second recognition module is used to extract feature sequences from the terminal block image through the text recognition model; predict the feature sequences and output the probability distribution of all characters; select the maximum output probability as the final predicted string to obtain the text recognition result; the text recognition result includes a first string and a second string; select the first string and the second string to obtain a first string box and a second string box; calculate the coordinates and area of ​​the first string box and the second string box.

[0038] Furthermore, the matching module is also used to calculate the first intersection-union ratio of the area of ​​the first string frame to the area of ​​the minimum outer rectangle of the terminal block region, and the second intersection-union ratio of the area of ​​the first string frame to the area of ​​the minimum outer rectangle of the terminal block wiring pipe. If the first intersection-union ratio is greater than the first preset threshold, the first string is determined to be the terminal sequence number; if the second intersection-union ratio is greater than the second preset threshold, the first string is determined to be the wiring pipe number.

[0039] After determining that the first string is the terminal sequence number, calculate the third intersection ratio of the area of ​​the second string with the area of ​​the smallest outer rectangle of the terminal block wiring pipe. If the third intersection ratio is greater than the third threshold, then determine that the second string is the wiring pipe number; or, after determining that the second string is the wiring pipe number, calculate the fourth intersection ratio of the area of ​​the second string with the area of ​​the smallest outer rectangle of the terminal block area. If the fourth intersection ratio is greater than the fourth threshold, then determine that the second string is the terminal sequence number.

[0040] The present invention provides a matching method and apparatus for terminal blocks and terminal block wiring. It obtains the terminal block area and terminal block wiring pipe through image recognition, then identifies the string in the image through a text recognition model, and identifies which area the string belongs to through intersection-union ratio calculation. It can effectively match the terminal blocks and terminal block wiring, providing an effective basis for checking whether the wiring is correct and reducing labor costs. Attached Figure Description

[0041] Figure 1 A flowchart illustrating one embodiment of the matching method for terminal blocks and terminal block wiring provided by the present invention.

[0042] Figure 2 This is a flowchart of one embodiment of the terminal block and terminal block wiring pipe in the matching method for terminal blocks and terminal block wiring provided by the present invention.

[0043] Figure 3 This is a schematic diagram of one embodiment of the matching device for terminal blocks and terminal block wiring provided by the present invention. Detailed Implementation

[0044] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0045] refer to Figure 1 In some embodiments, a matching method for terminal blocks and terminal block wiring is provided, comprising:

[0046] S1. Acquire images of the on-site terminal blocks and perform preprocessing;

[0047] S2. Input the terminal block image into a pre-trained image recognition neural network model and a text recognition model, respectively;

[0048] S3. Identify the terminal block and terminal block wiring pipe in the terminal block image using the image recognition neural network model;

[0049] S4. Recognize the text in the terminal block image using the text recognition model;

[0050] S5. Match the recognition results of the image recognition neural network model with the recognition results of the text recognition model to obtain the correspondence between the terminal block, the terminal block wiring pipe and the text, and generate a text file.

[0051] Specifically, in step S1, the preprocessing of the on-site terminal block image includes, but is not limited to, noise reduction, grayscale processing, and binarization of the terminal block image.

[0052] Further, refer to Figure 2 Each acquired terminal block image is the smallest unit of the terminal block image, including a terminal block 101 and a corresponding terminal block wiring pipe 102. The terminal block 101 area is provided with a terminal sequence number 103, and the terminal block wiring pipe 102 is provided with a wiring pipe sequence number 104.

[0053] Further, in step S2, training the image recognition neural network model includes:

[0054] Collect training images and label the terminal block area and terminal block wiring pipe area in the training images to form a training set;

[0055] Establish an image recognition neural network model containing learnable parameters;

[0056] The training set is input into the image recognition neural network model, and the parameters of the image recognition neural network model are updated based on the gradient descent method until the image recognition neural network model converges.

[0057] The text recognition model is a CRNN neural network model, in which the feature extraction network utilizes the ResNet34 network backbone framework. This framework not only has high-accuracy feature extraction capabilities, but can also be easily deployed for inference on mobile devices. The RNN recurrent neural network layer utilizes the LSTM long short-term memory neural network. Compared with ordinary recurrent neural networks, this neural network avoids the gradient vanishing and gradient exploding problems during training and can perform better in longer sequences.

[0058] Further, in step S3, identifying the terminal block and terminal block wiring conduit in the terminal block image using the image recognition neural network model includes:

[0059] The edge contours of the terminal blocks and terminal block conduits in the terminal block image are extracted, and the minimum bounding rectangle is used to replace the edge contour point set. The endpoint coordinates and area of ​​the minimum bounding rectangle of the terminal block area, as well as the endpoint coordinates and area of ​​the minimum bounding rectangle of the terminal block conduit, are calculated.

[0060] Specifically, after obtaining the endpoint coordinates of the smallest circumscribed rectangle of the terminal block area, its area can be calculated based on the endpoint coordinates.

[0061] Further, in step S4, recognizing the text in the terminal block image using the text recognition model includes:

[0062] Extract feature sequences from the terminal block image;

[0063] Predict the feature sequence and output the probability distribution of all characters;

[0064] Select the output probability with the highest probability as the final predicted string to obtain the text recognition result.

[0065] Specifically, the text recognition result includes a first string and a second string;

[0066] The method further includes:

[0067] Select the first string and the second string by drawing a box to obtain a first string box and a second string box;

[0068] Calculate the coordinates and area of ​​the first and second string boxes.

[0069] Specifically, the area can be calculated based on the endpoint coordinates of the first and second string boxes.

[0070] Further, in step S5, the recognition results of the image recognition neural network model and the recognition results of the text recognition model are matched, including:

[0071] Calculate the first intersection-to-union ratio of the area of ​​the first string frame to the area of ​​the minimum outer rectangle of the terminal block region, and the second intersection-to-union ratio of the area of ​​the first string frame to the area of ​​the minimum outer rectangle of the terminal block wiring pipe. If the first intersection-to-union ratio is greater than the first preset threshold, then the first string is determined to be the terminal sequence number; if the second intersection-to-union ratio is greater than the second preset threshold, then the first string is determined to be the wiring pipe number.

[0072] After determining that the first string is the terminal sequence number, calculate the third intersection ratio of the area of ​​the second string with the area of ​​the smallest outer rectangle of the terminal block wiring pipe. If the third intersection ratio is greater than the third threshold, then determine that the second string is the wiring pipe number; or, after determining that the second string is the wiring pipe number, calculate the fourth intersection ratio of the area of ​​the second string with the area of ​​the smallest outer rectangle of the terminal block area. If the fourth intersection ratio is greater than the fourth threshold, then determine that the second string is the terminal sequence number.

[0073] Considering that the wiring of terminal blocks in actual engineering sites may be irregular, for example, the wiring of terminal block 1 may overlap with the wiring of terminal block 2, that is, the cable may be bent during actual wiring. Therefore, by calculating the intersection-parallel ratio, we can more accurately distinguish whether the string belongs to the terminal block area or the terminal block wiring pipe, that is, whether the string is the terminal sequence number or the wiring pipe number.

[0074] After determining whether the string represents a terminal block sequence number or a conduit number, they are matched one by one. Multiple terminal block images are collected, identified, and matched to obtain the correspondence between multiple terminal block sequence numbers and conduit numbers. This correspondence is then generated into a text file, for example:

[0075] {"number":"8","info":"YN8-101 / YMB80"},

[0076] {"number":"9","info":"3330CT-109 / A"},

[0077] {"number":"10","info":"1YFA-13"},

[0078] {"number":"11","info":"4n-p5:30"}

[0079] Among them, "8", "9", "10", and "11" are terminal sequence numbers, and "YN8-101 / YMB80", "3330CT-109 / A", "1YFA-13", and "4n-p5:30" are wiring pipe numbers.

[0080] After the text file is generated, technicians can check and compare it to determine whether the terminal blocks and terminal block wiring are connected correctly.

[0081] As another alternative implementation, a standard text file can be pre-created, and the generated text file can be compared with the standard text file to determine whether the wiring is correct based on the comparison result.

[0082] The method provided in the above embodiments obtains the terminal block area and terminal block wiring through image recognition, then identifies the string in the image through a text recognition model, and identifies which area the string belongs to through intersection-union ratio calculation. This can effectively match the terminal block and terminal block wiring, providing a valid basis for checking whether the wiring is correct and reducing labor costs.

[0083] In some embodiments, reference Figure 3 A matching device for terminal blocks and terminal block wiring is provided, comprising:

[0084] Preprocessing module 101 is used to acquire images of the on-site terminal blocks and perform preprocessing.

[0085] Input module 102 is used to input the terminal block image into a pre-trained image recognition neural network model and a text recognition model, respectively;

[0086] The first identification module 103 is used to identify the terminal block and terminal block wiring pipe in the terminal block image through the image recognition neural network model;

[0087] The second recognition module 104 is used to recognize the text in the terminal block image through the text recognition model;

[0088] The matching module 105 is used to match the recognition results of the image recognition neural network model and the recognition results of the text recognition model to obtain the correspondence between the terminal block and the terminal block wiring pipe and their text, and generate a text file.

[0089] Furthermore, the device also includes a training module 106, used to acquire training images and label the terminal block area and terminal block wiring pipe area in the training images to form a training set; establish an image recognition neural network model containing learnable parameters; input the training set into the neural network model, and update the parameters of the image recognition neural network model based on the gradient descent method until the image recognition neural network model converges.

[0090] Furthermore, the first recognition module 103 is also used to extract the edge contours of the terminal block and the terminal block wiring pipe in the terminal block image through the image recognition neural network model, and to use the minimum bounding rectangle to replace the edge contour point set, calculate the endpoint coordinates and area of ​​the minimum bounding rectangle of the terminal block area, and the endpoint coordinates and area of ​​the minimum bounding rectangle of the terminal block wiring pipe.

[0091] Further, the second recognition module 104 is used to extract feature sequences from the terminal block image through the text recognition model; predict the feature sequences and output the probability distribution of all characters; select the maximum output probability as the final predicted string to obtain the text recognition result; the text recognition result includes the terminal sequence number located in the terminal block area and the terminal block wiring pipe number located in the terminal block wiring pipe; select the terminal sequence number and the terminal block wiring pipe number by box selection to obtain the terminal sequence number box and the terminal block wiring pipe number box; calculate the coordinates and area of ​​the terminal sequence number box and the terminal block wiring pipe number box.

[0092] Furthermore, the matching module 105 is also used to calculate a first intersection-union ratio between the minimum outer rectangle area of ​​the terminal block region and the area of ​​the terminal sequence number frame. If the first intersection-union ratio is greater than a first preset threshold, the terminal sequence number is matched with the terminal block region. The matching module 105 is also used to calculate a second intersection-union ratio between the minimum outer rectangle area of ​​the terminal block wiring pipe and the area of ​​the terminal block wiring pipe number frame. If the second intersection-union ratio is greater than a second preset threshold, the terminal block wiring pipe number is matched with the terminal block wiring pipe.

[0093] The matching device for terminal blocks and terminal block wiring provided in the above embodiments obtains the terminal block area and terminal block wiring pipe through image recognition, then identifies the string in the image through text recognition model, and identifies which area the string belongs to through intersection-union ratio calculation. It can effectively match the terminal blocks and terminal block wiring, provide an effective basis for checking whether the wiring is correct, and reduce labor costs.

[0094] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention. Clearly, those skilled in the art can make various alterations and modifications to the invention without departing from its spirit and scope. Thus, if these modifications and modifications of the invention fall within the scope of the claims and their equivalents, the invention is also intended to include these modifications and modifications.

Claims

1. A matching method for terminal blocks and terminal block wiring, characterized in that, include: The terminal block image is collected and preprocessed. The terminal block image includes a terminal block and the corresponding terminal block wiring pipe. The terminal block area is marked with a terminal sequence number, and the terminal block wiring pipe is marked with a wiring pipe number. The terminal block images are respectively input into a pre-trained image recognition neural network model and a text recognition model; The image recognition neural network model is used to identify the terminal blocks and terminal block wiring pipes in the terminal block image: the edge contours of the terminal blocks and terminal block wiring pipes in the terminal block image are extracted, and the minimum bounding rectangle is used to replace the edge contour point set. The endpoint coordinates and area of ​​the minimum bounding rectangle of the terminal block area, as well as the endpoint coordinates and area of ​​the minimum bounding rectangle of the terminal block wiring pipe are calculated. The text recognition model identifies the text in the terminal block image by: extracting feature sequences from the terminal block image; predicting the feature sequences and outputting the probability distribution of all characters; selecting the character with the highest output probability as the final predicted string to obtain the text recognition result. The text recognition result includes a first string and a second string; the method further includes: selecting the first string and the second string to obtain a first string frame and a second string frame; and calculating the coordinates and area of ​​the first string frame and the second string frame. The recognition results of the image recognition neural network model and the recognition results of the text recognition model are matched: the first intersection-union ratio (IUU) of the area of ​​the first string frame with the area of ​​the minimum outer rectangle of the terminal block region and the second IUU of the area of ​​the first string frame with the area of ​​the minimum outer rectangle of the terminal block wiring pipe are calculated respectively. If the first IUU is greater than a first preset threshold, the first string is determined to be a terminal sequence number; if the second IUU is greater than a second preset threshold, the first string is determined to be a wiring pipe number. After determining that the first string is a terminal sequence number, the third IUU of the area of ​​the second string with the area of ​​the minimum outer rectangle of the terminal block wiring pipe is calculated. If the third IUU is greater than a third threshold, the second string is determined to be a wiring pipe number. Alternatively, after determining that the first string is a wiring pipe number, the fourth IUU of the area of ​​the second string with the area of ​​the minimum outer rectangle of the terminal block region is calculated. If the fourth IUU is greater than a fourth threshold, the second string is determined to be a terminal sequence number. Obtain the correspondence between terminal blocks, terminal block conduits, and text, and generate a text file.

2. The method according to claim 1, characterized in that, Training the image recognition neural network model includes: Collect training images and label the terminal block area and terminal block wiring pipe area in the training images to form a training set; Establish an image recognition neural network model containing learnable parameters; The training set is input into the image recognition neural network model, and the parameters of the image recognition neural network model are updated based on the gradient descent method until the image recognition neural network model converges.

3. A matching device for terminal blocks and terminal block wiring, characterized in that, include: The preprocessing module is used to acquire and preprocess images of terminal blocks on site. The terminal block image includes a terminal block and a corresponding terminal block wiring pipe. The terminal block area is provided with a terminal sequence number, and the terminal block wiring pipe is provided with a wiring pipe number. The input module is used to input the terminal block image into a pre-trained image recognition neural network model and a text recognition model, respectively; The first identification module is used to identify the terminal block and terminal block wiring pipe in the terminal block image through the image recognition neural network model; The second recognition module is used to recognize the text in the terminal block image through the text recognition model; The matching module is used to match the recognition results of the image recognition neural network model with the recognition results of the text recognition model to obtain the correspondence between the terminal block and the terminal block wiring pipe and their text, and generate a text file; The first recognition module is also used to extract the edge contours of the terminal block and terminal block wiring pipe in the terminal block image through the image recognition neural network model, and use the minimum bounding rectangle to replace the edge contour point set, calculate the endpoint coordinates and area of ​​the minimum bounding rectangle of the terminal block area, and the endpoint coordinates and area of ​​the minimum bounding rectangle of the terminal block wiring pipe. The second recognition module is used to extract feature sequences from the terminal block image using the text recognition model; predict the feature sequences and output the probability distribution of all characters; select the maximum output probability as the final predicted string to obtain the text recognition result; the text recognition result includes a first string and a second string; select the first string and the second string to obtain a first string box and a second string box; calculate the coordinates and area of ​​the first string box and the second string box. The matching module is further configured to calculate the first intersection-union ratio of the area of ​​the first string frame to the area of ​​the minimum outer rectangle of the terminal block region, and the second intersection-union ratio of the area of ​​the first string frame to the area of ​​the minimum outer rectangle of the terminal block wiring pipe. If the first intersection-union ratio is greater than the first preset threshold, the first string is determined to be the terminal sequence number; if the second intersection-union ratio is greater than the second preset threshold, the first string is determined to be the wiring pipe number. After determining that the first string is the terminal sequence number, calculate the third intersection ratio of the area of ​​the second string with the area of ​​the smallest outer rectangle of the terminal block wiring pipe. If the third intersection ratio is greater than the third threshold, then determine that the second string is the wiring pipe number; or, after determining that the first string is the wiring pipe number, calculate the fourth intersection ratio of the area of ​​the second string with the area of ​​the smallest outer rectangle of the terminal block area. If the fourth intersection ratio is greater than the fourth threshold, then determine that the second string is the terminal sequence number.

4. The apparatus according to claim 3, characterized in that, The device further includes a training module for acquiring training images and labeling the terminal block area and terminal block wiring area in the training images to form a training set; establishing an image recognition neural network model containing learnable parameters; inputting the training set into the image recognition neural network model and updating the parameters of the image recognition neural network model based on the gradient descent method until the image recognition neural network model converges.

Images