A method for counting materials of distribution poles

Abstract

The present invention relates to the field of material counting. The present invention discloses a method for counting materials for distribution poles and towers, which includes the following steps: Step 1: Use a drone to collect data of distribution poles and towers; Step 2: Mark the detection frames of the materials in each picture in D to obtain the detection data set Ddet; Step 3: Mark the material detection frames in each pole and tower in D; Step 5: Use the Dmatch data to train the cross-view frame matching model Match; Step 6: Obtain the final number of each material; The present invention utilizes computer vision for counting, realizes the material counting function based on drone flight inspection, avoids problems such as slow efficiency and misjudgment in manual counting, uses a drone to fly around the distribution pole and tower to collect data, and the included angle between adjacent pictures is not greater than θ degrees, ensuring the full coverage of the materials around the pole and tower, being able to obtain the image information of all angles of the pole and tower, avoiding the omission of materials caused by perspective limitations, having high collection efficiency and comprehensive data, and laying a solid foundation for subsequent accurate counting.

Description

Technical Field

[0001] The present invention relates to the technical field of material counting, and particularly relates to a method for counting materials on distribution poles and towers. Background Art

[0002] Distribution poles and towers contain many materials, such as insulators, wire clamps, etc. The function of each material is very important. At the same time, each pole and tower has a standard, that is, how many of each material should be on the pole and tower. Therefore, when the construction of the pole and tower is completed or during daily inspections, it is necessary to count the number of materials on the pole and tower to avoid disasters caused by missing materials.

[0003] Manually counting the number of materials on each pole and tower is cumbersome and time-consuming. Using drones for inspections combined with computer vision algorithms can facilitate and quickly count. Since there are many targets in the pole and tower, when taking a single picture, some materials may be blocked, resulting in inability to count. Based on this, we designed to take pictures of the same pole and tower from multiple perspectives and combine these pictures for material counting.

[0004] Based on this, a method for counting materials on distribution poles and towers is proposed here to solve the above problems. Summary of the Invention

[0005] In order to overcome the above technical problems, the purpose of the present invention is to provide a method for counting materials on distribution poles and towers, which uses computer vision for counting to achieve the function of material counting based on drone flight, and avoids problems such as slow efficiency and misjudgment in manual counting. The purpose of the present invention can be achieved through the following technical solutions:

[0006] A method for counting materials on distribution poles and towers includes the following steps:

[0007] Step 1: Use a drone to collect data of the distribution pole and tower Wherein i represents the i-th pole and tower, and 0 represents the first picture taken of the i-th pole and tower. 1 represents the second picture taken of the i-th pole and tower, and so on. Here, it is required that the pictures collected for each pole and tower are taken around the pole and tower, and the included angle between adjacent pictures is not greater than θ degrees;

[0008] Step 2: Mark the detection frames of the materials in each picture in D to obtain the detection data set Ddet;

[0009] Step 3: Mark the relationship of the material detection frames in each pole and tower in D in the corresponding pictures to obtain the matching relationship data set Dmatch;

[0010] Step 4: Train the material detection model Det using Ddet data;

[0011] Step 5: Train the cross - perspective box matching model Match using Dmatch data;

[0012] Step 6: Input the image to be tested into the trained Det to obtain the detection result Input the detection result into the trained matching model Match to obtain the result of the label of each box

[0013] According to the cls result and the instance result, obtain the final number of each material.

[0014] Preferably, for the said Step 2, it includes the following steps:

[0015] Step 2.1: The annotation format is the standard detection format, denoted as where k represents the k - th pole tower, represents the first picture taken in the k - th pole tower, represents the first material box annotated in the first picture taken in the k - th pole tower, represents the category to which the first material box annotated in the first picture taken in the k - th pole tower belongs, and so on for others.

[0016] Preferably, for the said Step 3, it includes the following steps:

[0017] Step 3.1: For the relationship between the annotation and where where represents whether the material in the first picture taken in the k - th pole tower appears in the second picture taken in the k - th pole tower. If it appears, record its index in the second picture as idx. Taking idx0 as an example, if and belong to the same material, then idx0 = 1. If does not appear in the second picture, then idx0 = - 1.

[0018] Preferably, for the said Step 4, it includes the following steps:

[0019] Step 4.1: The training of the material detection model uses the commonly used yolov8 model in the field to train and obtain Det.

[0020] Preferably, it includes the following steps:

[0021] Step 5.1: During training, a batch of b samples are input into the network for training. Here, we take b = 1 as an example to illustrate the input: Select the i-th and j-th pictures in the k-th pole tower, then there is

[0022] According to It can be obtained that is an M*N matrix, and the value at the (ii, jj) position represents whether the ii-th box in and the jj-th box in

[0023] Preferably, Step 5.2: Input the i-th picture into a backbone network to obtain Fi, input the j-th picture into a backbone network to obtain Fj, and use ROIAlign according to to obtain the features of each material box Input this feature, box information, and cls information into the subsequent information interaction network to obtain the predicted relationship between boxes Subsequently, it is optimized based on the BCE loss function commonly used in the field; finally, a converged model Match is obtained.

[0024] Preferably, for the said Step 6, it includes the following steps:

[0025] Step 6.1: Input L + 1 pictures taken by a certain pole tower Here, we require that the L + 1 pictures are taken around the pole tower, and the included angle between adjacent pictures is not greater than θ degrees. They are respectively input into the trained Det to obtain

[0026]

[0027] Preferably, Step 6.2: Traverse and input the i-th picture and its material detection result and the j-th picture and its material detection result into the Match model to obtain the predicted relationship Assign a label instance to each box, where the value of instance is the label referring to a certain material, and the label of each material is independent; the order of traversing the pictures is (0, 1), (0, 2)..., (0, L), (1, 2), (1, 3)....(1, L),...., (L - 1, L); where (0, 1) means and And so on for others;

[0028] Step 6.3: Count the number of allocated tags to finally obtain the quantity of each material.

[0029] Advantages of the present invention:

[0030] 1. Utilize computer vision counting to achieve the material counting function based on drone cruise flight, avoiding problems such as slow efficiency and misjudgment in manual counting. The drone is used to collect data by flying around the distribution power pole, and the angle between adjacent pictures is not greater than θ degrees, ensuring the full coverage of the materials around the pole, being able to obtain image information of each angle of the pole, avoiding the omission of materials caused by limited perspective, with high collection efficiency and comprehensive data, laying a solid foundation for subsequent accurate counting.

[0031] 2. The annotation of the material detection frame in the picture adopts a standard detection format, clarifying the corresponding relationship between the pole, the picture, the material frame, and the category to which it belongs, making the annotation data highly standardized and consistent, facilitating subsequent data processing and model training, and improving the quality and usability of the data.

[0032] 3. When counting materials, first input the picture into the material detection model to obtain the detection result, then input the detection result into the cross-perspective frame matching model to obtain the label of each frame, and finally count the material quantity according to the label. The whole process is rigorous and logically clear, which can effectively reduce the counting error, improve the accuracy and reliability of counting, and provide accurate data support for the material management of the distribution power pole. Description of the Drawings

[0033] The present invention will be further described below in conjunction with the drawings.

[0034] Figure 1 It is the data collection and model training flow chart of the present invention;

[0035] Figure 2 It is the material counting flow chart of the present invention. Detailed Embodiments

[0036] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0037] Embodiment

[0038] Please refer to Figure 1 - Figure 2As shown in the figure, a method for counting materials of distribution poles includes:

[0039] Step 1: Use a drone to collect data of distribution poles Among them i in represents the i-th pole, 0 represents the first picture taken in the i-th pole, 1 in represents the second picture taken in the i-th pole, and so on. Here, it is required that the pictures collected for each pole are taken around the pole, and the included angle between adjacent pictures is not greater than θ degrees; in this embodiment, we collect 3 pictures for each pole; in actual use, other numbers of pictures can be collected, and the number of pictures collected for each pole can be different, and θ is taken as 45;

[0040] Step 2: Mark the detection frames of the materials in each picture in D to obtain the detection data set Ddet;

[0041] Step 3: Mark the relationship of the material detection frames (already marked in Step 2) in each pole in D in the corresponding pictures to obtain the matching relationship data set Dmatch;

[0042] Step 4: Use the Ddet data to train the material detection model Det; the detection model selected in this embodiment is YOLOv8;

[0043] Step 5: Use the Dmatch data to train the cross-view frame matching model Match;

[0044] Step 6: Input the picture to be tested into the trained Det to obtain the detection result Input the detection result into the trained matching model Match to obtain the result of the label of each frame

[0045] According to the cls result and the instance result, obtain the final number of each material;

[0046] For Step 2, it includes the following steps:

[0047] Step 2.1: The marking format is the standard detection format, denoted as where k represents the k-th pole, represents the first picture taken in the k-th pole, represents the first material frame marked in the first picture taken in the k-th pole, Denote the category to which the first material frame labeled in the first picture taken in the k-th pole tower belongs. Similarly for others; in this embodiment, the labeled materials are insulators, cross arms, and wire clamps. That is, the value range of cls is insulators, cross arms, and wire clamps. More material types can be labeled in actual use;

[0048] For step 3, it includes the following steps:

[0049] Step 3.1: For Label And The relationship between Among them Denote whether the material in the first picture taken in the k-th pole tower appears in the second picture taken in the k-th pole tower. If it appears, then record its index in the second picture as idx. Taking idx0 as an example, if And Belong to the same material, then idx0 = 1. If Does not appear in the second picture, then idx0 = -1;

[0050] For step 4, it includes the following steps:

[0051] Step 4.1: The training of the material detection model is carried out using the widely used yolov8 model in the field to obtain Det;

[0052] For step 5, it includes the following steps:

[0053] Step 5.1: During training, b samples are input into the network for training. Here, we take b = 1 as an example to illustrate the input: Select the i-th picture and the j-th picture in the k-th pole tower, then there is According to Can obtain Is an M*N matrix, and the value at the (ii, jj) position represents Whether the ii-th frame in And the jj-th frame in

[0054] Belong to the same material. If it is the same material, the corresponding value is 1. If not, the corresponding value is 2; Input the i-th picture into a backbone network to obtain Fi, input the j-th picture into a backbone network to obtain Fj, and use ROIAlign according to Input the feature, box information, and cls information into the subsequent information interaction network to obtain the predicted relationships between boxes. Subsequently, it is optimized based on the BCE loss function commonly used in the field; finally, a converged model Match is obtained. In this embodiment, the subsequent information interaction network is composed of transformer layers, and other networks such as CNN or GNN can also be used.

[0055] For step 6, it includes the following steps:

[0056] Step 6.1: Input L + 1 images taken of a certain pole tower. Here, we require that the L + 1 images are taken around the pole tower, and the angle between adjacent images is not greater than θ degrees. They are respectively input into the trained Det to obtain In this embodiment, L is taken as 3 and θ is taken as 45.

[0057] Step 6.2: Traverse and input the i-th image and its material detection result, and the j-th image and its material detection result into the Match model to obtain the predicted relationship. Assign a label instance to each box, where the value of instance is a label referring to a certain material (e.g., insulator 1, insulator 2, clamp 1, clamp 2...). The labels of each material are independent. The order of traversing the images is (0, 1), (0, 2),..., (0, L), (1, 2), (1, 3),...., (1, L),...., (L - 1, L); where (0, 1) means and And so on for the others.

[0058] Step 6.3: Count the number of assigned labels to finally obtain the quantity of each material.

[0059] The preferred embodiments of the present invention disclosed above are only used to help illustrate the present invention. The preferred embodiments do not describe all details in detail, nor do they limit the invention to only the specific embodiments. Obviously, many modifications and changes can be made according to the content of this specification. These embodiments are selected and specifically described in this specification to better explain the principle and practical application of the present invention, so that those skilled in the art can well understand and utilize the present invention. The present invention is only limited by the claims and their full scope and equivalents.

Claims

1. A method for counting materials of distribution poles and towers, characterized in that Including the following steps: Step 1: Use a drone to collect distribution tower data where i in it represents the i-th tower, 0 represents the first picture taken in the i-th tower, 1 in it represents the second picture taken in the i-th tower, and so on. Here, it is required that the pictures collected for each tower are taken around the tower, and the included angle between adjacent pictures is not greater than θ degrees; Step 2: Mark the detection boxes of the materials in each picture in D to obtain the detection dataset Ddet; Step 3: Mark the relationship between the material detection frames in each tower pole in D and the corresponding images to obtain the matching relationship dataset Dmatch; ​ Step 4: Use the Ddet data to train the material detection model Det; Step 5: Use the Dmatch data to train the cross-view box matching model Match; Step 6: Input L + 1 pictures to be tested into the trained Det which respectively represent the 1st, 2nd, …, Lth pictures taken of the pole tower, and obtain the detection results Traverse the input of the i-th picture and its detection result and the j-th picture and its detection result into the Match model. The order of traversing the pictures is (0, 1), (0, 2), …, (0, L), (1, 2), (1, 3), …, (1, L), …, (L - 1, L); where (0, 1) means and and so on for the others, and obtain the result of the label for each box where the value of instance is the label referring to a certain material, and the label of each material is independent; according to the cls result and the instance result, obtain the final number of each material.

2. For the method for counting materials of a distribution pole and tower according to claim 1, for the said step 2, it includes the following steps: Step 2.1: The annotation format is the standard detection format, denoted as where k represents the k-th pole tower, represents the first picture taken in the k-th pole tower, represents the first material frame annotated in the first picture taken in the k-th pole tower, represents the category to which the first material frame annotated in the first picture taken in the k-th pole tower belongs.

3. For the method for counting materials of a distribution pole and tower according to claim 1, for the said step 3, it includes the following steps: Step 3.1: For annotation and the relationship between where represents whether the material in the first picture taken in the k-th tower appears in the second picture taken in the k-th tower. If it appears, the index of it in the second picture is denoted as idx. Taking idx0 as an example, if and belong to the same material, then idx0 = 1. If does not appear in the second picture, then idx0 = -1.

4. For the method for counting materials of a distribution pole and tower according to claim 1, for the said step 4, it includes the following steps: Step 4.1: The training of the material detection model is carried out using the yolov8 model commonly used in the field to obtain Det.

5. For the method for counting materials of a distribution pole and tower according to claim 1, for the said step 5, it includes the following steps: Step 5.1: During training, a batch of b samples is input into the network for training. Here, we take b = 1 as an example to illustrate the input: Select the i-th and j-th pictures in the k-th tower pole. Then, we have According to we can obtain is an M*N matrix, and the value at the (ii,jj) position represents whether the ii-th box in and the jj-th box in belong to the same material. If they belong to the same material, the corresponding value is 1; otherwise, the corresponding value is 2.

6. The method for counting materials of a distribution pole according to claim 5, step 5.2: Input the i-th picture into a backbone network to obtain Fi, input the j-th picture into a backbone network to obtain Fj, and use ROIAlign according to to obtain the features of each material box Input the feature, box information, and cls information into the subsequent information interaction network to obtain the predicted relationship between boxes Subsequently, optimize based on the BCE loss function commonly used in the field; finally, obtain the converged model Match.

Images