Transformer Bolt Pose Detection Method, Device, Medium, Program Product and Equipment

Abstract

The present invention belongs to the field of bolt position detection. In order to solve the problem of inaccurate pose detection of bolts in existing transformers, a method, device, medium, program product and equipment for transformer bolt pose detection are provided. Among them, the method for transformer bolt pose detection includes identifying the type of the transformer and the positions of all the bolts exposed on the top surface; searching in the 3D model library for the corresponding 3D model of the transformer and the distribution law of the bolts on the top of the transformer; determining the occlusion area from the two-dimensional image of the transformer top surface, and combining with the corresponding 3D model of the transformer to predict the positions of the bolts occluded on the top surface of the current transformer; determining the 3D coordinates of all the bolts on the top surface of the current transformer; and determining the 6D poses of all the bolts on the top surface of the current transformer according to the 6D pose estimation network, which can adapt to different types of transformers, overcome the interference of complex environments, and accurately estimate the 6D poses.

Description

Technical Field

[0001] The present invention belongs to the field of bolt detection, and particularly relates to a method, device, medium, program product and equipment for detecting the pose of bolts on a transformer. Background Art

[0002] The statements in this part merely provide background technical information related to the present invention and do not necessarily constitute prior art.

[0003] With the rapid development of the power industry, transformers, as indispensable key equipment in the power system, are increasing in number day by day. However, after the end of the service life of transformers, how to carry out efficient and environmentally friendly recycling and treatment has become an urgent problem to be solved. Traditional transformer disassembly and recycling mainly rely on manual operation, which has problems such as low efficiency, high safety hazards, and low resource recovery rate. In order to achieve the automation of transformer recycling, the key lies in realizing the automatic identification and positioning of the positions of bolts on the transformer.

[0004] Currently, object detection and positioning technologies based on machine vision have been widely applied in the field of industrial automation. However, the automatic identification and positioning of the positions of bolts on transformers still face many challenges:

[0005] (1) There are various types of transformers and the bolt positions are not fixed: For different types of transformers, there are significant differences in their top structures, bolt numbers and position distributions, making it difficult to use a unified template for matching and identification.

[0006] (2) Complex environmental interference: The working environment of transformers is complex, and there may be interference factors such as oil stains, rust, and light changes, which affect the accuracy of image acquisition and target recognition. Summary of the Invention

[0007] In order to solve the technical problems existing in the above background art, the present invention provides a method, device, medium, program product and equipment for detecting the pose of bolts on a transformer, which can adapt to different types of transformers, overcome complex environmental interference, and accurately estimate the 6D pose.

[0008] In order to achieve the above object, the present invention adopts the following technical solutions:

[0009] The first aspect of the present invention provides a method for detecting the pose of bolts on a transformer.

[0010] A method for detecting the pose of bolts on a transformer includes:

[0011] Obtaining a two-dimensional image of the top surface of the transformer, extracting the features of the top of the transformer, and combining with a pre-trained target classification model to identify the type of the transformer and the positions of all the exposed bolts on the top surface;

[0012] Based on the characteristics of the transformer top and the identified transformer type, dynamically match the corresponding 3D model of the transformer and the distribution law of the bolts on the transformer top from the 3D model library;

[0013] Determine the occluded area from the two-dimensional image of the transformer top surface, and then combine the corresponding 3D model of the transformer to predict the positions of the occluded bolts on the current transformer top surface;

[0014] Based on the positions of all the exposed bolts on the current transformer top surface, the predicted positions of the occluded bolts on the current transformer top surface, and the distribution law of the bolts on the transformer top, determine the 3D coordinates of all the bolts on the current transformer top surface;

[0015] Based on the 3D coordinates of all the bolts on the current transformer top surface and the pre-trained 6D pose estimation network, determine the 6D poses of all the bolts on the current transformer top surface.

[0016] As an implementation manner, the distribution law of the bolts on the transformer top includes symmetric distribution and uniform distribution.

[0017] As an implementation manner, in training the 6D pose estimation network, use a 3D physics engine to generate simulation data of different 6D poses corresponding to 3D models of various types of transformers, and construct a training database.

[0018] As an implementation manner, the loss function of the 6D pose estimation network is set to , and its expression is:

[0019] ;

[0020] ;

[0021] ;

[0022] Among them, , and are all constant coefficients; is the position loss function; is the angle loss function; is the confidence of the th bolt position on the transformer top surface; is the total number of bolts on the transformer top surface; is the prediction confidence; is the position vector; is the predicted position vector; is the rotation angle vector; is the predicted rotation angle vector.

[0023] As an implementation manner, the 3D model library contains several 3D models of transformers, and each 3D model of transformer contains the 3D coordinates of bolts.

[0024] As an implementation manner, the top features of the transformer include the positions of the edges of the radiator fins and the positions of the terminal blocks.

[0025] The second aspect of the present invention provides a transformer bolt pose detection device.

[0026] A transformer bolt pose detection device includes:

[0027] A type and exposed bolt recognition module, which is used to obtain a two-dimensional image of the top surface of the transformer, extract the top features of the transformer, and combine a pre-trained target classification model to recognize the type of the transformer and the positions of all the exposed bolts on the top surface;

[0028] A dynamic matching module, which is used to dynamically match the corresponding 3D model of the transformer and the distribution law of the bolts on the top of the transformer from the 3D model library based on the top features of the transformer and the recognized type of the transformer;

[0029] An occluded bolt prediction module, which is used to determine the occluded area from the two-dimensional image of the top surface of the transformer, and then combine the corresponding 3D model of the transformer to predict the positions of the occluded bolts on the top surface of the current transformer;

[0030] A 3D coordinate determination module, which is used to determine the 3D coordinates of all the bolts on the top surface of the current transformer based on the positions of all the exposed bolts on the top surface of the current transformer, the predicted positions of the occluded bolts on the top surface of the current transformer, and the distribution law of the bolts on the top of the transformer;

[0031] A 6D pose determination module, which is used to determine the 6D poses of all the bolts on the top surface of the current transformer based on the 3D coordinates of all the bolts on the top surface of the current transformer and a pre-trained 6D pose estimation network.

[0032] The third aspect of the present invention provides a computer-readable storage medium.

[0033] A computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the steps in the transformer bolt pose detection method as described above are implemented.

[0034] The fourth aspect of the present invention provides a computer program product.

[0035] A computer program product includes computer programs / instructions, and when the computer programs / instructions are executed by a processor, the steps in the transformer bolt pose detection method as described above are implemented.

[0036] The fifth aspect of the present invention provides an electronic device.

[0037] An electronic device includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the steps in the transformer bolt pose detection method as described above are implemented.

[0038] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0039] (1) The present invention innovatively proposes a transformer bolt pose detection technology, which combines the two-dimensional image of the transformer top surface with the 3D model to predict the positions of the occluded bolts on the current transformer top surface. According to the positions of all the exposed bolts on the current transformer top surface, the predicted positions of the occluded bolts on the current transformer top surface, and the distribution law of the bolts on the transformer top, the 3D coordinates of all the bolts on the current transformer top surface are determined. Finally, using the 6D pose estimation network, the 6D poses of all the bolts on the current transformer top surface are determined, which can accurately estimate the 6D pose information of the transformer top, solve the problem that it is difficult for traditional methods to directly obtain the target 6D pose, and thus provide an accurate reference framework for the detection of the bolt positions on the transformer top; by simultaneously acquiring the 2D image and the depth map of the transformer top through a 3D structured light camera and combining the target classification network to classify the transformer type, it can effectively overcome the problem of low recognition accuracy of traditional methods in complex environments and lay a foundation for the precise positioning of the bolt positions on the subsequent transformer top.

[0040] (2) The present invention adopts the technical means of matching the bolt positions with the relative coordinates of the 3D model of the transformer top to obtain a high-precision positioning effect of the bolt positions on the transformer top: by combining the trained 6D pose estimation network with the 3D model of the transformer top surface and using the relative coordinate relationship between the bolt positions and the model, the actual positions of the bolts on the transformer top can be quickly and accurately output, meeting the high-precision requirements of automated disassembly.

[0041] The advantages of the additional aspects of the present invention will be partially given in the following description, partially will become obvious from the following description, or will be understood through the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The specification drawings constituting a part of the present invention are used to provide a further understanding of the present invention. The schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation to the present invention.

[0043] Figure 1 is a flowchart of the transformer bolt pose detection method according to an embodiment of the present invention;

[0044] Figure 2 is a schematic structural diagram of the transformer bolt pose detection device according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0045] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0046] It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present invention belongs.

[0047] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should also be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0048] Y indicates that otherwise, the singular form is also intended to include the plural form. In addition, it should also be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0049] Embodiment 1

[0050] Figure 1 A method for detecting the bolt pose of a transformer in an embodiment of the present invention is provided, including:

[0051] S101: Obtain a two-dimensional image of the top surface of the transformer, extract the top features of the transformer, and combine a pre-trained target classification model to identify the type of the transformer and the positions of all exposed bolts on the top surface.

[0052] In the specific implementation process, due to problems such as oil stains, rust, or uneven illumination on the top surface of the transformer, this embodiment uses multi-spectral imaging technology (visible light and infrared light fusion) to obtain a two-dimensional image of the top surface of the transformer and enhances the image quality by combining depth information.

[0053] In step S101, the target classification model selects the ResNet 101 network. Due to the fixed camera and the relatively small changes in the external environment on site, the classification difficulty of transformer type recognition is not high. Therefore, different classification networks can be selected. For example, ResNet 50 can be selected to reduce the amount of calculation.

[0054] Collect two-dimensional RGB images of the top surface of the transformer to form a training database. 500 to 1000 images are collected for each type of transformer.

[0055] The top features of the transformer herein include but are not limited to top dimensions, shapes, colors, etc.

[0056] The type of the transformer can be specifically determined according to the actual situation.

[0057] S102: Dynamically match the corresponding 3D model of the transformer and the distribution law of the bolts on the top of the transformer from the 3D model library based on the characteristics of the top of the transformer and the identified type of the transformer;

[0058] In this embodiment, the distribution law of the bolts on the top of the transformer includes, but is not limited to, symmetric distribution and uniform distribution.

[0059] Among them, in the 3D model library, a three-dimensional point cloud model of the top of each type of transformer is created in advance. It can be made using CAD or pre-3D scanning. The 3D model library contains several 3D models of transformers, and each 3D model of the transformer contains the 3D coordinates of the bolts.

[0060] S103: Determine the occluded area from the two-dimensional image of the top surface of the transformer, and then combine the corresponding 3D model of the transformer to predict the positions of the occluded bolts on the top surface of the current transformer.

[0061] The occluded area here includes, but is not limited to, the radiator area.

[0062] S104: Based on the positions of all the exposed bolts on the top surface of the current transformer, the predicted positions of the occluded bolts on the top surface of the current transformer, and the distribution law of the bolts on the top of the transformer, determine the 3D coordinates of all the bolts on the top surface of the current transformer.

[0063] S105: Based on the 3D coordinates of all the bolts on the top surface of the current transformer and the pre-trained 6D pose estimation network, determine the 6D poses of all the bolts on the top surface of the current transformer.

[0064] In reality, it is very difficult to ensure the accuracy of 6D annotation for 3D point cloud data, and it consumes a lot of time and energy. In this embodiment, a training database is constructed by using a 3D physics engine to generate simulated data with different 6D poses. Since a 3D physics engine is used, the annotation can be easily obtained , which are respectively the 3D coordinates and the angles around the X, Y, and Z axes of the preset coordinate system.

[0065] For each type of 3D model of the transformer, simulated data with different 6D poses is generated using a 3D physics engine.

[0066] Among them, the loss function of the 6D pose estimation network is set as , and its expression is:

[0067] ;

[0068] ;

[0069] ;

[0070] Among them, , and are all constant coefficients; is the position loss function; is the angle loss function; is the confidence of the th bolt position on the top surface of the transformer; is the total number of bolts on the top surface of the transformer; is the predicted confidence; is the position vector; is the predicted position vector; is the rotation angle vector; is the predicted rotation angle vector.

[0071] This embodiment adopts a technical means combining simulated data generation and deep learning, and can obtain an efficient and low-cost model training effect: a large amount of simulated data is generated through physical simulation, avoiding the high cost and high difficulty problems of real data acquisition. At the same time, combined with deep learning technology, the generalization ability and adaptability of the model are significantly improved, ensuring the accuracy and robustness of the detection of the bolt positions on the top of the transformer.

[0072] This embodiment combines the two-dimensional image of the top surface of the transformer with the 3D model to predict the positions of the occluded bolts on the current top surface of the transformer. According to the positions of all the exposed bolts on the current top surface of the transformer, the predicted positions of the occluded bolts on the current top surface of the transformer, and the distribution law of the bolts on the top of the transformer, the 3D coordinates of all the bolts on the current top surface of the transformer are determined. Finally, using the 6D pose estimation network, the 6D poses of all the bolts on the current top surface of the transformer are determined, which can accurately estimate the 6D pose information of the top of the transformer and solve the problem that it is difficult for traditional methods to directly obtain the 6D pose of the target, thus providing an accurate reference framework for the detection of the bolt positions on the top of the transformer; by simultaneously obtaining the 2D image and the depth map of the top of the transformer through a 3D structured light camera and combining the target classification network to classify the type of the transformer, the problem of low recognition accuracy of traditional methods in complex environments can be effectively overcome, laying a foundation for the precise positioning of the bolt positions on the top of the transformer in the future.

[0073] This embodiment realizes the high-precision and high-robustness detection of the bolt positions on the top of the transformer through technical means such as multi-modal data fusion, typed 3D model construction, 6D pose estimation network training, and bolt position matching, providing reliable technical support for the automatic disassembly and recycling of the transformer.

[0074] Embodiment 2

[0075] As Figure 2 shown, this embodiment of the present invention provides a transformer bolt pose detection device, including:

[0076] Type and Exposed Bolt Recognition Module 201, which is used to obtain the two-dimensional image of the transformer top surface, extract the top features of the transformer, and combine with the pre-trained target classification model to identify the type of the transformer and the positions of all exposed bolts on the top surface;

[0077] Dynamic Matching Module 202, which is used to dynamically match the corresponding 3D model of the transformer and the distribution law of the bolts on the transformer top surface from the 3D model library based on the top features of the transformer and the identified type of the transformer;

[0078] Occluded Bolt Prediction Module 203, which is used to determine the occlusion area from the two-dimensional image of the transformer top surface, and then combine with the corresponding 3D model of the transformer to predict the positions of the occluded bolts on the current transformer top surface;

[0079] 3D Coordinate Determination Module 204, which is used to determine the 3D coordinates of all bolts on the current transformer top surface based on the positions of all exposed bolts on the current transformer top surface, the predicted positions of the occluded bolts on the current transformer top surface, and the distribution law of the bolts on the transformer top surface;

[0080] 6D Pose Determination Module 205, which is used to determine the 6D poses of all bolts on the current transformer top surface based on the 3D coordinates of all bolts on the current transformer top surface and the pre-trained 6D pose estimation network.

[0081] It should be noted here that each module in the embodiments of the present invention corresponds to each step in Embodiment 1 one by one, and the specific implementation process is the same, so it will not be elaborated here.

[0082] Embodiment 3

[0083] This embodiment provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the steps in the transformer bolt pose detection method as described above.

[0084] Embodiment 4

[0085] This embodiment provides a computer program product, including a computer program / instructions, and when the computer program / instructions are executed by a processor, it implements the steps in the transformer bolt pose detection method as described above.

[0086] Embodiment 5

[0087] This embodiment provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it implements the steps in the transformer bolt pose detection method as described above.

[0088] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or means for implementing the functions specified in one block or multiple blocks.

[0089] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for detecting the pose of bolts of a transformer, characterized in that, Including: Obtain a two-dimensional image of the top surface of the transformer, extract the top features of the transformer, and combine with a pre-trained target classification model to identify the type of the transformer and the positions of all exposed bolts on the top surface; Based on the top features of the transformer and the identified type of the transformer, dynamically match the corresponding 3D model of the transformer and the distribution law of the bolts on the top of the transformer from the 3D model library; Determine the occlusion area from the two-dimensional image of the top surface of the transformer, and then combine with the corresponding 3D model of the transformer to predict the positions of the occluded bolts on the top surface of the current transformer; Based on the positions of all exposed bolts on the top surface of the current transformer, the predicted positions of the occluded bolts on the top surface of the current transformer, and the distribution law of the bolts on the top of the transformer, determine the 3D coordinates of all bolts on the top surface of the current transformer; Based on the 3D coordinates of all bolts on the top surface of the current transformer and a pre-trained 6D pose estimation network, determine the 6D poses of all bolts on the top surface of the current transformer; Among them, the loss function of the 6D pose estimation network is obtained by calculating the weighted sum of the square of the difference between the confidence corresponding to each bolt position on the top surface of the transformer and the predicted confidence, the position loss function, and the angle loss function, and then accumulating the weighted sums corresponding to each bolt position.

2. The transformer bolt pose detection method according to claim 1, characterized in that The distribution law of the bolts on the top of the transformer includes symmetric distribution and uniform distribution.

3. The transformer bolt pose detection method according to claim 1, characterized in that, In training the 6D pose estimation network, use a 3D physics engine to generate simulation data of different 6D poses corresponding to 3D models of various types of transformers, and construct a training database.

4. The transformer bolt pose detection method according to claim 1, characterized in that, The loss function of the 6D pose estimation network is set as , and its expression is: ； ； ； Among them, , and are all constant coefficients; is the position loss function; is the angle loss function; is the confidence of the th bolt position on the top surface of the transformer; is the total number of bolts on the top surface of the transformer; is the predicted confidence; is the position vector; is the predicted position vector; is the rotation angle vector; is the predicted rotation angle vector.

5. The transformer bolt pose detection method according to claim 1, wherein, The 3D model library contains several 3D models of transformers, and each 3D model of the transformer includes 3D coordinates of bolts.

6. The transformer bolt pose detection method according to claim 1, characterized in that, The top features of the transformer include the edge position of the radiator and the position of the terminal block.

7. A transformer bolt pose detection device, characterized in that, Including: A type and exposed bolt recognition module, which is used to obtain a two-dimensional image of the top surface of the transformer, extract the top features of the transformer, and combine with a pre-trained target classification model to identify the type of the transformer and the positions of all exposed bolts on the top surface; A dynamic matching module, which is used to dynamically match the corresponding 3D model of the transformer and the distribution law of the bolts on the top of the transformer from the 3D model library based on the top features of the transformer and the identified type of the transformer; An occluded bolt prediction module, which is used to determine the occlusion area from the two-dimensional image of the top surface of the transformer, and then combine with the corresponding 3D model of the transformer to predict the positions of the occluded bolts on the top surface of the current transformer; A 3D coordinate determination module, which is used to determine the 3D coordinates of all bolts on the top surface of the current transformer based on the positions of all exposed bolts on the top surface of the current transformer, the predicted positions of the occluded bolts on the top surface of the current transformer, and the distribution law of the bolts on the top of the transformer; A 6D pose determination module, which is used to determine the 6D poses of all bolts on the top surface of the current transformer based on the 3D coordinates of all bolts on the top surface of the current transformer and a pre-trained 6D pose estimation network; Among them, the loss function of the 6D pose estimation network is the weighted sum of the sum of the squares of the differences between the confidence levels corresponding to the positions of the bolts on the top surface of the transformer and the predicted confidence levels, the position loss function, and the angle loss function, and then the weighted sums corresponding to the positions of the bolts are accumulated and calculated.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by a processor, it implements the steps in the transformer bolt pose detection method according to any one of claims 1-6.

9. A computer program product, comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by a processor, it implements the steps in the transformer bolt pose detection method according to any one of claims 1-6.

10. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the transformer bolt pose detection method according to any one of claims 1-6.

Images