Transformer bolt pose detection method and device, medium, program product and equipment

Abstract

The invention belongs to the field of bolt position detection, and provides a transformer bolt pose detection method and device, a medium, a program product and equipment in order to solve the problem that existing transformer bolt pose detection is inaccurate. The transformer bolt pose detection method comprises the following steps: identifying the type of a transformer and the positions of all exposed bolts on the top surface; searching and matching a corresponding transformer 3D model and a distribution rule of bolts at the top of the transformer from a 3D model library; determining a sheltered area from the two-dimensional image of the top surface of the transformer, and predicting the position of a sheltered bolt on the top surface of the current transformer in combination with the corresponding transformer 3D model; determining 3D coordinates of all bolts on the top surface of the current transformer; and the 6D poses of all the bolts on the top surface of the current transformer are determined according to the 6D pose estimation network, and the method can adapt to different types of transformers, overcome complex environment interference and accurately estimate the 6D poses.

Description

Technical Field

[0001] The present invention belongs to the field of bolt detection, and in particular relates to a transformer bolt posture detection method, device, medium, program product and equipment. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] With the rapid development of the power industry, transformers are an indispensable key equipment in the power system, and their number is increasing day by day. However, how to recycle transformers efficiently and environmentally friendly after the end of their service life has become an urgent problem to be solved. Traditional transformer disassembly and recycling mainly relies on manual operation, which has problems such as low efficiency, great safety hazards, and low resource recovery rate. In order to realize the automation of transformer recycling, the key is to realize the automatic identification and positioning of the transformer bolt position.

[0004] At present, target detection and positioning technology based on machine vision has been widely used in the field of industrial automation. However, the automatic identification and positioning of transformer bolt positions still faces many challenges: (1) Transformer types are diverse and bolt positions are not fixed: Different types of transformers have large differences in top structure, number of bolts, and position distribution, making it difficult to use a unified template for matching and identification.

[0005] (2) Complex environmental interference: The working environment of the transformer is complex, and there may be interference factors such as oil pollution, rust, and light changes, which may affect the accuracy of image acquisition and target recognition. Summary of the invention

[0006] In order to solve the technical problems existing in the above-mentioned background technology, the present invention provides a transformer bolt posture detection method, device, medium, program product and equipment, which can adapt to different types of transformers, overcome complex environmental interference, and accurately estimate 6D posture.

[0007] In order to achieve the above object, the present invention adopts the following technical solution: A first aspect of the present invention provides a method for detecting the bolt posture of a transformer.

[0008] A transformer bolt posture detection method, comprising: Obtain a two-dimensional image of the transformer top surface, extract the transformer top features, and combine with the pre-trained target classification model to identify the type of transformer and the locations of all exposed bolts on the top surface; Based on the transformer top features and the identified transformer type, dynamically match the corresponding transformer 3D model and the distribution pattern of the transformer top bolts from the 3D model library; Determine the blocked area from the two-dimensional image of the transformer top surface, and then predict the position of the blocked bolts on the current transformer top surface in combination with the corresponding transformer 3D model; Based on the positions of all exposed bolts on the current transformer top surface, the predicted positions of the blocked bolts on the current transformer top surface, and the distribution pattern of the bolts on the transformer top, the 3D coordinates of all bolts on the current transformer top surface are determined; Based on the 3D coordinates of all bolts on the current transformer top surface and the pre-trained 6D pose estimation network, the 6D poses of all bolts on the current transformer top surface are determined.

[0009] As an implementation mode, the distribution pattern of the transformer top bolts includes symmetrical distribution and uniform distribution.

[0010] As an implementation mode, in training the 6D pose estimation network, a 3D physics engine is used to generate simulation data of different 6D poses corresponding to various types of transformer 3D models to construct a training database.

[0011] As an implementation, the loss function of the 6D pose estimation network is set to , whose expression is: ; ; ; in, , and All are constant coefficients; is the position loss function; is the angle loss function; The first Confidence level of each bolt position; is the total number of bolts on the top surface of the transformer; is the prediction confidence; is the position vector; is the predicted position vector; is the rotation angle vector; is the predicted rotation angle vector.

[0012] As an implementation manner, the 3D model library includes several transformer 3D models, and each transformer 3D model includes bolt 3D coordinates.

[0013] As an implementation manner, the transformer top features include heat sink edge positions and terminal positions.

[0014] A second aspect of the present invention provides a transformer bolt posture detection device.

[0015] A transformer bolt posture detection device, comprising: Type and exposed bolt recognition module, which is used to obtain a two-dimensional image of the transformer top surface, extract the transformer top features, and identify the type of transformer and the locations of all exposed bolts on the top surface in combination with a pre-trained target classification model; A dynamic matching module is used to dynamically match the corresponding transformer 3D model and the distribution pattern of the transformer top bolts from the 3D model library based on the transformer top features and the identified transformer type; A blocked bolt prediction module is used to determine the blocked area from the two-dimensional image of the transformer top surface, and then predict the position of the blocked bolt on the current transformer top surface in combination with the corresponding transformer 3D model; A 3D coordinate determination module is used to determine the 3D coordinates of all bolts on the top surface of the current transformer based on the positions of all exposed bolts on the top surface of the current transformer, the predicted positions of the blocked bolts on the top surface of the current transformer, and the distribution pattern of the bolts on the top of the transformer; The 6D pose determination module is used to determine the 6D pose of all bolts on the top surface of the current transformer based on the 3D coordinates of all bolts on the top surface of the current transformer and a pre-trained 6D pose estimation network.

[0016] A third aspect of the present invention provides a computer-readable storage medium.

[0017] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps in the transformer bolt posture detection method as described above.

[0018] A fourth aspect of the present invention provides a computer program product.

[0019] A computer program product includes a computer program / instruction, which, when executed by a processor, implements the steps in the transformer bolt posture detection method as described above.

[0020] A fifth aspect of the present invention provides an electronic device.

[0021] An electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in the transformer bolt posture detection method as described above are implemented.

[0022] Compared with the prior art, the present invention has the following beneficial effects: (1) The present invention innovatively proposes a transformer bolt posture detection technology, which combines the two-dimensional image of the transformer top surface with the 3D model to predict the position of the currently obscured bolts on the transformer top surface. According to the position of all exposed bolts on the transformer top surface, the predicted position of the currently obscured bolts on the transformer top surface and the distribution law of the bolts on the transformer top, the 3D coordinates of all bolts on the transformer top surface are determined. Finally, the 6D posture estimation network is used to determine the 6D posture of all bolts on the transformer top surface. The 6D posture information of the transformer top can be accurately estimated, which solves the problem that the traditional method is difficult to directly obtain the target 6D posture, thereby providing an accurate reference frame for the detection of the bolt position on the transformer top. The 2D image and depth map of the transformer top are simultaneously obtained by the 3D structured light camera, and the transformer type is classified by combining the target classification network. This can effectively overcome the problem of low recognition accuracy of the traditional method in complex environments, and lay the foundation for the subsequent precise positioning of the bolt position on the transformer top.

[0023] (2) The present invention adopts the technical means of matching the relative coordinates of the bolt position with the 3D model of the transformer top to obtain a high-precision positioning effect of the bolt position on the transformer top: by combining the trained 6D pose estimation network with the 3D model of the transformer top surface, and utilizing the relative coordinate relationship between the bolt position and the model, the actual position of the bolt on the transformer top can be output quickly and accurately, meeting the high-precision requirements of automated disassembly.

[0024] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0026] Figure 1 is a flow chart of a transformer bolt posture detection method according to an embodiment of the present invention; Figure 2 It is a schematic structural diagram of a transformer bolt posture detection device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0027] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0028] It should be noted that the following detailed descriptions are all illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0029] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly states otherwise, Y points out that otherwise the singular form is intended to include the plural form. In addition, it should be understood that when the terms "include" and / or "comprises" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or their combinations.

[0030] Embodiment 1 Figure 1 A transformer bolt posture detection method according to an embodiment of the present invention is provided, comprising: S101: Obtain a two-dimensional image of the transformer top surface, extract the transformer top features, and identify the type of transformer and the positions of all exposed bolts on the top surface by combining with a pre-trained target classification model.

[0031] In the specific implementation process, since there may be problems such as oil pollution, rust or uneven lighting on the top surface of the transformer, this embodiment adopts multispectral imaging technology (fusion of visible light and infrared light) to obtain a two-dimensional image of the top surface of the transformer, and combines depth information to enhance the image quality.

[0032] In step S101, the target classification model selects the ResNet 101 network. Due to the fixed nature of the camera and relatively small changes in the external environment on site, the classification difficulty of transformer type recognition is not high, so different classification networks can be selected, such as ResNet 50, to reduce the amount of calculation.

[0033] Collect 2D RGB images of the top surface of the transformer to make a training database. Collect 500 to 1000 images for each type of transformer.

[0034] The transformer top features herein include but are not limited to top size, shape, color, etc.

[0035] The type of transformer can be classified according to actual conditions.

[0036] S102: dynamically matching a corresponding transformer 3D model and a distribution pattern of transformer top bolts from a 3D model library based on the transformer top features and the identified transformer type; In this embodiment, the distribution pattern of the transformer top bolts includes but is not limited to symmetrical distribution and uniform distribution.

[0037] In the 3D model library, a 3D point cloud model of the top of each type of transformer is pre-created. It can be made using CAD or pre-3D scanning. The 3D model library contains several types of transformer 3D models, and each transformer 3D model contains bolt 3D coordinates.

[0038] S103: Determine the blocked area from the two-dimensional image of the transformer top surface, and then predict the position of the blocked bolts on the current transformer top surface in combination with the corresponding transformer 3D model.

[0039] The shielding area here includes but is not limited to the heat sink area.

[0040] S104: Based on the positions of all exposed bolts on the current transformer top surface, the predicted positions of the blocked bolts on the current transformer top surface, and the distribution pattern of the bolts on the transformer top, the 3D coordinates of all bolts on the current transformer top surface are determined.

[0041] S105: Based on the 3D coordinates of all the bolts on the current transformer top surface and a pre-trained 6D pose estimation network, determine the 6D pose of all the bolts on the current transformer top surface.

[0042] In reality, it is difficult to ensure the accuracy of 6D annotation of 3D point cloud data, and it takes a lot of time and effort. This embodiment uses a 3D physics engine to generate simulated data with different 6D poses to build a training database. , which are the 3D coordinates and the angles around the X, Y, and Z axes of the preset coordinate system.

[0043] For each type of transformer 3D model, a 3D physics engine is used to generate simulation data with different 6D poses.

[0044] Among them, the loss function of the 6D pose estimation network is set to , whose expression is: ; ; ; in, , and All are constant coefficients; is the position loss function; is the angle loss function; The first Confidence level of each bolt position; is the total number of bolts on the top surface of the transformer; is the prediction confidence; is the position vector; is the predicted position vector; is the rotation angle vector; is the predicted rotation angle vector.

[0045] This embodiment adopts a technical means that combines simulation data generation with deep learning to achieve efficient and low-cost model training effects: a large amount of simulation data is generated through physical simulation, avoiding the high cost and difficulty of real data collection. At the same time, combined with deep learning technology, the generalization ability and adaptability of the model are significantly improved, ensuring the accuracy and robustness of the transformer top bolt position detection.

[0046] This embodiment combines the two-dimensional image of the transformer top surface with the 3D model to predict the position of the currently obscured bolts on the transformer top surface. According to the positions of all exposed bolts on the current transformer top surface, the predicted positions of the currently obscured bolts on the transformer top surface and the distribution pattern of the bolts on the transformer top, the 3D coordinates of all bolts on the current transformer top surface are determined. Finally, the 6D pose estimation network is used to determine the 6D pose of all bolts on the current transformer top surface, which can accurately estimate the 6D pose information of the transformer top, solving the problem that the traditional method is difficult to directly obtain the target 6D pose, thereby providing an accurate reference frame for the detection of the bolt position on the transformer top. The 2D image and depth map of the transformer top are simultaneously obtained by the 3D structured light camera, and the transformer type is classified in combination with the target classification network, which can effectively overcome the problem of low recognition accuracy of the traditional method in complex environments, and lay the foundation for the subsequent precise positioning of the bolt position on the transformer top.

[0047] This embodiment achieves high-precision and high-robustness detection of the bolt position on the top of the transformer through technical means such as multimodal data fusion, typed 3D model construction, 6D pose estimation network training, and bolt position matching, providing reliable technical support for the automated disassembly and recycling of the transformer.

[0048] Embodiment 2 like Figure 2 As shown, an embodiment of the present invention provides a transformer bolt posture detection device, comprising: The type and exposed bolt identification module 201 is used to obtain a two-dimensional image of the transformer top surface, extract the transformer top features, and identify the type of the transformer and the positions of all exposed bolts on the top surface in combination with a pre-trained target classification model; A dynamic matching module 202 is used to dynamically match the corresponding transformer 3D model and the distribution pattern of the transformer top bolts from the 3D model library based on the transformer top features and the identified transformer type; The blocked bolt prediction module 203 is used to determine the blocked area from the two-dimensional image of the transformer top surface, and then predict the position of the blocked bolt on the current transformer top surface in combination with the corresponding transformer 3D model; 3D coordinate determination module 204, which is used to determine the 3D coordinates of all bolts on the current transformer top surface based on the positions of all exposed bolts on the current transformer top surface, the predicted positions of the blocked bolts on the current transformer top surface, and the distribution pattern of the bolts on the transformer top; The 6D pose determination module 205 is used to determine the 6D pose of all bolts on the current transformer top surface based on the 3D coordinates of all bolts on the current transformer top surface and a pre-trained 6D pose estimation network.

[0049] It should be noted here that each module in the embodiment of the present invention corresponds one-to-one to each step in the first embodiment, and the specific implementation process is the same, which will not be described in detail here.

[0050] Embodiment 3 This embodiment provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps in the transformer bolt posture detection method as described above are implemented.

[0051] Embodiment 4 This embodiment provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps in the transformer bolt posture detection method as described above.

[0052] Embodiment 5 This embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in the transformer bolt posture detection method as described above are implemented.

[0053] The present invention is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products of the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram and the combination of the processes and / or blocks in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0054] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for detecting the position and posture of transformer bolts, characterized in that: include: Obtain a two-dimensional image of the transformer top surface, extract the transformer top features, and combine with the pre-trained target classification model to identify the type of transformer and the locations of all exposed bolts on the top surface; Based on the transformer top features and the identified transformer type, dynamically match the corresponding transformer 3D model and the distribution pattern of the transformer top bolts from the 3D model library; Determine the blocked area from the two-dimensional image of the transformer top surface, and then predict the position of the blocked bolts on the current transformer top surface in combination with the corresponding transformer 3D model; Based on the positions of all exposed bolts on the current transformer top surface, the predicted positions of the blocked bolts on the current transformer top surface, and the distribution pattern of the bolts on the transformer top, the 3D coordinates of all bolts on the current transformer top surface are determined; Based on the 3D coordinates of all bolts on the current transformer top surface and the pre-trained 6D pose estimation network, the 6D poses of all bolts on the current transformer top surface are determined.

2. The transformer bolt posture detection method according to claim 1, characterized in that: The distribution rules of the transformer top bolts include symmetrical distribution and uniform distribution.

3. The transformer bolt posture detection method according to claim 1, characterized in that: In training the 6D pose estimation network, a 3D physics engine is used to generate simulation data of different 6D poses corresponding to various types of transformer 3D models to construct a training database.

4. The transformer bolt posture detection method according to claim 1, characterized in that: The loss function of the 6D pose estimation network is set to , whose expression is: ； ； ； in, , and All are constant coefficients; is the position loss function; is the angle loss function; The first Confidence level of each bolt position; is the total number of bolts on the top surface of the transformer; is the prediction confidence; is the position vector; is the predicted position vector; is the rotation angle vector; is the predicted rotation angle vector.

5. The transformer bolt posture detection method according to claim 1, characterized in that: The 3D model library includes several transformer 3D models, and each transformer 3D model includes bolt 3D coordinates.

6. The transformer bolt posture detection method according to claim 1, characterized in that: The transformer top features include the heat sink edge position and the connection terminal position.

7. A transformer bolt posture detection device, characterized in that: include: Type and exposed bolt recognition module, which is used to obtain a two-dimensional image of the transformer top surface, extract the transformer top features, and identify the type of transformer and the locations of all exposed bolts on the top surface in combination with a pre-trained target classification model; A dynamic matching module is used to dynamically match the corresponding transformer 3D model and the distribution pattern of the transformer top bolts from the 3D model library based on the transformer top features and the identified transformer type; A blocked bolt prediction module is used to determine the blocked area from the two-dimensional image of the transformer top surface, and then predict the position of the blocked bolt on the current transformer top surface in combination with the corresponding transformer 3D model; A 3D coordinate determination module is used to determine the 3D coordinates of all bolts on the top surface of the current transformer based on the positions of all exposed bolts on the top surface of the current transformer, the predicted positions of the blocked bolts on the top surface of the current transformer, and the distribution pattern of the bolts on the top of the transformer; The 6D pose determination module is used to determine the 6D pose of all bolts on the top surface of the current transformer based on the 3D coordinates of all bolts on the top surface of the current transformer and a pre-trained 6D pose estimation network.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps in the transformer bolt posture detection method as described in any one of claims 1 to 6 are implemented.

9. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the steps in the transformer bolt posture detection method as described in any one of claims 1-6 are implemented.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps in the transformer bolt posture detection method as described in any one of claims 1-6 are implemented.

Images