Knife switch state detection method based on improved SAM model and three-dimensional digital twinning

Abstract

The invention relates to a disconnecting link state detection method based on an improved SAM model and three-dimensional digital twinning, and the method comprises the steps: carrying out the detection of the state of a disconnecting link through the target distance, the shooting visual angle and position of an inspection picture, and the cooperation and association of the feature points of the disconnecting link in a two-dimensional picture and the feature points in a disconnecting link three-dimensional digital twinning model; and updating the state of the three-dimensional digital twinborn model of the disconnecting link, and outputting the disconnecting link included angle in the three-dimensional digital twinborn model of the disconnecting link to determine the state of the disconnecting link. Compared with the prior art, on the premise that the problem that a two-dimensional included angle cannot reflect the real state of the disconnecting link due to non-uniform visual angles in the prior art can be solved, the three-dimensional included angle is adopted as the basis for detecting the state of the disconnecting link, and the rotating plane of the disconnecting link is determined through the distance ratio; and the calculation speed and accuracy can be greatly improved.

Description

Technical Field

[0001] The present invention relates to the field of disconnector state detection, and in particular to a disconnector state detection method based on an improved SAM model and three-dimensional digital twin. Background Art

[0002] With the development of smart grids, the condition monitoring of power equipment has become increasingly important. Disconnectors are one of the common switching devices in the power system, and their states directly affect the safe operation of the power system. At present, the detection of disconnector states mainly relies on manual inspections or rule-based methods, which have problems such as low efficiency and poor accuracy. In recent years, deep learning technologies have achieved remarkable results in the field of image processing. In particular, image segmentation technology can effectively extract target objects from complex backgrounds. For example, Chinese Patent CN114665608A discloses an intelligent perception inspection system and method for a substation, which relates to the technical field of power system inspection. The system includes: an edge-end data acquisition and analysis subsystem, a station-end management subsystem, and a cloud artificial intelligence subsystem. Among them, the edge-end data acquisition and analysis subsystem is used to collect target inspection data based on inspection tasks and perform edge analysis on the target inspection data based on an artificial intelligence model to obtain inspection analysis results; the station-end management subsystem is used to generate and issue inspection tasks, manage inspection-related data, and generate an original sample set for training and optimizing the artificial intelligence model; the cloud artificial intelligence subsystem is used to train and optimize the artificial intelligence model according to the original sample set and issue the artificial intelligence model. Through the reuse of existing substation equipment and the deployment of intelligent equipment, the intelligent and automated inspection of the substation is realized, the manual operation and maintenance cost is saved, and high-quality operation and maintenance are achieved. Specifically, in the above inspection system and method, it is determined whether two straight lines corresponding to the disconnector arms connected to both ends of the disconnector are included in the disconnector image. If not, it is determined that the state of the disconnector is the open state. If so, the included angle between the two straight lines is calculated, and it is determined whether the included angle is less than a preset angle threshold. If less, it is determined that the state of the disconnector is the closed state. If greater, it is determined that the state of the disconnector is the not-fully-closed state.

[0003] However, existing technologies including the above inspection system and method all have certain limitations. They directly calculate angles based on the collected images, and their accuracy is limited by the shooting position and perspective. To ensure the accuracy of their state detection, it is required that the images of the training samples and the inference samples be taken at similar positions and perspectives. Summary of the Invention

[0004] The purpose of the present invention is to provide a disconnector state detection method based on an improved SAM model and three-dimensional digital twin.

[0005] The purpose of the present invention can be achieved by the following technical solutions:

[0006] A method for detecting the state of a disconnecting switch based on an improved SAM model and three-dimensional digital twin, comprising:

[0007] Step S1: Obtain the inspection photo containing the disconnecting switch, the target distance, and the perspective and position where the inspection photo is taken, and input the inspection photo into the trained first target detection model to obtain the type of the disconnecting switch and the first feature coordinate sequence for characterizing the position of the disconnecting switch;

[0008] Step S2: Judge whether the current inspection photo meets the requirements based on the first feature coordinate sequence;

[0009] Step S3: When the inspection photo meets the requirements, based on the type of the disconnecting switch, find the corresponding three-dimensional digital twin model of the disconnecting switch, the disconnecting switch feature point extraction model, and the disconnecting switch attribute information, where the disconnecting switch attribute information at least includes the standard length of the base;

[0010] Step S4: Crop the image of the disconnecting switch part from the inspection photo based on the first feature coordinate sequence as the target detection image, and input the target detection image into the disconnecting switch feature point extraction model to obtain multiple disconnecting switch feature point coordinates, where the disconnecting switch feature point coordinates at least include the coordinates at both ends of the base, the coordinates of the end points of the disconnecting switch arm, the coordinates of the root of the disconnecting switch arm, and the coordinates of the midpoint of the disconnecting switch arm;

[0011] Step S5: Calculate the pixel length of the base based on the coordinates at both ends of the base, combine the physical length of a single pixel obtained from the target distance to obtain the two-dimensional length of the base, and determine the angle between the perspective where the inspection photo is taken and the axis of the base based on the ratio of the two-dimensional length of the base to the standard length of the base, and further obtain the position of the rotation plane of the three-dimensional digital twin model of the disconnecting switch;

[0012] Step S6: Project the disconnecting switch feature point coordinates along the perspective axis of the inspection photo to the rotation plane of the three-dimensional digital twin model of the disconnecting switch to obtain the projection points of each disconnecting switch feature point, and register the corresponding feature points in the three-dimensional digital twin model of the disconnecting switch and all the projection points to update the state of the three-dimensional digital twin model of the disconnecting switch;

[0013] Step S7: Output the angle between the disconnecting switches based on the updated three-dimensional digital twin model of the disconnecting switch, and obtain the state of the disconnecting switch based on the angle between the disconnecting switches.

[0014] The first feature coordinate sequence is composed of the abscissa and ordinate of the upper left vertex and the lower right vertex of the disconnecting switch detection frame.

[0015] The said step S2 includes:

[0016] Step S2-1: Calculate the pixel distance pixel distance1 between the upper left vertex and the lower right vertex of the disconnecting switch detection frame:

[0017]

[0018] where: x Top Left is the abscissa of the upper left vertex of the disconnecting switch detection frame, x lower right is the abscissa of the lower right vertex of the disconnecting switch detection frame, y Top Left is the ordinate of the upper left vertex of the disconnecting switch detection frame, y Top Left is the ordinate of the lower right vertex of the disconnecting switch detection frame;

[0019] Step S2-2: Determine whether the pixel distance pixel distance1 between the upper left vertex and the lower right vertex of the disconnecting switch detection frame is greater than a pre-configured first pixel threshold distance. If it is, execute Step S2-3; otherwise, determine that the current inspection photo does not meet the requirements;

[0020] Step S2-3: Calculate the distances from the upper left vertex of the disconnecting switch detection frame to the upper boundary and the left boundary of the image respectively, and calculate the distances from the lower right vertex of the disconnecting switch detection frame to the lower boundary and the right boundary of the image respectively. If the distances from the upper left vertex of the disconnecting switch detection frame to the upper boundary and the left boundary of the image, and the distances from the lower right vertex of the disconnecting switch detection frame to the lower boundary and the right boundary of the image are all greater than a pre-configured second pixel threshold distance, then determine that the current inspection photo meets the requirements; otherwise, determine that the current inspection photo does not meet the requirements.

[0021] The disconnecting switch detection frame is rectangular

[0022] The inspection photo is collected by a portable collector, an unmanned aerial vehicle inspection platform or a ground robot inspection platform.

[0023] The first target detection model is the SAM model.

[0024] The step S5 includes:

[0025] Step S5-1: Calculate the pixel length of the base based on the coordinates at both ends of the base, and obtain the two-dimensional length of the base by combining the physical length of a single pixel obtained from the target distance;

[0026] Step S5-2: Determine the angle between the perspective of the inspection photo shooting and the axis of the base based on the ratio of the two-dimensional length of the base to the standard length of the base;

[0027] Step S5-3: Based on the angle between the perspective of the inspection photo shooting and the axis of the base, and combined with the coordinates at both ends of the base, determine the position of the axis of the base, and use the vertical plane passing through the axis of the base as the position of the rotation plane of the three-dimensional digital twin model of the disconnecting switch.

[0028] In the step S7:

[0029] If the angle between the disconnecting switches is between 15° and 96°, the state of the disconnecting switches is abnormal opening;

[0030] If the angle between the disconnecting switches is between 96° and 178°, the state of the disconnecting switches is abnormal closing;

[0031] If the angle between the disconnecting switches is between 0° and 15°, the state of the disconnecting switches is normal opening;

[0032] If the angle between the disconnecting switches is between 178° and 180°, the state of the disconnecting switches is normal closing.

[0033] A charging pile plug-and-charge function testing device includes a memory, a processor, and a program stored in the memory. When the processor executes the program, the above-mentioned method is implemented.

[0034] A storage medium stores a program, and when the program is executed, the above-mentioned method is implemented.

[0035] Compared with the prior art, the present invention has the following beneficial effects:

[0036] 1. On the premise of solving the problem that the two-dimensional angle in the prior art cannot reflect the true state of the disconnecting switches due to inconsistent perspectives, the three-dimensional angle can be used as the basis for detecting the state of the disconnecting switches. And by determining the rotation plane of the disconnecting switches through the distance ratio, the calculation speed and accuracy can be greatly improved.

[0037] 2. By filtering the inspection photos according to the position of the disconnecting switches, the accuracy of subsequent feature point detection can be ensured, and enough feature points can be guaranteed in the target detection image, so as to improve the state of the subsequent three-dimensional digital twin model of the disconnecting switches can be updated.

[0038] 3. After determining the position of the base axis, using the vertical plane passing through the base axis as the position of the rotation plane of the three-dimensional digital twin model of the disconnecting switches can simplify the process of obtaining the rotation plane of the disconnecting switches, improve the speed, and through methods such as ratio calculation, use fewer resources, have a faster speed, reach a higher accuracy, and achieve better results.

[0039] 4. The division of the four states of the disconnecting switches has more information, and by combining 15° and 178° as the boundary values for normal opening and closing respectively, the health of the disconnecting switches can be more truly reflected. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a schematic flow chart of the main steps of the method of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0041] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. This embodiment is implemented on the premise of the technical solution of the present invention, and gives a detailed implementation manner and a specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0042] Embodiment 1

[0043] A disconnector state detection method based on an improved SAM model and 3D digital twin, as Figure 1 shown, includes:

[0044] Step S1: Obtain inspection photos containing disconnectors, the target distance, and the perspective and position where the inspection photos are taken, and input the inspection photos into a trained first target detection model to obtain the disconnector type and the first feature coordinate sequence for characterizing the disconnector position;

[0045] Generally, the inspection photos are collected by a portable collector, an unmanned aerial vehicle inspection platform or a ground robot inspection platform.

[0046] The first target detection model is the SAM model. In its training stage, a large number of inspection photos need to be collected, and the inspection photos are labeled to obtain training samples for training. For this, in order to improve the reduction of target detection accuracy due to weather reasons, it is also necessary to obtain inspection photos under different weather conditions, that is, supplement the collection of some unclear equipment pictures, and collect them under different time periods and different climate conditions. In some embodiments, a data augmentation algorithm is also used to perform data augmentation processing on the collected pictures.

[0047] On the basis of completing the construction of the deep learning neural network model, all data are labeled, converted into the format required for training, and the model is trained and tested.

[0048] In addition, in this embodiment, the disconnector detection frame is rectangular, and the first feature coordinate sequence is composed of the abscissa and ordinate of the upper left vertex and the lower right vertex of the disconnector detection frame.

[0049] Step S2: Judge whether the current inspection photo meets the requirements based on the first feature coordinate sequence, including:

[0050] Step S2-1: Calculate the pixel distance pixel distance1 between the upper left vertex and the lower right vertex of the disconnector detection frame:

[0051]

[0052] where: x Top Left is the abscissa of the upper left vertex of the disconnector detection frame, x lower right is the abscissa of the lower right vertex of the disconnector detection frame, yTop Left is the vertical coordinate of the upper left vertex of the disconnect switch detection frame, y Top Left is the vertical coordinate of the lower right vertex of the disconnect switch detection frame;

[0053] Step S2-2: Determine whether the pixel distance pixel distance1 between the upper left vertex and the lower right vertex of the disconnect switch detection frame is greater than the pre-configured first pixel threshold distance. If so, execute Step S2-3; otherwise, determine that the current inspection photo does not meet the requirements;

[0054] Step S2-3: Calculate the distances from the upper left vertex of the disconnect switch detection frame to the upper boundary and the left boundary of the image respectively, and calculate the distances from the lower right vertex of the disconnect switch detection frame to the lower boundary and the right boundary of the image respectively. If the distances from the upper left vertex of the disconnect switch detection frame to the upper boundary and the left boundary of the image, and the distances from the lower right vertex of the disconnect switch detection frame to the lower boundary and the right boundary of the image are all greater than the pre-configured second pixel threshold distance, then determine that the current inspection photo meets the requirements; otherwise, determine that the current inspection photo does not meet the requirements.

[0055] In this way, filtering the inspection photos based on the position of the disconnect switch can ensure the accuracy of subsequent feature point detection, ensure that there are enough feature points in the target detection image, and improve the state of the subsequent 3D digital twin model of the disconnect switch can be updated.

[0056] Step S3: When the inspection photo meets the requirements, based on the disconnect switch type, find the corresponding 3D digital twin model of the disconnect switch, the disconnect switch feature point extraction model, and the disconnect switch attribute information, where the disconnect switch attribute information includes at least the standard length of the base;

[0057] Generally, the disconnect switch types include single-pole type disconnect switches, double-pole type disconnect switches, triple-pole type, and their respective sub-types.

[0058] Step S4: Crop the image of the disconnect switch part from the inspection photo based on the first feature coordinate sequence as the target detection image, and input the target detection image into the disconnect switch feature point extraction model to obtain multiple disconnect switch feature point coordinates, where the disconnect switch feature point coordinates include at least the coordinates of both ends of the base, the coordinates of the end points of the disconnect switch arm, the coordinates of the root of the disconnect switch arm, and the coordinates of the midpoint of the disconnect switch arm;

[0059] Step S5: Calculate the pixel length of the base based on the coordinates of both ends of the base, combine the physical length of a single pixel obtained from the target distance to obtain the two-dimensional length of the base, and determine the angle between the perspective of the inspection photo and the axis of the base based on the ratio of the two-dimensional length of the base and the standard length of the base, and further obtain the position of the rotation plane of the 3D digital twin model of the disconnect switch, specifically including:

[0060] Step S5-1: Calculate the pixel length of the base based on the coordinates at both ends of the base, and combine the physical length of a single pixel obtained from the target distance to obtain the two-dimensional length of the base;

[0061] Step S5-2: Determine the angle between the perspective of the inspection photo and the axis of the base based on the ratio of the two-dimensional length of the base to the standard length of the base;

[0062] Step S5-3: Based on the angle between the perspective of the inspection photo and the axis of the base, and combine the coordinates at both ends of the base to determine the position of the axis of the base, and use the vertical plane passing through the axis of the base as the position of the rotation plane of the three-dimensional digital twin model of the disconnecting switch.

[0063] Step S6: Project the coordinates of the disconnecting switch feature points along the perspective axis of the inspection photo onto the rotation plane of the three-dimensional digital twin model of the disconnecting switch to obtain the projection points of each disconnecting switch feature point, and register the corresponding feature points and all projection points in the three-dimensional digital twin model of the disconnecting switch to update the state of the three-dimensional digital twin model of the disconnecting switch;

[0064] After determining the position of the axis of the base, using the vertical plane passing through the axis of the base as the position of the rotation plane of the three-dimensional digital twin model of the disconnecting switch can simplify the process of obtaining the rotation plane of the disconnecting switch, improve the speed, through methods such as ratio calculation, use fewer resources, have a faster speed, achieve higher accuracy, and achieve better results.

[0065] Step S7: Output the angle between the disconnecting switches based on the updated three-dimensional digital twin model of the disconnecting switch, and obtain the state of the disconnecting switch based on the angle between the disconnecting switches. Among them, for the state of the disconnecting switch, in this embodiment, if the angle between the disconnecting switches is between 15° and 96°, the state of the disconnecting switch is abnormal opening; if the angle between the disconnecting switches is between 96° and 178°, the state of the disconnecting switch is abnormal closing; if the angle between the disconnecting switches is between 0° and 15°, the state of the disconnecting switch is normal opening; if the angle between the disconnecting switches is between 178° and 180°, the state of the disconnecting switch is normal closing. The division of the four states of the disconnecting switch has more information, and by using 15° and 178° as the boundary values for normal opening and closing respectively, it can more truly reflect the health of the disconnecting switch.

[0066] In this embodiment, in order to reduce the use of hardware resources, the detection model and classification model of the disconnecting switch are combined. The combined first target detection model greatly reduces the use of hardware resources while ensuring that the accuracy rate is not affected.

[0067] In order to simultaneously take into account the detection effects of various types of situations, solutions such as equipment type, model training parameter adjustment, and image enhancement are used in combination, so that the image detection in various situations has further improvement.

[0068] Embodiment 2

[0069] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs.

Claims

1. A switch state detection method based on an improved SAM model and three-dimensional digital twin, characterized in that: include: Step S1: obtaining an inspection photo containing a knife switch, a target distance, and a viewing angle and position of the inspection photo, and inputting the inspection photo into a trained first target detection model to obtain a knife switch type and a first feature coordinate sequence for characterizing the position of the knife switch; Step S2: judging whether the current inspection photo meets the requirements based on the first characteristic coordinate sequence; Step S3: When the inspection photo meets the requirements, based on the type of the knife switch, the corresponding three-dimensional digital twin model of the knife switch, the knife switch feature point extraction model and the knife switch attribute information are searched and obtained, wherein the knife switch attribute information at least includes the standard length of the base; Step S4: based on the first feature coordinate sequence, an image of the knife switch part is cropped from the inspection photo as a target detection image, and the target detection image is input into the knife switch feature point extraction model to obtain a plurality of knife switch feature point coordinates, wherein the knife switch feature point coordinates at least include the coordinates of both ends of the base, the coordinates of the end points of the knife switch arm, the coordinates of the root of the knife switch arm, and the coordinates of the midpoint of the knife switch arm; Step S5: Calculate the pixel length of the base based on the coordinates at both ends of the base, and obtain the two-dimensional length of the base by combining the physical length of a single pixel obtained from the target distance. Determine the viewing angle of the inspection photo and the angle of the base axis based on the ratio of the two-dimensional length of the base to the standard length of the base, and further obtain the position of the rotation plane of the three-dimensional digital twin model of the knife switch. Step S6: Project the coordinates of the knife switch feature points along the viewing angle axis of the inspection photo onto the rotation plane of the knife switch three-dimensional digital twin model to obtain the projection points of each knife switch feature point, and align the corresponding feature points in the knife switch three-dimensional digital twin model with all the projection points to update the state of the knife switch three-dimensional digital twin model; Step S7: output the knife switch angle based on the updated knife switch three-dimensional digital twin model, and obtain the knife switch state based on the knife switch angle.

2. According to claim 1, a switch state detection method based on an improved SAM model and three-dimensional digital twin is characterized in that: The first characteristic coordinate sequence is composed of the horizontal coordinate and the vertical coordinate of the upper left vertex and the lower right vertex of the knife gate detection frame.

3. A switch state detection method based on an improved SAM model and three-dimensional digital twin according to claim 2, characterized in that: The step S2 comprises: Step S2-1: Calculate the pixel distance 1 between the upper left vertex and the lower right vertex of the knife gate detection frame: Where: x TopLeft is the horizontal coordinate of the upper left vertex of the knife switch detection frame, x lowerright is the horizontal coordinate of the lower right vertex of the knife switch detection frame, y TopLeft is the ordinate of the upper left vertex of the knife switch detection frame, y TopLeft is the ordinate of the lower right vertex of the knife switch detection frame; Step S2-2: determine whether the pixel distance pixel distance1 between the upper left vertex and the lower right vertex of the knife switch detection frame is greater than the pre-configured first pixel threshold distance. If yes, execute step S2-3; otherwise, determine that the current inspection photo does not meet the requirements; Step S2-3: Calculate the distance from the upper left vertex of the knife gate detection frame to the upper boundary and the left boundary of the image respectively, and calculate the distance from the lower right vertex of the knife gate detection frame to the lower boundary and the right boundary of the image respectively. If the distance from the upper left vertex of the knife gate detection frame to the upper boundary and the left boundary of the image, and the distance from the lower right vertex of the knife gate detection frame to the lower boundary and the right boundary of the image are all greater than the preconfigured second pixel threshold distance, then it is determined that the current inspection photo meets the requirements; otherwise, it is determined that the current inspection photo does not meet the requirements.

4. A switch state detection method based on an improved SAM model and three-dimensional digital twin according to claim 2, characterized in that: The knife switch detection frame is rectangular.

5. The switch state detection method based on the improved SAM model and three-dimensional digital twin according to claim 1 is characterized in that: The inspection photos are collected by a portable collector, an unmanned aerial vehicle inspection platform or a ground robot inspection platform.

6. The switch state detection method based on improved SAM model and three-dimensional digital twin according to claim 1 is characterized in that: The first target detection model is a SAM model.

7. The switch state detection method based on improved SAM model and three-dimensional digital twin according to claim 1 is characterized in that: The step S5 comprises: Step S5-1: Calculate the pixel length of the base based on the coordinates of both ends of the base, and obtain the two-dimensional length of the base by combining the physical length of a single pixel obtained from the target distance; Step S5-2: determining the angle between the viewing angle of the inspection photo and the axis of the base based on the ratio of the two-dimensional length of the base to the standard length of the base; Step S5-3: Based on the viewing angle of the inspection photo and the angle of the base axis, the position of the base axis is determined in combination with the coordinates at both ends of the base, and the vertical plane passing through the base axis is used as the position of the rotation plane of the three-dimensional digital twin model of the knife switch.

8. The switch state detection method based on improved SAM model and three-dimensional digital twin according to claim 1 is characterized in that: In step S7: If the knife switch angle is between 15° and 96°, the knife switch status is abnormal opening; If the knife switch angle is between 96° and 178°, the knife switch status is abnormal closing; If the knife switch angle is between 0° and 15°, the knife switch status is normal; If the knife switch angle is between 178° and 180°, the knife switch status is normal.

9. A charging pile plug-and-charge function test device, comprising a memory, a processor, and a program stored in the memory, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 8 is implemented.

10. A storage medium having a program stored thereon, characterized in that: When the program is executed, the method according to any one of claims 1 to 8 is implemented.

Images