A tag-assisted secondary identification device and method for power switch status

Abstract

This invention discloses a tag-assisted secondary identification device and method for power switch status, relating to the field of image processing and recognition. The device includes a tag-assisted module, an image acquisition module, a neural network module, a recognition correction module, and an output module. The method includes the following steps: Step 1, performing tag-assisted operation on the rotary switches on the power switch cabinet panel; Step 2, acquiring the RGB image and depth information of the switching devices on the power switch cabinet panel; Step 3, using the RGB image as input to the neural network model for calculation and recognition, obtaining the recognition information of multiple switching devices contained in the RGB image; Step 4, filtering out the recognition information whose initial category is rotary switch, and determining whether the recognition confidence is lower than a specified threshold. If so, performing secondary recognition correction on the RGB image to obtain the final category and final state of the rotary switch; Step 5, outputting the final recognition result after secondary recognition correction in Step 4.

Description

Technical Field

[0001] This invention relates to the field of image processing and recognition, and in particular to a tag-assisted secondary recognition device and method for power switch status. Background Technology

[0002] Power switchgear is a crucial piece of power distribution equipment, playing a vital role in power generation, transmission, distribution, energy conversion, and consumption through switching, control, and protection. The safe and reliable operation of power switchgear is of paramount importance to the power system. The switchgear panel is a critical interface, typically housing various switching devices such as large and small circuit breakers and rotary switches. Due to the large number and complex types of switching devices in power switchgear, manual inspection is not only labor-intensive and inefficient but also prone to missed or false inspections under the immense workload. Automating switchgear inspection can significantly reduce the daily workload of management personnel. However, existing image recognition technologies still require improvement for the automatic identification of switchgear types and statuses.

[0003] 1. Traditional image processing technology is difficult to detect multiple devices at one time while ensuring accuracy when detecting a large number of switchgear types. It is prone to missed detections or false detections. It is also difficult to detect complex switchgear such as circuit breakers. It cannot meet the requirements of automatic inspection of switchgear with dense layout and variety of switchgear in power switch cabinets.

[0004] 2. The differences between power switchgear are small, and it is difficult to effectively distinguish similar switches using neural network image recognition methods alone, making training difficult;

[0005] 3. Power scenarios have high requirements for the accuracy of identifying the types and states of switching equipment. In actual applications, the number of trainable samples is small. Relying solely on image recognition methods with poor robustness of network models results in low accuracy and cannot meet the application requirements.

[0006] Feng Guojin disclosed a method for identifying indicator lights and switch statuses in a power switchgear in his Chinese invention patent application, "A Method and Device for Identifying Indicator Lights and Switch Statuses in Power Switchgear" (application number: CN202210855218.X). This method uses traditional image processing technology to automatically detect indicators and switches on multiple switchgear units and identifies their statuses using different schemes based on the target category. However, this method cannot distinguish between different types of rotary switches for power switchgear and cannot identify more complex circuit breakers; it can only identify predefined single rotary switches. Therefore, the versatility and applicability of this method need improvement.

[0007] Therefore, those skilled in the art are dedicated to developing a new device and method for automatic identification of the type and state of power switches, in order to solve the aforementioned problems existing in the prior art. Summary of the Invention

[0008] In view of the above-mentioned deficiencies of the prior art, the technical problem to be solved by the present invention is how to accurately and automatically identify and detect the type and status of power switches when faced with a variety of power switches with little difference.

[0009] After the widespread application of neural networks, traditional image recognition technology has significantly improved the computation time and accuracy of image recognition by utilizing neural networks. However, when faced with different power switchgear devices that have little difference in appearance, neural networks trained on small datasets cannot achieve ideal recognition results. Furthermore, traditional image processing techniques struggle to accurately locate and identify switchgear devices with complex structures, and it is also difficult to manually extract features for category differentiation.

[0010] The technical solution provided in this invention adopts a method that combines neural networks and traditional image processing techniques. It uses tags to help increase the differential features of switching devices and adds a secondary recognition and correction module at the back end of the neural network. This can better adapt to different types and status recognition scenarios of switching devices with small differences, thereby solving the problem that it is difficult to automatically identify the type and status of different knob switches due to their small differences.

[0011] To achieve the above objectives, the present invention provides a tag-assisted secondary identification device for power switch status, comprising:

[0012] A label assist module, which performs label assist operation on the rotary switch by affixing an auxiliary label to the rotary switch;

[0013] An image acquisition module, connected to the tag auxiliary module, performs image acquisition operations on the switching device to obtain image signals containing multiple relevant features of the switching device, wherein the switching device includes a non-rotary switch and the rotary switch;

[0014] A neural network module is connected to the image acquisition module. The neural network module takes the image signal as the input of the neural network to obtain the identification information of multiple switching devices contained in the image signal. The identification information includes initial type, state, identification confidence level and location information.

[0015] The identification and correction module is connected to the neural network module. It combines the identification information of multiple switching devices contained in the image signal. When the initial type of a certain switching device is the rotary switch and the identification confidence is lower than a specified threshold, it combines the positioning information and several features and rotation angles of the auxiliary label, and uses image processing technology to perform secondary identification and correction on the image signal to obtain the final type and final state of the rotary switch, and replaces the initial type and state of the rotary switch.

[0016] The output module is connected to the neural network module and the recognition and correction module. It outputs the recognition information of the non-knob switch device identified by the neural network module and the knob switch identified by the recognition and correction module, including the positioning detection frames of multiple switch devices, the final type, and the final state.

[0017] Furthermore, in the label assistance module, the features of the auxiliary label include color features, pattern texture features, shape features, or size features; the color features may include red, yellow, and green; the pattern texture features may include diagonal stripes, grids, and checkerboard patterns; and the shape features may be long rectangles.

[0018] Furthermore, in the image acquisition module, the image acquisition operation refers to using a depth camera to acquire images of the power switch cabinet panel, and the image signal includes RGB images and depth information.

[0019] Furthermore, in the neural network module, the neural network uses the YOLOv5s detection model.

[0020] Furthermore, in the recognition and correction module, the image processing techniques include image segmentation and cropping, binarized image thresholding, pixel statistics, contour detection, and minimum bounding rectangle.

[0021] This invention also provides a tag-assisted secondary identification method for power switch status, the method comprising the following steps:

[0022] Step 1: Perform labeling assistance on the rotary switches on the power switchgear panel by attaching auxiliary labels to the rotary switches;

[0023] Step 2: Use a depth camera to acquire images of the power switch cabinet panel, and obtain RGB images and depth information of the switching equipment on the power switch cabinet panel;

[0024] Step 3: Use the RGB image as input to the trained neural network model to perform calculation and recognition, and obtain the recognition information of multiple switching devices contained in the RGB image. The recognition information includes initial type, state, recognition confidence and location information.

[0025] Step 4: Filter out the identification information of the knob switch as the initial type, and determine whether the identification confidence is lower than a specified threshold. If so, combine the positioning information and several features and rotation angles of the auxiliary label, and use image processing technology to perform secondary identification and correction on the RGB image to obtain the final type and final state of the knob switch, which replaces the initial type and state of the knob switch.

[0026] Step 5: Output the final recognition result after the secondary recognition and correction in step 4.

[0027] Further, step 1 includes the following sub-steps:

[0028] Step 1.1: Determine the types and quantities of the rotary switches on the power switch cabinet panel;

[0029] Step 1.2: Prepare auxiliary labels that are the same number of types as the stated number and include the stated features. Establish a label matching library in which the number of types of rotary switches corresponds one-to-one with the stated features of the auxiliary labels. The stated features include color features, pattern texture features, shape features and size features, and the shape feature is a long rectangle.

[0030] Step 1.3: Affix the auxiliary labels, which include the aforementioned features, to the handle ends of different types of rotary switches. The handle ends will rotate and shift as the state of the rotary switch changes.

[0031] Furthermore, step 2 includes the following sub-steps:

[0032] Step 2.1: Use the program and the depth camera to take pictures and sample the power switch cabinet panel;

[0033] Step 2.2: Generate the RGB image and depth information including various features of the switching device, and place them in the image queue of the program;

[0034] Step 2.3: When the program detects that the current image queue is empty, it returns to step 2.1.

[0035] Furthermore, the neural network model in step 3 is selected as the YOLOv5s detection model, which includes multiple convolutional layers and pooling layers; step 3 includes the following steps:

[0036] Step 3.1: Retrieve the RGB image to be identified and detected and the depth information from the image queue in Step 2.2;

[0037] Step 3.2: Convert the RGB image into the input format of the YOLOv5s detection model and input it into the YOLOv5s detection model for recognition calculation;

[0038] Step 3.3: Generate identification information for multiple switching devices, including the initial type, the state, the identification confidence level, and the location information.

[0039] Furthermore, step 4 includes the following sub-steps:

[0040] Step 4.1: Filter out the identification information whose initial category is the rotary switch and whose identification confidence level is lower than a specified threshold;

[0041] Step 4.2: Based on the positioning information in the recognition result information, segment and crop out a specific region of the image from the RGB image;

[0042] Step 4.3: For the specific region image, use the binarized image thresholding segmentation method to segment the pixel points of the color features of each of the auxiliary tags in the tag matching library, count the number of pixels of each color feature, select the color with the most pixels as the color of the identified auxiliary tag, and then generate the corresponding pixel model.

[0043] Step 4.4: Based on the color of the identified auxiliary label, retrieve the corresponding final type of the rotary switch from the label matching library;

[0044] Step 4.5: Perform contour detection on the pixel model and generate the minimum bounding rectangle;

[0045] Step 4.6: Find the angle θ between the long side of the smallest bounding rectangle and the horizontal line, and determine the final state of the rotary switch based on θ;

[0046] Step 4.7: Replace the initial type and state in the corresponding identification information in Step 4.1 with the final type from Step 4.4 and the final state from Step 4.6.

[0047] The tag-assisted secondary identification device and method for power switch status provided by this invention has at least the following technical advantages:

[0048] 1. The technical solution proposed in this embodiment of the invention uses the YOLOv5s detection model to automatically detect, locate and identify multiple types and states of power switches at one time. During model training and detection, auxiliary labels with bright colors and pattern textures are used to improve the differences between switch types and increase the features of the switching equipment. This can significantly improve the model's accuracy in identifying and locating switch types with small differences.

[0049] 2. Based on the primary recognition result of the YOLOv5s detection model, the recognition correction device and method in the technical solution proposed in this embodiment of the invention divides and cuts the rotary switch in the detection frame, and combines the color, pattern texture and angle of the auxiliary label, and uses image processing technology to perform secondary recognition correction on the rotary switch. This not only makes full use of the powerful recognition and positioning function of the YOLOv5s detection model, but also overcomes the problem of low recognition accuracy caused by the single model due to light, angle and unclear recognition features.

[0050] The following will further explain the concept, specific structure, and technical effects of the present invention in conjunction with the accompanying drawings, so as to fully understand the purpose, features, and effects of the present invention. Attached Figure Description

[0051] Figure 1 This is a schematic diagram of a method flow according to a preferred embodiment of the present invention;

[0052] Figure 2 yes Figure 1 A schematic diagram of the device module structure of the preferred embodiment is shown below;

[0053] Figure 3 yes Figure 1 A schematic diagram of a label-assisted rotary switch in a preferred embodiment is shown. Detailed Implementation

[0054] The following description, with reference to the accompanying drawings, illustrates several preferred embodiments of the present invention to make its technical content clearer and easier to understand. The present invention can be embodied in many different forms, and the scope of protection of the present invention is not limited to the embodiments mentioned herein.

[0055] Traditional image recognition technology, after the widespread application of neural networks, has greatly improved the computation time and recognition accuracy of image recognition. However, when faced with different switching devices with similar characteristics, neural networks trained on small datasets cannot achieve ideal recognition results. Furthermore, traditional image processing techniques struggle to accurately locate and recognize structurally complex switching devices, and manually extract features for category differentiation is also difficult.

[0056] To address the problems existing in the prior art, this invention proposes a tag-assisted secondary identification device and method for power switch status, used to identify the types and statuses of different switchgear with minor differences within a power switch cabinet. This technical solution combines neural networks and traditional image processing techniques, using tags to enhance the differential features of the switchgear, and adding a secondary identification correction module at the back end of the neural network to obtain more accurate identification results.

[0057] like Figure 1 As shown, first, auxiliary labels are affixed to the rotary switches, and a label matching library is established to ensure a one-to-one correspondence between the labels and the switches. A schematic diagram of the rotary switches under the auxiliary labels is shown below. Figure 3 As shown. During recognition, images are acquired using a depth camera and input into a neural network model for initial recognition. The model determines if the type is a rotary switch. If so, it further checks if the recognition confidence level is less than a threshold. If so, the image information is segmented and cropped based on the recognition bounding box information. The type of rotary switch is then identified based on a label matching library. The rotary switch state is determined by the calculated rotary switch angle. Finally, all recognition results after secondary recognition and correction are output, including type, state, and bounding box. Figure 2 for Figure 1 The illustrated embodiment shows a module structure diagram, and the device includes the following modules:

[0058] The labeling assistance module performs labeling assistance on the rotary switch by affixing auxiliary labels to the rotary switch. Specifically, in order to improve the differentiation of rotary switch equipment types and statuses, auxiliary labels with bright colors, pattern textures, long rectangular shapes, and appropriate sizes are affixed to the rotary switch equipment. Colors may include red, yellow, green, etc., and patterns and textures may include diagonal stripes, grids, checkerboard patterns, etc.

[0059] The image acquisition module, connected to the tag auxiliary module, performs image acquisition operations on the switching equipment, obtaining image signals containing relevant features of multiple switching devices, including non-rotary switches and rotary switches. The image acquisition operation refers to using a depth camera to acquire images of the power switchgear panel; the image signals include RGB images and depth information.

[0060] The neural network module, connected to the image acquisition module, uses the image signal as input to obtain identification information for multiple switching devices contained in the image signal. This identification information includes initial type, state, recognition confidence level, and location information. The neural network model is a crucial component of image recognition. The YOLOv5s detection model is selected, using a pre-acquired image dataset containing various switches as input to obtain the initial type, state, recognition confidence level, and location information of multiple switching devices in the image signal. When the differences between switching devices are small, the accuracy of the neural network in recognizing these images may decrease, potentially due to indistinct image features leading to incorrect identification of switch types and states.

[0061] The identification and correction module, connected to the neural network module, aims to accurately identify switches with minimal differences. It combines the initial types, states, identification confidence levels, and location information of multiple switches output by the neural network module. When the initial type of a switch is a rotary switch and the identification confidence level is below a specified threshold, it uses image processing techniques to perform secondary identification and correction on the image signal, incorporating location information, features of auxiliary tags, and rotation angles. This results in the final type and state of the rotary switch, replacing its initial type and state. The image processing techniques include image segmentation and cropping, binarized image thresholding, pixel statistics, contour detection, and minimum bounding rectangle determination.

[0062] The output module connects the neural network module and the recognition and correction module, and outputs the identification information of non-knob switches identified by the neural network module and knob switches identified by the recognition and correction module, including the positioning detection frames of multiple switches, the final type, and the final state.

[0063] The tag-assisted secondary identification method for power switch status proposed in this embodiment of the invention includes the following steps:

[0064] Step 1: Apply auxiliary labels to the rotary switches on the power switchgear panel.

[0065] Step 2: Use a depth camera to capture images of the power switchgear panel, obtaining RGB images and depth information of the switching equipment on the power switchgear panel;

[0066] Step 3: Use the RGB image as input to the trained neural network model to perform calculations and recognition, and obtain the recognition information of multiple switching devices contained in the RGB image. The recognition information includes the initial type, state, recognition confidence and location information.

[0067] Step 4: Filter out the identification information of the initial type as knob switch, and determine whether the identification confidence is lower than the specified threshold. If so, combine the positioning information and several features and rotation angles of the auxiliary label, and use image processing technology to perform secondary identification and correction on the RGB image to obtain the final type and final state of the knob switch, which replaces the initial type and state of the knob switch.

[0068] Step 5: Output the final recognition result after the secondary recognition and correction in Step 4.

[0069] Specifically, step 1 includes the following sub-steps:

[0070] Step 1.1: Determine the types and quantities of rotary switches on the power switchgear panel;

[0071] Step 1.2: Prepare auxiliary labels that match the number of types of rotary switches and include several features. Establish a label matching library that corresponds one-to-one with the number of types of rotary switches and the features of the auxiliary labels. The features include color features, pattern texture features, shape features and size features, and the shape feature is a long rectangle.

[0072] Step 1.3: Attach auxiliary labels with several features to the handle ends of different types of rotary switches. The handle ends will rotate and shift as the state of the rotary switch changes.

[0073] Specifically, step 2 includes the following sub-steps:

[0074] Step 2.1: Use the program and depth camera to take pictures of the power switch cabinet panel;

[0075] Step 2.2: Generate RGB images and depth information including various features of the switching device, and place them in the program's image queue;

[0076] Step 2.3: When the program detects that the current image queue is empty, return to step 2.1.

[0077] Specifically, the neural network model used in step 3 is the YOLOv5s detection model, which includes multiple convolutional and pooling layers; step 3 includes the following steps:

[0078] Step 3.1: Retrieve the RGB image and depth information to be identified and detected from the image queue in Step 2.2;

[0079] Step 3.2: Convert the RGB image to the input format of the YOLOv5s detection model and input it into the YOLOv5s detection model for recognition calculation;

[0080] Step 3.3: Generate identification information for multiple switching devices, including initial type, status, identification confidence level, and location information.

[0081] Specifically, step 4 includes the following sub-steps:

[0082] Step 4.1: Filter out the identification information whose initial category is rotary switch and whose identification confidence level is lower than the specified threshold;

[0083] Step 4.2: Based on the location information in the recognition results, segment and crop out specific regions of the image from the RGB image;

[0084] Step 4.3: For a specific region image, use the binarized image thresholding segmentation method to segment the pixel points of the color features of each auxiliary label in the label matching library, count the number of pixels of each color feature, select the color with the most pixels as the color of the identified auxiliary label, and then generate the corresponding pixel model.

[0085] Step 4.4: Based on the color of the identified auxiliary label, retrieve the final type of the corresponding rotary switch from the label matching library;

[0086] Step 4.5: Perform contour detection on the pixel model and generate the minimum bounding rectangle;

[0087] Step 4.6: Find the angle θ between the longer side of the smallest bounding rectangle and the horizontal line, and determine the final state of the knob switch based on θ.

[0088] Step 4.7: Replace the initial type and state in the corresponding identification information in Step 4.1 with the final type from Step 4.4 and the final state from Step 4.6.

[0089] Specifically, the confidence threshold in step 4.1 is calibrated based on the recognition performance of the neural network model in step 3 and the difference in the knob switch.

[0090] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A tag-assisted secondary identification device for power switch status, characterized in that, include: A label assist module, which performs label assist operation on the rotary switch by affixing an auxiliary label to the rotary switch; An image acquisition module, connected to the tag auxiliary module, performs image acquisition operations on the switching device to obtain image signals containing multiple relevant features of the switching device, wherein the switching device includes a non-rotary switch and the rotary switch; A neural network module is connected to the image acquisition module. The neural network module takes the image signal as the input of the neural network to obtain the identification information of multiple switching devices contained in the image signal. The identification information includes initial type, state, identification confidence level and location information. The identification and correction module is connected to the neural network module. It combines the identification information of multiple switching devices contained in the image signal. When the initial type of a certain switching device is the rotary switch and the identification confidence is lower than a specified threshold, it combines the positioning information and several features and rotation angles of the auxiliary label, and uses image processing technology to perform secondary identification and correction on the image signal to obtain the final type and final state of the rotary switch, and replaces the initial type and state of the rotary switch. The output module is connected to the neural network module and the recognition and correction module. It outputs the recognition information of the non-knob switch device identified by the neural network module and the knob switch identified by the recognition and correction module, including the positioning detection frames of multiple switch devices, the final type, and the final state.

2. The tag-assisted secondary identification device for power switch status as described in claim 1, characterized in that, In the label assistance module, the features of the auxiliary label include color features, pattern texture features, shape features, or size features; the color features may include red, yellow, and green; the pattern texture features may include diagonal stripes, grids, and checkerboard patterns; and the shape features may be long rectangles.

3. The tag-assisted secondary identification device for power switch status as described in claim 1, characterized in that, In the image acquisition module, the image acquisition operation refers to using a depth camera to acquire images of the power switch cabinet panel, and the image signal includes RGB image and depth information.

4. The tag-assisted secondary identification device for power switch status as described in claim 1, characterized in that, In the neural network module, the neural network uses the yolov5s detection model.

5. The tag-assisted secondary identification device for power switch status as described in claim 1, characterized in that, In the recognition and correction module, the image processing techniques include image segmentation and cropping, binarized image thresholding, pixel statistics, contour detection, and minimum bounding rectangle.

6. A tag-assisted secondary identification method for power switch status, characterized in that, The method includes the following steps: Step 1: Perform labeling assistance on the rotary switches on the power switchgear panel by attaching auxiliary labels to the rotary switches; Step 2: Use a depth camera to acquire images of the power switch cabinet panel, and obtain RGB images and depth information of the switching equipment on the power switch cabinet panel; Step 3: Use the RGB image as input to the trained neural network model to perform calculation and recognition, and obtain the recognition information of multiple switching devices contained in the RGB image. The recognition information includes initial type, state, recognition confidence and location information. Step 4: Filter out the identification information of the knob switch as the initial type, and determine whether the identification confidence is lower than a specified threshold. If so, combine the positioning information and several features and rotation angles of the auxiliary label, and use image processing technology to perform secondary identification and correction on the RGB image to obtain the final type and final state of the knob switch, which replaces the initial type and state of the knob switch. Step 5: Output the final recognition result after the secondary recognition and correction in step 4.

7. The tag-assisted secondary identification method for power switch status as described in claim 6, characterized in that, Step 1 includes the following sub-steps: Step 1.1: Determine the types and quantities of the rotary switches on the power switch cabinet panel; Step 1.2: Prepare auxiliary labels that are the same number of types as the stated number and include the stated features. Establish a label matching library in which the number of types of rotary switches corresponds one-to-one with the stated features of the auxiliary labels. The stated features include color features, pattern texture features, shape features and size features, and the shape feature is a long rectangle. Step 1.3: Affix the auxiliary labels, which include the aforementioned features, to the handle ends of different types of rotary switches. The handle ends will rotate and shift as the state of the rotary switch changes.

8. The tag-assisted secondary identification method for power switch status as described in claim 7, characterized in that, Step 2 includes the following sub-steps: Step 2.1: Use the program and the depth camera to take pictures and sample the power switch cabinet panel; Step 2.2: Generate the RGB image and depth information including various features of the switching device, and place them in the image queue of the program; Step 2.3: When the program detects that the current image queue is empty, it returns to step 2.

1.

9. The tag-assisted secondary identification method for power switch status as described in claim 8, characterized in that, The neural network model used in step 3 is the YOLOv5s detection model, which includes multiple convolutional layers and pooling layers; step 3 includes the following steps: Step 3.1: Retrieve the RGB image to be identified and detected and the depth information from the image queue in Step 2.2; Step 3.2: Convert the RGB image into the input format of the YOLOv5s detection model and input it into the YOLOv5s detection model for recognition calculation; Step 3.3: Generate identification information for multiple switching devices, including the initial type, the state, the identification confidence level, and the location information.

10. The tag-assisted secondary identification method for power switch status as described in claim 9, characterized in that, Step 4 includes the following sub-steps: Step 4.1: Filter out the identification information whose initial category is the rotary switch and whose identification confidence level is lower than a specified threshold; Step 4.2: Based on the positioning information in the recognition result information, segment and crop out a specific region of the image from the RGB image; Step 4.3: For the specific region image, use the binarized image thresholding segmentation method to segment the pixel points of the color features of each of the auxiliary tags in the tag matching library, count the number of pixels of each color feature, select the color with the most pixels as the color of the identified auxiliary tag, and then generate the corresponding pixel model. Step 4.4: Based on the color of the identified auxiliary label, retrieve the corresponding final type of the rotary switch from the label matching library; Step 4.5: Perform contour detection on the pixel model and generate the minimum bounding rectangle; Step 4.6: Find the angle θ between the long side of the smallest bounding rectangle and the horizontal line, and determine the final state of the rotary switch based on θ; Step 4.7: Replace the initial type and state in the corresponding identification information in Step 4.1 with the final type from Step 4.4 and the final state from Step 4.6.

Images