Switch cabinet knob state detection system based on deep learning
Patent Information
- Application Number
- CN202410966196.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-18
- Publication Date
- 2024-11-15
AI Technical Summary
There is a lack of effective methods in the prior art to detect the state of the knob on the switch cabinet, resulting in misoperation of personnel and safety hazards, and manual inspection is difficult to ensure accuracy and comprehensiveness.
Using a deep learning-based knob status detection system on the switch cabinet, the video acquisition module, the deep learning recognition detection module and the key point recognition module are used to analyze the knob status on the switch cabinet in real time, and use manual annotation and deep learning phases. Combined with methods, accurate detection of knob status is achieved.
Real-time identification and monitoring of the knob status of the switch cabinet is realized, the reliability of the substation operation and the safety of staff are improved, the power accidents caused by misoperation of personnel are reduced, and the reliable operation of the equipment is ensured.
Smart Images

Figure CN118968376A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of sensor design and manufacturing, and in particular to a knob state detection system on a switch cabinet based on deep learning. Background Art
[0002] With the widespread use of power systems, the transmission, conversion and storage of electricity have undergone great changes. With the development of science and technology, various power equipment have been designed and manufactured to meet the demand for power use, and the functions of power equipment have become more and more complete. In substations and distribution stations, switch cabinets are devices used to reduce high voltage to the voltage level required by users through transformers. They are equipped with a comprehensive system that integrates protection, metering and distribution. The knob on the switch cabinet can switch the on and off state of the remote (near) circuit, which is convenient for operation and plays a role in protecting the safety of staff and distribution equipment. After the switch cabinet is put into actual operation, it needs to be used and maintained by professionals. The switch cabinet is an important equipment in the power system. Improper use or misoperation by personnel will lead to serious accidents. and economic losses. With the development of intelligent and automated technology, problems in the use and operation of equipment can be discovered in time through the use of electronic devices in many fields. However, there is still a lack of effective detection methods for monitoring the knobs on switch cabinets. Manpower is still needed to detect the status of the switch cabinet knobs. Personnel detection will inevitably lead to negligence and omissions, and the accuracy is difficult to guarantee. For this reason, the applicant proposes a switch cabinet knob status detection system based on deep learning according to the needs of switch cabinet knob status detection. The system adopts a method combining manual punctuation and deep learning, collects video data, analyzes the target knob status on the switch cabinet in real time, and feeds back the accurate switch status to the staff, so as to improve the reliability of substation operation and the safety of the staff. Summary of the invention
[0003] In order to solve the above technical problems, the present invention proposes a knob state detection system on a switch cabinet based on deep learning, by setting a video acquisition module, a deep learning recognition detection module and a key point recognition module; the video acquisition module includes a fixed camera set up in front of the knob device, a video analysis server, and network cables and equipment connecting the camera and the server; the deep learning recognition detection module uses the Yolov5 network model, including an input end, a Backbone and a loss function; the key point recognition module uses the HRNet network model, and its key components include an entry flow, a stage, multiple cross-scale connections, an exit flow and a loss function; the detection method collects images through the video acquisition module, and the key point recognition module identifies the knob area detected by the deep learning recognition detection module, and judges in combination with the output result of the deep learning recognition detection module to determine the working state of the knob, thereby ensuring the safety of the operators and improving the work efficiency of the operators.
[0004] To achieve the above object, the technical solution adopted by the present invention is:
[0005] A switch cabinet knob state detection system based on deep learning includes a video acquisition module, a deep learning recognition detection module and a key point recognition module, characterized in that: the video acquisition module of the switch cabinet knob state detection method based on deep learning includes a fixed camera set up in front of the knob device, a video analysis server and network cables and equipment connecting the camera and the server; the deep learning recognition detection module uses the Yolov5 network model, including an input end, a Backbone, and a loss function; CSPDarknet53 is used as the backbone network, SPPF is used as the additional module of Neck, CSP-PAN is used as the feature fusion module of the Neck network, and YOLOv3 is used as the head network. The deep learning recognition detection module reads the key frames in the image sequence, and the key frames are selected according to the uniform time. The key frames are scaled to 640*640 one by one, and then the image R Each channel of GB is divided by 255, and the normalized image is passed into the Yolov5 detection model, retaining all confidence results approximately around the set threshold; the loss functions used in the Yolov5 network model include classification loss, target confidence loss and bounding box regression loss; the key point recognition module uses the HRNet network model, and its key components include entry flow, stage, multiple cross-scale connections, exit flow and loss function; the HRNet network model adopts a multi-resolution parallel branch structure, each branch of which has a different resolution but the same depth. During the training process, a multi-scale loss function is used to optimize feature maps of different resolutions; the switch cabinet knob state detection method based on deep learning collects images through a video acquisition module, and the key point recognition module recognizes the knob area detected by the deep learning recognition detection module, and makes a judgment based on the output result of the deep learning recognition detection module to determine the working state of the knob.
[0006] Furthermore, the training steps of the Yolov5 network model in the deep learning recognition and detection module are:
[0007] Step 1: Use a camera to capture images of the knob equipment in the substation, manually mark the external rectangular frame of each knob in the image, and the on and off in the rectangular frame. If the rectangular frame contains an indicator light, the indicator light needs to be marked as off or dimmed according to its status;
[0008] Step 2: Construct a Yolov5 target detection network model;
[0009] Step 3: Input the labeled data in step 1 into the model in step 2 and train the model using stochastic gradient descent until the model converges.
[0010] Furthermore, the training steps of the HRNet network model in the key point recognition module are:
[0011] Step 1: crop the knob area marked in the deep learning recognition module, and mark the cropped image, with the end point being the center point of the knob and the start point being the text indicating whether the knob is pointing to "open" or "close";
[0012] Step 2: Construct a HRNet key point recognition network model.
[0013] Step 3: Input the data annotated on the cropped image into the constructed HRNet key point recognition network model, and train the model using the stochastic gradient descent method until the model converges.
[0014] Furthermore, the classification loss of the loss function in the Yolov5 network model in the deep learning recognition detection module uses a binary cross entropy loss formula to handle multi-label classification problems, allowing each detection box to predict multiple categories. The binary cross entropy loss formula is expressed as:
[0015]
[0016] Where: N is the total number of samples in the data set;
[0017] M is the total number of categories;
[0018] yic is a binary indicator (0 or 1), which is 1 if sample i belongs to category c and 0 otherwise;
[0019] Pic is the probability that the model predicts that sample i belongs to category c.
[0020] Furthermore, the target confidence loss of the loss function in the Yolov5 network model in the deep learning recognition and detection module also needs to consider the IoU between the predicted box and the true box when calculating, so as to adjust the loss value. If the IoU between the predicted box and the true box is high, the model's confidence prediction of the predicted box is more accurate and the loss is smaller; conversely, if the IoU is low, the loss is greater; the formula of the target confidence loss can be expressed as:
[0021]
[0022] Where: Npos is the number of positive samples;
[0023] Nneg is the number of negative samples;
[0024] pi is the confidence prediction of the i-th prediction box;
[0025] λneg is the weight of the negative sample loss, which is used to balance the positive and negative sample losses.
[0026] Furthermore, the bounding box regression loss of the loss function in the Yolov5 network model in the deep learning recognition and detection module adopts CIoU as the loss function of the bounding box regression; the CIoU loss function is an improved IoU loss function, which not only considers the overlapping area of the bounding box, but also considers the size, scale and direction of the bounding box, thereby improving the accuracy of the bounding box prediction; the calculation formula of the CIoU loss function is as follows:
[0027]
[0028] Where: IoU represents the intersection over union ratio between the predicted bounding box and the true bounding box;
[0029] ρcio represents the Euclidean distance from the center point of the predicted bounding box to the center point of the true bounding box;
[0030] c represents the diagonal length of the true bounding box, which is used to normalize the center point distance;
[0031] α and β are weights used to balance the size and scale factors, which are dynamically adjusted according to the size of IoU.
[0032] Furthermore, the key point recognition module uses the HRNet network model to calculate the difference between the predicted heatmap and the true label using MSE Loss; its formula is expressed as:
[0033]
[0034] Where: N is the total number of key points in the heat map;
[0035] Hi is the heat map of the i-th key point in the prediction heat map;
[0036] Gi is the heatmap of the i-th keypoint in the true label heatmap, which is generated by applying a Gaussian kernel at the keypoint location.
[0037] The benefits of this application are:
[0038] 1. The method for detecting the knob status on the switch cabinet based on deep learning uses the deep learning target detection method and the content of the present invention: through the connection and cooperation of the video acquisition module, the deep learning recognition detection module and the key point recognition module, the real-time recognition and monitoring of the knob status of the substation switch cabinet is realized;
[0039] 2. The real-time identification and monitoring function of the knob status of the substation switch cabinet realized by the knob status detection method on the switch cabinet based on deep learning not only ensures the safety of operators in real time and improves their work efficiency, but also solves the power accidents caused by personnel's misoperation, providing a strong guarantee for the reliable operation of substation equipment. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] Figure 1 It is a schematic diagram of the overall component connection and data flow of the present invention;
[0041] Figure 2 This is a schematic diagram of installing the video acquisition module of the present invention;
[0042] Figure 3 It is a grid diagram of the deep learning recognition detection module of the present invention;
[0043] Figure 4 This is a grid diagram of the key point identification module of the present invention. DETAILED DESCRIPTION
[0044] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments:
[0045] like Figure 1As shown, a switch cabinet knob state detection system based on deep learning is shown, including a video acquisition module, a deep learning recognition detection module and a key point recognition module. The video acquisition module of the switch cabinet knob state detection method based on deep learning includes a fixed camera installed in front of the knob device, a video analysis server, and network cables and equipment connecting the camera and the server; the deep learning recognition detection module uses the Yolov5 network model, including an input end, a Backbone, and a loss function; the model supports single GPU training, the input image size is 640*640, CSPDarknet53 is used as the backbone network, SPPF is used as the additional module of Neck, CSP-PAN is used as the feature fusion module of the Neck network, and YOLOv3 is used as the head network. The deep learning recognition detection module reads the key frames in the image sequence, and the key frames are selected according to the uniform time, and the key frames are scaled one by one. to 640*640, and then divide each RGB channel of the image by 255, pass the normalized image into the yolov5 detection model, and retain all confidence results approximately setting the threshold; the loss functions used in the Yolov5 network model shown include classification loss, target confidence loss and bounding box regression loss; the key point recognition module shown uses the HRNet network model, and its key components include entry flow, stage, multiple cross-scale connections, exit flow and loss function; the HRNet network model adopts a multi-resolution parallel branch structure, each branch of which has a different resolution but the same depth. During the training process, a multi-scale loss function is used to optimize feature maps of different resolutions; the switch cabinet knob state detection method based on deep learning collects images through the video acquisition module, and the key point recognition module recognizes the knob area detected by the deep learning recognition detection module, and judges and determines the working state of the knob based on the output result of the deep learning recognition detection module.
[0046] like Figure 2 As shown, the video acquisition module is used to collect knob video data. The processor takes frames of the knob video data at fixed time intervals for analyzing the knob state. Knob data of various categories and angles are obtained by taking frames from the collected video data. Labelme is used to mark the knob area and the indicator lights in the area with rectangular frames, and the key points of the knob are marked.
[0047] like Figure 3 As shown, the deep learning recognition and detection module uses the Yolov5 network model, which mainly includes an input end, a backbone, and a loss function, which is specifically:
[0048] Input: Using the Mosaic data augmentation method, first create a large enough canvas, then randomly generate a point on the canvas and place four puzzle pieces around this point; for each puzzle piece, it is necessary to determine its placement area on the canvas and the image area. If the image exceeds the canvas, it needs to be cropped. Finally, the coordinates of the rectangular box need to be updated so that its position and size relative to the canvas remain unchanged;
[0049] Backbone: Backbone is built based on the following key components: CSPDarknet53 as its backbone network, which uses the Cross Stage Partial Network (CSPNet) structure to reduce the amount of computation and improve feature extraction efficiency; CSPNet reduces the amount of computation by sharing features at different stages of the network; it consists of multiple "parts", each of which contains multiple convolutional layers, but each part is only connected to a small part of the previous part, thereby reducing the number of parameters and computational complexity; SPPF module is used to further extract multi-scale features, which is similar to Spatial Pyramid Pooling (SPP), but adds a Focus mechanism to process feature maps of different sizes; the Focus module concatenates feature maps of different sizes and then processes them through convolutional layers to adapt to feature inputs of different sizes; PANet enhances the representation of features by aggregating features at multiple scales;
[0050] Loss function: The loss function of YOLOv5 mainly includes: classification loss, target confidence loss and bounding box regression loss, specifically:
[0051] Classification loss: The classification loss of the loss function in the Yolov5 network model uses the binary cross entropy loss formula to handle multi-label classification problems, allowing each detection box to predict multiple categories. The binary cross entropy loss formula is expressed as:
[0052]
[0053] Where: N is the total number of samples in the data set;
[0054] M is the total number of categories;
[0055] yic is a binary indicator (0 or 1), which is 1 if sample i belongs to category c and 0 otherwise;
[0056] Pic is the probability that the model predicts that sample i belongs to category c.
[0057] Target confidence loss: The target confidence loss of the loss function in the Yolov5 network model also needs to consider the IoU between the predicted box and the true box when calculating, so as to adjust the loss value. If the IoU between the predicted box and the true box is high, the model's confidence prediction of the predicted box is more accurate and the loss is smaller; conversely, if the IoU is low, the loss is greater; the formula for target confidence loss can be expressed as:
[0058]
[0059] Where: Npos is the number of positive samples;
[0060] Nneg is the number of negative samples;
[0061] pi is the confidence prediction of the i-th prediction box;
[0062] λneg is the weight of the negative sample loss, which is used to balance the positive and negative sample losses.
[0063] Bounding box regression loss: The bounding box regression loss of the loss function in the Yolov5 network model uses CIoU as the loss function for bounding box regression; the CIoU loss function is an improved IoU loss function that not only considers the overlapping area of the bounding box, but also the size, scale, and direction of the bounding box, thereby improving the accuracy of bounding box prediction; the calculation formula of the CIoU loss function is as follows:
[0064]
[0065] Where: IoU represents the intersection over union ratio between the predicted bounding box and the true bounding box;
[0066] ρcio represents the Euclidean distance from the center point of the predicted bounding box to the center point of the true bounding box;
[0067] c represents the diagonal length of the true bounding box, which is used to normalize the center point distance;
[0068] α and β are weights used to balance the size and scale factor, which are dynamically adjusted according to the size of IoU.
[0069] The training steps of the Yolov5 network model in the deep learning recognition and detection module shown are:
[0070] Step 1: Use a camera to capture images of the knob equipment in the substation, manually mark the external rectangular frame of each knob in the image, and the on and off in the rectangular frame. If the rectangular frame contains an indicator light, the indicator light needs to be marked as off or dimmed according to its status;
[0071] Step 2: Construct a Yolov5 target detection network model;
[0072] Step 3: Input the labeled data in step 1 into the model in step 2 and train the model using stochastic gradient descent until the model converges.
[0073] like Figure 4 As shown, the key point recognition module uses the HRNet network model, whose key components include entry flow, stage, multiple cross-scale connections, exit flow and loss function, specifically:
[0074] Inlet flow: The inlet flow is the beginning of the network, responsible for receiving the input image and performing preliminary feature extraction on it. It usually includes some convolutional layers and batch normalization layers to capture the preliminary features of the image.
[0075] Stage: HRNet consists of multiple stages, each of which consists of multiple parallel convolutional layers for learning features at different scales. The convolutional layer of each stage outputs multiple feature maps with the same spatial resolution but may have different depths or numbers of channels;
[0076] Multiple cross-scale connections: In HRNet, feature maps at different stages are fused through cross-scale connections, which allow the network to share information at different scales; cross-scale connections are usually implemented through 1x1 convolution and addition operations, integrating feature maps at different stages to increase the diversity and richness of features. This connection method helps the network utilize multi-scale feature information in subsequent stages;
[0077] Exit Stream: The exit stream is the last part of the network, which is used to generate the final task-specific output, such as classification, detection, or segmentation. The exit stream may include upsampling operations to upsample the high-resolution feature maps to the desired output size; it usually also includes some convolutional layers to further extract features and generate the final output;
[0078] 1. Loss function: HRNet uses MSE Loss to calculate the difference between the predicted heatmap and the true label; its formula is expressed as:
[0079]
[0080] Where: N is the total number of key points in the heat map;
[0081] Hi is the heat map of the i-th key point in the prediction heat map;
[0082] Gi is the heatmap of the i-th keypoint in the true label heatmap, which is generated by applying a Gaussian kernel at the keypoint location.
[0083] The training steps of the HRNet network model in the key point recognition module shown are:
[0084] Step 1: crop the knob area marked in the deep learning recognition module, and mark the cropped image, with the end point being the center point of the knob and the start point being the text indicating whether the knob is pointing to "open" or "close";
[0085] Step 2: Construct a HRNet key point recognition network model.
[0086] Step 3: Input the data annotated on the cropped image into the constructed HRNet key point recognition network model, and train the model using the stochastic gradient descent method until the model converges.
[0087] like Figure 1 As shown, the switch cabinet knob state detection method based on deep learning combines the video acquisition module, the deep learning recognition detection module, the key point recognition module and the subsequent processing, and its process steps are as follows:
[0088] Step 1: The video stream data is transmitted to the video analysis server through the video acquisition module. The server converts the video stream into a continuous image sequence and extracts key frames;
[0089] Step 2: The key frame is passed into the deep learning recognition and detection module, and the detected knob area is used as the input of the key point recognition module;
[0090] Step 3: The key point recognition module recognizes the end point and the starting point of the knob according to the input data;
[0091] Step 4. Connect the end point and the starting point of the knob into a straight line as D, and the center points of the boxes of the text "open" and "close" are F and H respectively; first calculate whether D is approximately parallel to the Y axis of the image. If not, calculate the vertical distances from F and H to the straight line D as FD and HD respectively. Assuming FD>HD, the knob state is closed, otherwise it is open; if D is parallel to the Y axis of the image, it is necessary to judge the state of the indicator light. Assuming that one of the indicator lights is on, it is necessary to calculate the distance from the indicator light to F and D. The text with a smaller distance is the current state of the knob; the real-time recognition and monitoring function of the status of the substation switch cabinet knob is realized, which not only ensures the safety of the operators and improves the work efficiency of the operators in real time, but also solves the power accidents caused by the operators' misoperation, and provides a strong guarantee for the reliable operation of the substation equipment.
[0092] The above description is only a preferred embodiment of the present invention and does not constitute any other form of limitation to the present invention. Any modification or equivalent change made based on the technical essence of the present invention still falls within the scope of protection required by the present invention.
Claims
1. A switch cabinet knob state detection system based on deep learning, comprising a video acquisition module, a deep learning recognition detection module and a key point recognition module, characterized in that: The video acquisition module of the switch cabinet knob state detection method based on deep learning includes a fixed camera set up in front of the knob device, a video analysis server, and network cables and equipment connecting the camera and the server; the deep learning recognition detection module uses the Yolov5 network model, including an input end, a Backbone, and a loss function; using CSPDarknet53 as the backbone network, SPPF as the additional module of Neck, CSP-PAN as the feature fusion module of the Neck network, and YOLOv3 as the head network, the deep learning recognition detection module reads the key frames in the image sequence, the key frames are selected according to the uniform time, the key frames are scaled to 640*640 one by one, and then each channel of the image RGB is divided by 255, and the normalized image is passed to the Yolov5 detection The test model retains all results with confidence levels approximately equal to a set threshold; the loss functions used in the Yolov5 network model include classification loss, target confidence loss, and bounding box regression loss; the key point recognition module uses the HRNet network model, and its key components include entry flow, stage, multiple cross-scale connections, exit flow, and loss function; the HRNet network model adopts a multi-resolution parallel branch structure, each branch of which has a different resolution but the same depth. During the training process, a multi-scale loss function is used to optimize feature maps of different resolutions; the switch cabinet knob state detection method based on deep learning collects images through a video acquisition module, and the key point recognition module recognizes the knob area detected by the deep learning recognition detection module, and makes a judgment based on the output result of the deep learning recognition detection module to determine the working state of the knob.
2. According to claim 1, a switch cabinet knob state detection system based on deep learning is characterized in that: The training steps of the Yolov5 network model in the deep learning recognition and detection module are: Step 1: Use a camera to capture images of the knob equipment in the substation, manually mark the external rectangular frame of each knob in the image, and the on and off in the rectangular frame. If the rectangular frame contains an indicator light, the indicator light needs to be marked as off or dimmed according to its status; Step 2: Construct a Yolov5 target detection network model; Step 3: Input the labeled data in step 1 into the model in step 2 and train the model using stochastic gradient descent until the model converges.
3. According to the deep learning-based switch cabinet knob state detection system of claim 1, it is characterized by: The training steps of the HRNet network model in the key point recognition module are: Step 1: crop the knob area marked in the deep learning recognition module, and mark the cropped image, with the end point being the center point of the knob and the start point being the text indicating whether the knob is pointing to "open" or "close"; Step 2: Construct a HRNet key point recognition network model. Step 3: Input the data annotated for the cropped image into the constructed HRNet key point recognition network model, and train the model using the stochastic gradient descent method until the model converges.
4. According to claim 1, a switch cabinet knob state detection system based on deep learning is characterized in that: The classification loss of the loss function in the Yolov5 network model in the deep learning recognition and detection module uses a binary cross entropy loss formula to handle multi-label classification problems, allowing each detection box to predict multiple categories. The binary cross entropy loss formula is expressed as: Where: N is the total number of samples in the data set; M is the total number of categories; yic is a binary indicator (0 or 1), which is 1 if sample i belongs to category c and 0 otherwise; Pic is the probability that the model predicts that sample i belongs to category c.
5. According to the deep learning-based switch cabinet knob state detection system of claim 1, it is characterized by: The target confidence loss of the loss function in the Yolov5 network model in the deep learning recognition and detection module also needs to consider the IoU between the predicted box and the true box when calculating, so as to adjust the loss value. If the IoU between the predicted box and the true box is high, the model's confidence prediction of the predicted box is more accurate and the loss is smaller; conversely, if the IoU is low, the loss is greater; the formula of the target confidence loss can be expressed as: Where: Npos is the number of positive samples; Nneg is the number of negative samples; pi is the confidence prediction of the i-th prediction box; λneg is the weight of the negative sample loss, which is used to balance the positive and negative sample losses.
6. According to a deep learning-based switch cabinet knob state detection system according to claim 1, it is characterized in that: The bounding box regression loss of the loss function in the Yolov5 network model in the deep learning recognition and detection module adopts CIoU as the loss function of bounding box regression; the CIoU loss function is an improved IoU loss function, which not only considers the overlapping area of the bounding box, but also considers the size, scale and direction of the bounding box, thereby improving the accuracy of bounding box prediction; the calculation formula of the CIoU loss function is as follows: Where: IoU represents the intersection over union ratio between the predicted bounding box and the true bounding box; ρcio represents the Euclidean distance from the center point of the predicted bounding box to the center point of the true bounding box; c represents the diagonal length of the true bounding box, which is used to normalize the center point distance; α and β are weights used to balance the size and scale factors, which are dynamically adjusted according to the size of IoU.
7. The switch cabinet knob state detection system based on deep learning according to claim 1 is characterized in that: The key point recognition module uses the HRNet network model to calculate the difference between the predicted heatmap and the true label using MSE Loss; Its formula is expressed as: Where: N is the total number of key points in the heat map; Hi is the heat map of the i-th key point in the prediction heat map; Gi is the heatmap of the i-th keypoint in the true label heatmap, which is generated by applying a Gaussian kernel at the keypoint location.
Citation Information
Patent Citations
Pointer type instrument reading identification method and system
CN115019294A
Human body posture estimation method based on two-stage weighted mean square loss function
CN115063890A
Small target detection method based on deep learning
CN116612292A
Open scene camouflage personnel small target detection method based on deep learning
CN116824485A
Power switch cabinet knob state identification method and device based on deep learning
CN117315566A
Cited By
Wrench switch identification method based on target detection model
CN119919878A
