A power transformation knob equipment state recognition method

Abstract

The application discloses a power transformation knob equipment state recognition method, and relates to the technical field of image processing. The method comprises the following steps: obtaining a first knob image of a power transformation knob equipment; inputting the first knob image into an image feature extraction model to obtain query features output by the image feature extraction model; wherein the image feature extraction model is a model obtained by training a preset neural network model by using sample knob images with preset category labels, and is used for feature extraction of knob images; comparing the query features with image features of each third knob image in a knob image library constructed in advance to determine a candidate image from a comparison result; and determining a power transformation knob equipment state based on first angle information of the first knob image and second angle information of the candidate image, so as to improve the robustness and universality of power transformation knob equipment state recognition.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to a power transformation knob device state recognition method. BACKGROUND

[0002] In the operation and maintenance process of a power transformation station, knob devices are often used to indicate the working state of the device, and the pointer switch angle corresponds to a specific operating state. Therefore, the state of the knob device needs to be checked regularly to ensure stable operation of the power transformation station. Traditional manual inspection has problems such as low efficiency and easy errors; existing image detection and recognition-based solutions focus on overall device detection and lack accurate recognition and state judgment of knob-type small components. In particular, in the case of knob-type structures and small differences in pointer switch angles and background occlusion scenes, a single detection model is difficult to adapt. Therefore, there is an urgent need for a knob device state recognition method with high robustness and strong universality. SUMMARY

[0003] The present application aims to at least partially solve one of the technical problems in the related art. To this end, one object of the present application is to provide a power transformation knob device state recognition method to improve the robustness and universality of power transformation knob device state recognition.

[0004] According to a first aspect of an embodiment of the present application, a power transformation knob device state recognition method is provided, the method comprising:

[0005] obtaining a first knob image of a power transformation knob device;

[0006] inputting the first knob image into an image feature extraction model to obtain a query feature output by the image feature extraction model; wherein the image feature extraction model is a model obtained by training a preset neural network model using sample knob images with preset class labels, and is used for feature extraction of knob images;

[0007] comparing the query feature with image features of each third knob image in a pre-constructed knob image library to determine a candidate image from the comparison result;

[0008] determining a power transformation knob device state based on first angle information of the first knob image and second angle information of the candidate image.

[0009] Optionally, the image feature extraction model is trained in the following manner:

[0010] obtaining a feature map of the sample knob image;

[0011] extracting diversified global features from the feature map;

[0012] concatenating the diversified global features to obtain a combined feature;

[0013] A total loss is determined according to the combination feature, and a model parameter of the neural network model is adjusted based on the total loss.

[0014] Optionally, the diversified global feature includes a first global feature, a second global feature, and a third global feature; wherein,

[0015] The first global feature is obtained based on global average pooling calculation of each channel of the feature map;

[0016] The second global feature is obtained based on maximum pooling calculation on the feature map and summation calculation based on a calculation result;

[0017] The third global feature is obtained based on vector residual coding on the feature map.

[0018] Optionally, the third global feature is obtained based on vector residual coding on the feature map, including:

[0019] A local feature corresponding to the feature map is determined;

[0020] A residual vector of the local feature relative to a clustering center is calculated by setting the clustering center;

[0021] A sum feature is obtained by weighted summation of a membership degree of the clustering center based on the residual vector and the local feature;

[0022] The sum feature is processed to obtain the third global feature.

[0023] Optionally, the total loss is determined based on a ranking loss of a main module and a classification loss of an auxiliary module;

[0024] The main module determines the ranking loss in the following manner:

[0025]

[0026] wherein, represents the ranking loss; represents a batch size; represents an i-th sample knob image, represents a j-th sample knob image, represents a positive sample with the largest distance in the same category as the anchor point within a batch; represents a positive sample with the largest distance in the same category as the anchor point within a batch; represents a combination feature of the i-th sample knob image, represents a combination feature of the j-th sample knob image; represents a k-th sample knob image, represents a positive sample with the largest distance in the same category as the anchor point within a batch; a negative sample with the smallest distance in the heterogeneous sample; denotes a combined feature of the kth sample knob image; denotes a margin hyperparameter; denotes calculating a loss for a case where a boundary condition is not met;

[0027] The auxiliary module determines the classification loss in the following manner:

[0028]

[0029] wherein, denotes the classification loss; B denotes a batch size; C1 denotes a total number of classes; denotes a weight of the cth class; denotes a one-hot encoding of a true label of the ith sample; denotes a cth column of a classifier weight matrix, which corresponds to a linear transformation parameter of the class c, denotes a kth column of the classifier weight matrix; denotes a first global feature of the ith sample knob image.

[0030] Optionally, the knob image library is constructed in the following manner:

[0031] Obtaining second knob images collected on different substations and different device models;

[0032] Detecting a knob image region of the second knob image, and cropping the knob image region to obtain a third knob image;

[0033] Classifying the third knob image, and constructing a knob image library with class labels and angle information based on a classification result.

[0034] Optionally, the first angle information is obtained in the following manner:

[0035] Inputting the first knob image into the trained segmentation model to obtain a pointer segmentation region output by the segmentation model;

[0036] The segmentation model is a model obtained by training a preset second neural network model using a second sample knob image with a preset annotation box, and is used for image segmentation of knob images;

[0037] Determining the first angle information based on the pointer segmentation region.

[0038] Optionally, determining the first angle information based on the pointer segmentation region comprises:

[0039] acquire the maximum connected region of each category in the pointer segmentation region;

[0040] determine the first angle information based on the centroid of each maximum connected region.

[0041] According to a second aspect of the embodiment of the present application, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program.

[0042] According to a third aspect of the embodiment of the present application, an electronic device is provided, and the electronic device comprises a memory and a processor.

[0043] The scheme provided by the embodiment of the present application improves the robustness and generalization ability of the knob type recognition by constructing a knob image library and introducing an image feature extraction model, and is suitable for diversified knob structures.

[0044] Additional aspects and advantages of the present application will be made apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0045] Figure 1 is a flowchart of a first substation knob device state recognition method provided by the embodiment of the present application;

[0046] Figure 2 is a flowchart of model training provided by the embodiment of the present application;

[0047] Figure 3 is a scene diagram of image segmentation provided by the embodiment of the present application;

[0048] Figure 4 is a flowchart of a second substation knob device state recognition method provided by the embodiment of the present application;

[0049] Figure 5 is a structural diagram of an electronic device provided by the embodiment of the present application. DETAILED DESCRIPTION

[0050] Embodiments of the present application are described below in detail with reference to the accompanying drawings, wherein the same or similar components or components having the same or similar functions are denoted by the same or similar reference numerals throughout. The embodiments described below by reference to the accompanying drawings are exemplary and are intended to explain the present application, and should not be understood as limiting the present application.

[0051] A power transformation knob device state recognition method, a storage medium and an electronic device are described below with reference to the accompanying drawings.

[0052] In one embodiment of the present application, referring to Figure 1 , a power transformation knob device state recognition method is provided, which comprises the following steps S101-S104.

[0053] S101: Obtain a first knob image of a power transformation knob device.

[0054] In embodiments of the present application, knob images of different types of power transformation knob devices can be collected on different substations and different device models as the first knob image. In one specific embodiment, the collected knob image can also be input into a target detection model to obtain an image of the knob region output by the target detection model as the first knob image. The target detection model can be a YOLO model, a FASTER RCNN model, etc. in the prior art, which will not be described in detail here.

[0055] S102: Input the first knob image into an image feature extraction model to obtain query features output by the image feature extraction model.

[0056] The image feature extraction model is a model obtained by training a preset neural network model using sample knob images with preset class labels, and is used for feature extraction of knob images.

[0057] During training, a batch of sample knob images , wherein, represents the batch size (Batch Size); represents the height and width of the original image; represents the number of RGB channels. R is a collection of a batch of sample knob images.

[0058] In one embodiment of the present application, the above neural network model is trained in the following manner to enable it to output query features.

[0059] Obtain a feature map of the sample knob image; extract diversified global features from the feature map; splice the diversified global features to obtain combined features; determine the total loss according to the combined features, and adjust the model parameters of the neural network model based on the total loss.

[0060] In one embodiment, the neural network model to be trained comprises a pre-trained ResNeXt-50 backbone network. The sample knob image is input into the backbone network, the last fully connected classification layer in the backbone network is removed, and a feature map F at the end of Stage4 in the backbone network is output.

[0061]

[0062] wherein Backbone() represents the output of the backbone network, X is the sample knob image, R is a set of a batch of sample knob images, represents the batch size, represents the number of output channels at the end of Stage4. , represents the height and width of the original image. In the formulas of all embodiments herein, formula parameters such as B, C, H, etc. have the same meaning, and are not repeated hereinafter.

[0063] Global features refer to global descriptors, that is, features extracted from the feature map of the entire image. Diversified global features refer to extracting multiple types of global features, so that the feature representation capability is stronger, and the output combined features of the trained model are more accurate.

[0064] The specific method of adjusting the model parameters based on the total loss can be to update the model parameters in the way of back propagation and gradient descent, to reach a predetermined number of iterations or convergence of the loss function of the total loss, which makes the trained model can receive the first knob image as input, and can output the first knob image as query feature in the way of obtaining combined features.

[0065] The types of global features are described in subsequent embodiments, which are not described here.

[0066] The first knob image is normalized and size-adjusted and then input into the trained feature extraction model.

[0067] S103: Comparing the query feature with the image features of each third knob image in the pre-constructed knob image library, and determining a candidate image from the comparison result.

[0068] The image features of each third knob image can be obtained by principal component analysis, histogram of oriented gradients, or other feature extraction algorithms, or can be obtained by the image feature extraction model in step S102, and the embodiments of the present application do not limit this.

[0069] In the case of obtaining the image features, the comparison manner can be to calculate the cosine distance between the feature vector corresponding to the image features and the query feature vector corresponding to the query features, to obtain a cosine distance matrix of all features in the knob image library:

[0070]

[0071] wherein, denotes the cosine distance, denotes the query feature vector, denotes the image features of the i th third knob image in the knob feature library. In the embodiments of the present application, "first", "second", "third" and the like are used to distinguish from the knob images in step S101, and do not represent a limitation in order or category.

[0072] In the comparison result, the Top-K index is returned according to the cosine distance sorting, for example, in the case of K=1, a third knob image with the closest cosine distance can be obtained as a candidate image. In other embodiments, K can also be an integer greater than 1.

[0073] S104: determining the state of the electrical knob device based on the first angle information of the first knob image and the second angle information of the candidate image.

[0074] When constructing the knob image library, the second angle information of the third knob image can be pre-stored, which includes the angle of rotation in the third knob image and the scale corresponding to different angles, and the state label of different states represented by different angles, as shown in Table 1.

[0075] Table 1

[0076]

[0077] In Table 1, id and xuanniu_name respectively represent the number and name of the third knob image, num_scale represents the total number of states of the electrical knob device, and xuanniu1 represents the name, num_scale=2 represents the existence of two states of the electrical knob device, and xuanniu_scale is the angle corresponding to different states of the electrical knob device. Each angle corresponds to one state of the electrical knob device, and different states of the electrical knob device are marked with different state labels L. |-90|0| represents two angles of -90 degrees and 0 degrees corresponding to the state of the electrical knob device, so num_scale=2. In the second row, |-90|90|45| represents three states of the electrical knob device, and accordingly, num_scale=3.

[0078] The first angle information is an angle of the knob in the first knob image. The second angle information can be used to select an angle of a third knob image from the candidate images that is closest to the angle of the knob in the knob image, so that the angle corresponding to the angle of the third knob image determined is obtained as the state of the electrical rotary knob device of the first knob image. For example, if the first angle information is -5 degrees, the closest state is the state of the electrical rotary knob device represented by the angle 0 in the id=0 row.

[0079] The scheme provided by the embodiment of the present application improves the robustness and generalization ability of the knob type recognition by constructing a knob image library and introducing an image feature extraction model, and is suitable for diversified knob structures. On the basis of the feature extraction of the model, the knob image library is searched and compared, so that the angle information in the knob image can be accurately determined, instead of only relying on the model to extract the features, so that better robustness and universality can be provided.

[0080] The following embodiment describes the types of global features in step S102.

[0081] In one embodiment of the present application, the diversified global features include: a first global feature, a second global feature and a third global feature; wherein,

[0082] The first global feature is obtained based on global average pooling calculation of each channel of the feature map;

[0083] The second global feature is obtained based on summation calculation of the calculation result of the maximum pooling calculation on the feature map;

[0084] The third global feature is obtained based on vector residual coding of the feature map.

[0085] Specifically, the global average pooling calculation of the first global feature is performed in the following manner:

[0086]

[0087] wherein, represents the first global feature of the bth sample knob image, represents the height and width of the feature map of the bth sample knob image of the feature map, represents the pixel value of the i-th row and j-th column in the bth sample knob image, R represents a set of sample knob images in a batch, and C represents the number of output channels.

[0088] For the second global feature, first generate multi-scale regions on the feature map F of each sample knob image. Specifically, set the number of levels L = 3, and the multi-scale regions include square regions generated by sliding windows with 3 different sizes on the feature map F as RoI (Region of Interest) regions of interest.

[0089] For each region of interest, use the RoIAlign method to obtain the region features of the region of interest

[0090]

[0091] wherein, B represents the batch size; C represents the output channel number; is the fixed spatial size after alignment, which can be set to 7*7.

[0092] The above maximum pooling calculation process includes maximum pooling on , and the pooling result is :

[0093]

[0094] wherein, MaxPool() represents the maximum pooling, and the remaining parameters have the same meaning as the parameters represented by the same letters in the foregoing formula; L2 normalization is performed on each to obtain . Finally, sum all RoIs to obtain:

[0095]

[0096] wherein, represents the second global feature; the remaining parameters have the same meaning as the parameters represented by the same letters in the foregoing formula.

[0097] For the third global feature, the process of vector residual coding includes: determining the local feature corresponding to the feature map;

[0098] By setting the cluster center, the residual vector of the local feature relative to the cluster center is calculated;

[0099] Based on the residual vector and the membership degree of the local feature to the cluster center, the sum feature is obtained by weighted summation.

[0100] The sum feature is processed to obtain the third global feature.

[0101] Specifically, first expand the channel of the feature map F, and expand the feature map with H*W spatial dimensions into a feature vector with a dimension length of N ​​​, which is equivalent to converting the features of two-dimensional space into one-dimensional space features of the same representation, has:

[0102]

[0103] wherein, can be decomposed into dimensional local features of each sample knob image.

[0104] is mapped to the cluster center by convolution:

[0105]

[0106] wherein S represents the similarity score corresponding to each cluster center, and K is the number of cluster centers represents the convolution mapping.

[0107] The 1x1 convolution can change the number of channels without changing the spatial size (including height and width) of the feature map. In the clustering task, the input feature map may have multiple channels, each containing different aspects of feature information. Through 1x1 convolution, these multi-channel features can be linearly combined to integrate the information of different channels, making the features more compact and representative.

[0108] For example, the input feature map is 28x28x192, which can be mapped to K (the number of cluster centers) channels by 1x1 convolution to get 28x28xK output. In this way, the K values of each spatial position can be regarded as the association features of the position and the K cluster centers.

[0109] The membership matrix is obtained by Softmax:

[0110]

[0111] wherein, represents the membership of the nth local feature in the bth sample knob image to the kth cluster center; represents the similarity score of the nth local feature in the bth sample knob image to the kth cluster center, which is obtained from S in the foregoing embodiments.

[0112] Let the learnable cluster center be , and calculate the residual error:

[0113]

[0114] wherein, ​​is the residual vector of the nth local feature in the bth sample knob image relative to the kth cluster center, denotes the nth local feature in the bth sample knob image, denotes the feature of the kth cluster center.

[0115] The cluster centers can be initially determined by random initialization, and can be determined by re-computing the mean of all points in each cluster and taking these means as new cluster centers for a certain number of iterations.

[0116] The residual vectors are weighted and summed in the spatial dimension to obtain the sum feature:

[0117]

[0118] wherein, is the sum feature of the bth sample knob image relative to the kth cluster center. In all , the K C-dimensional vectors are concatenated to obtain a long vector V with a length of KxC:

[0119]

[0120] The long vector V is subjected to PCA dimension reduction and L2 normalization to output the third global feature. The PCA projection matrix is set to . The third global feature is obtained by the above method, wherein the K clustering method is used to establish a visual semantic dictionary, to provide semantic anchor points for local features, and to provide a weighting basis for the calculation of residual vectors by calculating the membership matrix, to improve the discriminability of semantic expression.

[0121] According to the above embodiment, the diversified global features are spliced to obtain a combined feature, that is, the feature vectors of the first, second and third global features are uniformly L2 normalized and spliced to obtain a combined descriptor :

[0122]

[0123] wherein, denotes the L2 normalized first global feature, denotes the L2 normalized second global feature, denotes the L2 normalized third global feature.

[0124] In an embodiment of the present application, the total loss is determined based on the ranking loss of the main module and the classification loss of the auxiliary module;

[0125] The main module uses the spliced combined descriptor feature to calculate the ranking loss and learn the relative distance between samples. Here, a hard sample triplet loss is used:

[0126]

[0127] wherein, represents the ranking loss; represents the batch size; represents the i-th sample knob image, represents the j-th sample knob image, represents the positive sample with the largest distance in the batch that is of the same class as the i-th sample knob image; represents the negative sample with the smallest distance in the batch that is of a different class as the i-th sample knob image; represents the combined feature of the i-th sample knob image, represents the combined feature of the j-th sample knob image; represents the k-th sample knob image, represents the negative sample with the smallest distance in the batch that is of a different class as the i-th sample knob image; represents the combined feature of the k-th sample knob image; represents the margin hyperparameter; represents , the loss is calculated for the case where the boundary condition is not met.

[0128] The margin hyperparameter has a preset value, for example, it can take 0.2~0.5.

[0129] The auxiliary module determines the classification loss in the following manner, specifically taking the output of the first global feature as input, and calculating the weighted cross-entropy loss:

[0130]

[0131] wherein, represents the classification loss; B represents the batch size; C1 represents the total number of classes, and each class corresponds to a class label; represents the weight of the c-th class; represents the one-hot encoding of the true label of the i-th sample, and the true label can be the true class of the pre-labeled sample knob image; represents the c-th column of the classifier weight matrix, which corresponds to the linear transformation parameter of class c, represents the k-th column of the classifier weight matrix; represents the first global feature of the i-th sample knob image. The classifier weight matrix is initialized as a C1×d parameter matrix by the function Linear(d, C1). Then in the training process, the matrix is updated by backpropagation and gradient descent.

[0132] The calculation of the above classification loss is an auxiliary training strategy, which functions to:​

[0133] 1) The guided model extracts class-discriminative representations;

[0134] 2) A more stable supervisory signal is applied to the backbone network;

[0135] 3) The main task, i.e., the ranking loss, does not rely too much on hard samples, which can cause feature collapse or unstable training.

[0136] The first global feature is selected as the input because among the three global features, the first global feature has the simplest structure, the most stable information, and the least position-sensitive feature representation, which is suitable as the input of the auxiliary classifier in the auxiliary module to guide the main module to learn discriminative semantic representations.

[0137] From a general perspective, the auxiliary module can improve the ability of the backbone network to extract semantic representations through a lightweight but strongly supervised classification auxiliary task, thereby enhancing the effect of the main task of matching retrieval.

[0138] The total loss is the weighted sum of the ranking loss and the classification loss:

[0139]

[0140] wherein is a balance factor, for example, which can be 0.5.

[0141] Figure 2 An embodiment of model training is described in the foregoing.

[0142] wherein, first, the candidate frame, the positive sample and the negative sample are input into the backbone network, and in the sample knob image, the candidate frame is a labeled frame in the image specified in advance, and the positive sample indicates that the input is a real and correct sample knob image of the target class in the current task.

[0143] The negative sample indicates that the input is not a sample of the current target class, but a sample of a different class or an incorrect sample knob image.

[0144] Here, the positive sample is consistent with the type of the candidate frame knob, and the negative sample is inconsistent. Either the positive sample or the negative sample can be input as one or more.

[0145] The first global feature is generated in the global average pooling module, the second global feature is generated in the multi-scale region maximum pooling module, and the third global feature is generated in the local feature vector aggregation module; wherein, the first global feature participates in generating the weighted cross-entropy loss of the auxiliary module, the three global features are spliced to generate the hard sample triplet loss, and the model parameters are adjusted based on the weighted sum of the two losses. Through the extraction of diversified global features, the trained model improves the robustness and generalization ability of knob type recognition, and adapts to diversified knob structures.

[0146] In one embodiment, the knob image library is constructed in the following manner:

[0147] Second knob images collected on different substations and different device models are obtained.

[0148] The knob image region of the second knob image is detected, and the third knob image is obtained by cropping the knob image region.

[0149] The third knob image is classified, and a knob image library with class labels and angle information is constructed based on the classification results.

[0150] The knob image region can be detected in a manner similar to the aforementioned step S101, and a target detection model or the like can be used to achieve this. The third knob image is an image formed by only the knob image region.

[0151] The angle information can be manually labeled, and the corresponding substation knob device state of different angles is indicated. See the embodiment of Table 1 described above.

[0152] In one embodiment of the present application, the first angle information is obtained in the following manner:

[0153] The first knob image is input into the trained segmentation model to obtain the pointer segmentation region output by the segmentation model.

[0154] The segmentation model is a model obtained by training a preset second neural network model using a second sample knob image with a preset annotation box, and is used for image segmentation of the knob image.

[0155] The first angle information is determined based on the pointer segmentation region.

[0156] The second sample knob image is a manually labeled knob image. When labeling, the pointer mask coordinates of the second sample knob image are converted into normalized polygon vertex coordinates as labels, thereby generating an annotation mask region. The image is input into the second neural network model, which is an improved YOLO11-seg model, for model training.

[0157] The main structure of YOLO11-seg remains unchanged, consisting of a Backbone, Neck, and Head. The second sample knob image is preprocessed and fed into the Backbone to extract multi-scale features. These features are then fused using the Neck's feature pyramid network and path aggregation network to obtain fused features. Finally, the Head detects and segments branches based on these fused features. During training, the model parameters of the second network are adjusted based on the segmentation loss, similar to the adjustment method used for the aforementioned training image feature extraction model.

[0158] In the path aggregation network, multi-scale features are upsampled in the following manner, and feature fusion is performed based on the upsampled features:

[0159] Because the upsampling process of nearest neighbor interpolation results in significant information loss and unclear recovery of small, localized areas, the standard upsampling in the YOLO11-seg decoder is replaced with learnable upsampling. The low-resolution feature map of the second sample knob image is input into Conv2d, and the number of output channels is set to... Using PixelShuffle(r) The image is restored to a C-channel image at r times resolution, and feature fusion is performed based on the image features of the r-times resolution image.

[0160] In training the segmentation head, Focal Tversky Loss is used as the main segmentation loss, with background suppression loss added. The overall loss function is defined as:

[0161]

[0162] in For Focal Tversky Loss, Background suppression loss; , These are the preset weight coefficients. Focal Tversky Loss is the loss calculated based on the difference between the segmented region output by the second neural model and the manually labeled segmented region; background suppression loss is the loss calculated based on the difference between the background region outside the segmented region output by the second neural model and the background region outside the manually labeled segmented region.

[0163] In one embodiment of the present invention, the original first knob image may be preprocessed before inputting the segmentation model, including:

[0164] Calculate the scaling ratio and the target size of the segmentation model input ( ):

[0165]

[0166] Compute the lateral offset of the padding offset and the longitudinal offset :

[0167]

[0168] Then, the equal scaling and the grayscale padding are performed

[0169]

[0170] wherein , represents the pre-processed first knob image, and ratio represents the scaling ratio, represents the equal scaling. () represents the grayscale padding function, and the parameter in the function represents the range of the grayscale padding. represents the original first knob image. , represents the height and the width of the original image.

[0171] The segmentation region output by the second neural network model is obtained in the following manner.

[0172] The pre-processed first knob image is input into the second neural network model, wherein the model is processed by a forward function f:

[0173]

[0174] wherein the candidate bounding box ; the confidence score ; the class index , wherein the class index corresponds to the class of the maximum probability after Softmax classification of each candidate bounding box; and the mask region .

[0175] The output of the forward function is processed by an NMS operator, wherein the NMS (Non-Maximum Suppression) operator is defined as , represents the confidence threshold, so that from all the candidate bounding boxes output by the model, the candidate bounding boxes with a confidence score lower than the confidence threshold are first filtered out. is the intersection-over-union threshold, for each class, the candidate bounding box with the highest confidence score is selected as the reserved box each time, and then other boxes with an IoU greater than with the reserved box are suppressed.

[0176] After screening by the NMS operator, the following is obtained:

[0177]

[0178] in, This represents the candidate bounding box after filtering. S, L, and M are the confidence score, category index, and mask region corresponding to B.

[0179] Restoring the original image scale and cropping the mask area specifically includes:

[0180] Set fill vector For each mask region Process according to the following formula:

[0181]

[0182] in, It is the candidate bounding box corresponding to the i-th mask region and whose size has been restored; These are candidate bounding boxes before the dimensions were restored. This is the ratio calculated in the previous embodiments.

[0183] Grouping by category and selecting the optimal segmentation region, predefined category set. ,for definition

[0184]

[0185] like ,Pick and output .

[0186] This represents the category index of the i-th candidate bounding box. Equivalent to choosing The highest confidence score in each category. The corresponding candidate bounding boxes are used as Therefore, it is. yes The mask area, It is the highest confidence score.

[0187] The final result is for each category triples .in, This is the segmented region output by the second neural network model. See also Figure 3 A, B, and C correspond to three different states of the substation rotary knob device. In this embodiment, knob pointer 3 points to state C. Area 1 is... Area 2 is .

[0188] In one embodiment of the present invention, determining the first angle information based on the pointer segmentation region includes:

[0189] Obtain the maximum connected region for each category within the pointer-segmented region;

[0190] The first angle information is determined based on the centroid of each largest connected region.

[0191] Specifically, in the pointer-segmented region, for Perform binarization processing, The pixel values ​​are normalized to the [0, 1] interval, and binarized with a threshold of 0.5. The connected components of each category are extracted from the binarization results and sorted in descending order of area.

[0192] After sorting by area in descending order, the connected region with the largest area in each category is selected as the largest connected region.

[0193] Maximum connected region for each category Calculate the centroid:

[0194]

[0195] in It is the number of pixels in the region. It is the centroid of the largest connected region of the i categories.

[0196] Let the centroid of the first region be... The second center of mass is The angle between the direction of the connecting line and the vertical direction is .

[0197]

[0198]

[0199] Calculated This means that after matching the first angle information with the second angle information, the pre-stored label L representing the substation knob device status is output, corresponding to the angle and scale.

[0200] In terms of state recognition, the knob pointer area is accurately segmented by training a segmentation model, combined with OpenCV's closing operation for noise reduction, and the pointer direction angle is obtained by contour fitting and centroid calculation. The knob state is then judged based on the preset angle threshold and scale information, thereby achieving high-precision state recognition even in complex backgrounds and with slight angle differences.

[0201] The following is passed Figure 4 The illustrated embodiment describes the overall process of the knob device status recognition method provided by the present invention.

[0202] The collected knob image is input into a first-level equipment switch detector to detect the knob region and perform image cropping on the original region.

[0203] The cropped knob image is classified to construct a knob image library with a category label and a scale range. The scale range is a scale corresponding to different angles of the second angle information.

[0204] An image feature extraction model is constructed and trained.

[0205] The knob image is input into the image feature extraction model, and the distance between the knob image and a third image in the knob image library is calculated to obtain category retrieval and scale range. The category retrieval is represented by the state label L, and the scale range is represented by the second angle information.

[0206] A segmentation model is constructed and trained.

[0207] The knob image is input into the segmentation model to obtain a mask region of the pointer segmentation.

[0208] The mask region is subjected to morphological denoising and binarization processing, the maximum connected region contour is extracted, the region centroid is calculated and the angle is calculated, and the state of the knob is matched and output, that is, the state of the power transformation knob equipment is determined.

[0209] In an embodiment of the present application, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program. When the computer program is executed by a processor, the power transformation knob equipment state recognition method of any one of the above embodiments is implemented.

[0210] In an embodiment of the present application, an electronic device is provided, which includes a memory and a processor. The memory stores a computer program, and when the computer program is executed by the processor, the power transformation knob equipment state recognition method of any one of the above embodiments is implemented.

[0211] Figure 5 is a structural block diagram of the electronic device of the embodiment of the present application.

[0212] As shown in Figure 5 , the electronic device 500 includes a processor 501 and a memory 503. The processor 501 and the memory 503 are connected, such as through a bus 502. Optionally, the electronic device 500 can also include a transceiver 504. It should be noted that in actual applications, the transceiver 504 is not limited to one, and the structure of the electronic device 500 does not constitute a limitation on the embodiments of the present application.

[0213] The processor 501 can be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic device, transistor logic device, hardware component, or any combination thereof. It can implement or execute various exemplary logical blocks, modules and circuits described in conjunction with the present disclosure. The processor 501 can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0214] The bus 502 can include a path for transmitting information between the above-mentioned components. The bus 502 can be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus 502 can be divided into an address bus, a data bus, a control bus, etc. For convenience of representation, Figure 5 Only one thick line is used in the figure, but it does not mean that there is only one bus or only one type of bus.

[0215] The memory 503 is used to store a computer program corresponding to the power transformation knob device state recognition method of the above-mentioned embodiments of the present application, which is controlled and executed by the processor 501. The processor 501 is used to execute the computer program stored in the memory 503 to realize the content shown in the above-mentioned method embodiments.

[0216] The electronic device 500 includes, but is not limited to, a mobile terminal such as a mobile phone, a notebook computer, a digital broadcast receiver, a PDA (Personal Digital Assistant), a PAD (Tablet Personal Computer), a PMP (Portable Multimedia Player), a car terminal (such as a car navigation terminal), and the like, and a fixed terminal such as a digital TV, a desktop computer, and the like. Figure 5 The electronic device 500 shown is only an example, and should not bring any limitation to the functions and use range of the embodiments of the present application.

[0217] It is to be appreciated that the above description and the examples that follow are intended to be illustrative only and that changes can be made to the description, as represented by the above listed elements, by the steps recited in the flow charts, and by the examples that follow, without departing from the spirit of the application. Accordingly, the scope of the present application is intended to be defined only by the appended claims.

[0218] It should be understood that aspects of the application can be implemented in hardware, software, firmware or a combination thereof. In the above embodiments, various steps or methods can be implemented in software or firmware that is stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any of the following technologies, known in the art, can be used: a hybrid of the above technologies, a combination of any of the above technologies, etc.

[0219] In the description of the present application, the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" are intended to mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present application. The illustrative appearances of the above-mentioned terms in various places in the specification are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0220] In the description of the application, it should be understood that the orientation or positional relationship indicated by terms such as "center", "longitudinal", "transverse", "length", "width", "thickness", "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", "axial", "radial", "circumferential" and the like is based on the orientation or positional relationship shown in the drawings, and is only for the purpose of facilitating the description of the application and simplifying the description, and therefore cannot be understood as indicating or implying that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the application.

[0221] In addition, the terms "first", "second" are only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the technical features indicated. Therefore, the features defined with "first", "second" can explicitly or implicitly include at least one of the features. In the description of the application, the meaning of "a plurality of" is at least two, such as two, three, etc., unless otherwise explicitly specified and limited.

[0222] In the present application, unless otherwise explicitly specified and limited, the terms "mounting", "connecting", "connecting", "fixing" and the like should be understood broadly, for example, it can be fixedly connected, or it can be detachably connected, or it can be integrated; it can be mechanically connected, or it can be electrically connected; it can be directly connected, or it can be indirectly connected through an intermediate medium; it can be the internal communication of two elements or the interaction relationship between two elements, unless otherwise explicitly limited. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0223] In the present application, unless otherwise explicitly specified and limited, the first feature "on" or "under" the second feature can be that the first and second features are in direct contact, or the first and second features are in indirect contact through an intermediate medium. Moreover, the first feature "above", "over" and "on" the second feature can be that the first feature is directly above or obliquely above the second feature, or only indicates that the horizontal height of the first feature is higher than that of the second feature. The first feature "below", "under" and "under" the second feature can be that the first feature is directly below or obliquely below the second feature, or only indicates that the horizontal height of the first feature is less than that of the second feature.

[0224] Although the embodiments of the present application have been shown and described above, it should be understood that the above embodiments are exemplary and cannot be understood as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above embodiments within the scope of the present application.

Claims

1. A method for identifying the status of a substation rotary control device, characterized in that, The method includes: Obtain the first knob image of the substation rotary control device; The first knob image is input into the image feature extraction model to obtain the query features output by the image feature extraction model; wherein, the image feature extraction model is a model obtained by training a preset neural network model with sample knob images of preset category labels, and is used to extract features from the knob image; The query features are compared with the image features of each third knob image in the pre-built knob image library, and candidate images are determined from the comparison results. The status of the substation knob device is determined based on the first angle information of the first knob image and the second angle information of the candidate image. The image feature extraction model is trained in the following manner: Obtain the feature map of the sample knob image; Extract diverse global features from the feature map; By concatenating the diverse global features, a combined feature is obtained; The total loss is determined based on the combined features, and the model parameters of the neural network model are adjusted based on the total loss. The diverse global features include: a first global feature, a second global feature, and a third global feature; wherein... The first global feature is obtained based on the global average pooling calculation of each channel of the feature map; The second global feature is obtained by performing max pooling calculation on the feature map and summing the calculation results. The third global feature is obtained based on vector residual encoding of the feature map; The third global feature is obtained based on vector residual encoding of the feature map, including: Determine the local features corresponding to the feature map; By setting cluster centers, the residual vector of the local features relative to the cluster centers is calculated; The sum of the membership degrees of the cluster centers is obtained by weighting and summing the residual vector and the local features; The aforementioned features are processed to obtain the third global feature.

2. The method according to claim 1, characterized in that, The total loss is determined based on the sorting loss of the main module and the classification loss of the auxiliary module; The main module determines the sorting loss in the following manner: in, This represents the sorting loss; Indicates batch size; This represents the image of the i-th sample knob. This represents the image of the j-th sample knob. This indicates searching for anchor points within a batch. The positive sample with the largest distance among samples of the same type; This represents the combined features of the i-th sample knob image. This represents the combined features of the j-th sample knob image; This represents the image of the k-th sample knob. This indicates searching for anchor points within a batch. The negative sample with the smallest distance among the outlier samples; This represents the combined features of the k-th sample knob image; This represents the margin hyperparameter; express Calculate the loss for cases where the boundary conditions are not met; The auxiliary module determines the classification loss in the following manner: in, B represents the classification loss; C1 represents the batch size; C1 represents the total number of categories. This represents the weight of the c-th category; This represents the one-hot encoding of the true label of the i-th sample; represents the c-th column of the classifier weight matrix, corresponding to the linear transformation parameters of class c. This represents the k-th column of the classifier weight matrix; This represents the first global feature of the i-th sample knob image.

3. The method according to claim 1, characterized in that, The knob image library is constructed in the following manner: Acquire images of the second knob from different substations and different equipment models; Detect the knob image area of ​​the second knob image, and crop the knob image area to obtain the third knob image; The third knob image is classified, and a knob image library with category labels and angle information is constructed based on the classification results.

4. The method according to claim 1, characterized in that, The first angle information is obtained in the following manner: Input the first knob image into the trained segmentation model to obtain the pointer segmentation region output by the segmentation model; The segmentation model is a model obtained by training a preset second neural network model using a second sample knob image with a preset labeled box, which is used to segment the knob image. The first angle information is determined based on the pointer segmentation region.

5. The method according to claim 4, characterized in that, Determining the first angle information based on the pointer segmentation region includes: Obtain the maximum connected region for each category within the pointer segmentation region; The first angle information is determined based on the centroid of each largest connected region.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the substation knob device status identification method as described in any one of claims 1-5.

7. An electronic device, characterized in that, include: Memory, processor; The memory stores a computer program, which, when executed by the processor, implements the substation knob device status identification method as described in any one of claims 1-5.

Images