Power transformation knob equipment state identification method

Abstract

The invention discloses a power transformation knob equipment state identification method, and relates to the technical field of image processing, and the method comprises the steps: obtaining a first knob image of power transformation knob equipment; inputting the first knob image into an image feature extraction model to obtain query features output by the image feature extraction model; wherein the image feature extraction model is a model obtained by training a preset neural network model by adopting a sample knob image of a preset category label, and is used for performing feature extraction on the knob image; comparing the query features with image features of third knob images in a pre-constructed knob image library, and determining candidate images from a comparison result; and determining the state of the power transformation knob equipment based on the first angle information of the first knob image and the second angle information of the candidate image so as to improve the robustness and universality of state identification of the power transformation knob equipment.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method for identifying the state of a power-changing knob device. Background Art

[0002] During substation operation and maintenance, knobs are often used to indicate the equipment's operating status, with the angle of their pointer switches corresponding to specific operating states. Therefore, regular checks on the status of knobs are necessary to ensure stable substation operation. Traditional manual inspections are inefficient and prone to errors. Existing image-based detection and recognition solutions focus on overall equipment inspection and lack the precise recognition and status judgment of small components such as knobs. A single detection model is particularly difficult to adapt to scenarios where knob structures and their pointer switch angles vary slightly, or where background obstructions exist. Therefore, a highly robust and versatile method for identifying the status of knobs is urgently needed. Summary of the Invention

[0003] The present invention aims to solve at least one of the technical problems in the related art to a certain extent. To this end, one object of the present invention is to provide a method for identifying the state of a power-changing knob device to improve the robustness and versatility of the state identification of the power-changing knob device.

[0004] According to a first aspect of an embodiment of the present invention, a method for identifying the state of a power-changing knob device is provided, the method comprising: Obtaining a first knob image of a power-changing knob device; Inputting the first knob image into an image feature extraction model to obtain query features output by the image feature extraction model; wherein the image feature extraction model is a model obtained by training a preset neural network model using sample knob images with preset category labels, and is used to extract features from the knob image; comparing the query feature with image features of each third knob image in a pre-built knob image library, and determining a candidate image from the comparison results; The state of the power knob device is determined based on the first angle information of the first knob image and the second angle information of the candidate image.

[0005] Optionally, the image feature extraction model is trained in the following manner: Obtaining a feature map of the sample knob image; Extracting diverse global features from the feature map; Splicing the diverse global features to obtain combined features; A total loss is determined according to the combined features, and model parameters of the neural network model are adjusted based on the total loss.

[0006] Optionally, the diversified global features include: a first global feature, a second global feature and a third global feature; wherein, The first global feature is obtained based on the global average pooling calculation of each channel of the feature map; The second global feature is obtained by performing a maximum pooling calculation on the feature map and summing the calculation results; The third global feature is obtained based on vector residual encoding of the feature map.

[0007] Optionally, the third global feature is obtained based on performing vector residual coding on the feature map, including: Determining a local feature corresponding to the feature map; By setting a cluster center, calculating the residual vector of the local feature relative to the cluster center; Performing weighted summation on the membership of the cluster center based on the residual vector and the local feature to obtain a sum feature; The sum feature is processed to obtain the third global feature.

[0008] Optionally, the total loss is determined based on the ranking loss of the main module and the classification loss of the auxiliary module; The main module determines the sorting loss in the following manner:

[0009] in, represents the ranking loss; Indicates the batch size; represents the i-th sample knob image, represents the j-th sample knob image, Indicates finding the anchor point within the batch The positive sample with the largest distance among the samples of the same type; represents the combined features of the i-th sample knob image, Represents the combined features of the j-th sample knob image; represents the kth sample knob image, Indicates finding the anchor point within the batch The negative sample with the smallest distance among heterogeneous samples; Represents the combined features of the k-th sample knob image; represents the margin hyperparameter; express , calculate the loss for cases where the boundary conditions are not met; The auxiliary module determines the classification loss as follows:

[0010] in, represents the classification loss; B represents the batch size; C1 represents the total number of categories; represents the weight of the cth category; Represents the one-hot encoding of the true label of the i-th sample; Represents the c-th column of the classifier weight matrix, corresponding to the linear transformation parameter of category c, represents the kth column of the classifier weight matrix; Represents the first global feature of the i-th sample knob image.

[0011] Optionally, the knob image library is constructed in the following manner: Obtain the second knob images collected at different substations and different equipment models; detecting a knob image region of the second knob image, and cropping the knob image region to obtain a third knob image; The third knob image is classified, and a knob image library with category labels and angle information is constructed based on the classification result.

[0012] Optionally, the first angle information is obtained in the following manner: Inputting the first knob image into the trained segmentation model to obtain the pointer segmentation area output by the segmentation model; The segmentation model is a model obtained by training a preset second neural network model using a second sample knob image with a preset annotation frame, and is used to perform image segmentation on the knob image; The first angle information is determined based on the pointer segmentation area.

[0013] Optionally, the determining the first angle information based on the pointer segmentation area includes: Obtain the maximum connected area of ​​each category in the pointer segmentation area; The first angle information is determined based on the centroid of each maximum connected area.

[0014] According to a second aspect of an embodiment of the present invention, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method for identifying the state of a power-changing knob device is implemented.

[0015] According to a third aspect of an embodiment of the present invention, an electronic device is provided, comprising: a memory and a processor; a computer program is stored in the memory, and when the computer program is executed by the processor, the above-mentioned method for identifying the state of a variable-power knob device is implemented.

[0016] The solution provided by the embodiments of the present invention addresses the problems of low recognition accuracy and poor adaptability in substation knob device status recognition. By constructing a knob image library and introducing an image feature extraction model, the robustness and generalization of knob type recognition are improved, adapting to diverse knob structures. In addition to the model's feature extraction, the system also incorporates retrieval and comparison of the knob image library, enabling more accurate determination of angle information in knob images, rather than relying solely on model-based feature extraction. This provides greater robustness and versatility.

[0017] Additional aspects and advantages of the present invention will be set forth in part in the description which follows and, in part, will be obvious from the description which follows, or may be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is a flow chart of a first method for identifying the state of a power-changing knob device provided by an embodiment of the present invention; Figure 2 This is a flow chart of a model training process provided by an embodiment of the present invention; Figure 3 This is a schematic diagram of an image segmentation scenario provided by an embodiment of the present invention; Figure 4 This is a flow chart of a second method for identifying the state of a power-changing knob device provided by an embodiment of the present invention; Figure 5 It is a structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0019] The following describes embodiments of the present invention in detail, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and are not to be construed as limiting the present invention.

[0020] The following describes a method for identifying the state of a power-changing knob device, a storage medium, and an electronic device according to embodiments of the present invention with reference to the accompanying drawings.

[0021] In one embodiment of the present invention, see Figure 1 , provides a method for identifying the status of a transformer knob device, which includes the following steps S101-S104.

[0022] S101: Obtain a first knob image of a power-changing knob device.

[0023] In embodiments of the present invention, knob images of different types of substation knobs can be collected at different substations and with different device models to serve as first knob images. In a specific embodiment, the collected knob images can also be input into an object detection model, and the object detection model outputs an image of the knob area as the first knob image. The object detection model can be a YOLO model, a Faster RCNN model, or other conventional models, which will not be described in detail here.

[0024] S102: Input the first knob image into the image feature extraction model to obtain query features output by the image feature extraction model.

[0025] Among them, the image feature extraction model is: a model obtained by training a preset neural network model using sample knob images with preset category labels, which is used to extract features from knob images.

[0026] During training, you can get a batch of sample knob images ,in, Indicates batch size (Batch Size); Indicates the original image height and width; Indicates the number of RGB channels. R is a batch of sample knob images.

[0027] In one embodiment of the present invention, the neural network model is trained in the following manner so that it can output query features.

[0028] A feature map of a sample knob image is obtained; diversified global features are extracted from the feature map; the diversified global features are concatenated to obtain combined features; a total loss is determined according to the combined features, and model parameters of the neural network model are adjusted based on the total loss.

[0029] In one embodiment, the neural network model to be trained includes a pre-trained ResNeXt-50 backbone network. The sample knob image is input into the backbone network, the last fully connected classification layer in the backbone network is removed, and the feature map F at the end of Stage 4 in the backbone network is output.

[0030]

[0031] Among them, Backbone() represents the backbone network output, X is the sample knob image, R is a collection of sample knob images in a batch, represents the batch size, Indicates the number of output channels at the end of Stage 4. , In the formulas of all embodiments herein, parameters such as B, C, H, etc., which have the same letters, have the same meanings and will not be repeated in subsequent embodiments.

[0032] Global features refer to global descriptors, which extract features of the entire image from the feature map. Diversified global features refer to extracting a variety of global features, which enhances the representational power of features and makes the combined features output by the trained model more accurate.

[0033] The specific method of adjusting the model parameters based on the total loss can be to update the model parameters by back propagation and gradient descent until a predetermined number of iterations or the loss function of the total loss converges, so that the trained model can receive the first knob image as input and output the first knob image as a query feature in the manner of obtaining the combined feature.

[0034] The types of global features will be described in subsequent embodiments and will not be described in detail here.

[0035] The first knob image is normalized and resized and then passed into the trained feature extraction model.

[0036] S103: Compare the query feature with the image features of each third knob image in the pre-built knob image library, and determine a candidate image from the comparison result.

[0037] The image features of each third knob image can be obtained through feature extraction algorithms such as principal component analysis and histogram of oriented gradients, or can be obtained by feature extraction according to the image feature extraction model in step S102, which is not limited in this embodiment of the present invention.

[0038] When image features are obtained, the comparison method can be to calculate the cosine distance between the feature vector corresponding to the image feature and the query feature vector corresponding to the query feature, and obtain the cosine distance matrix with all features in the knob image library:

[0039] in, represents the cosine distance, represents the query feature vector, Indicates the image feature of the i-th third knob image in the knob feature library. In the embodiment of the present invention, the use of "first", "second", "third", etc. is used to distinguish from the knob image in step S101, and does not mean a limitation on the order or type.

[0040] In the comparison results, the top-K indexes are returned by sorting by cosine distance. For example, when K is 1, a third knob image with the closest cosine distance can be obtained as a candidate image. In other embodiments, K can also be an integer greater than 1.

[0041] S104: Determine the state of the power-changing knob device based on the first angle information of the first knob image and the second angle information of the candidate image.

[0042] When constructing the knob image library, the second angle information of the third knob image can be pre-stored, including the rotation angle in the third knob image and the scales corresponding to different angles, as well as the state labels of different states represented by different angles, as shown in Table 1.

[0043] Table 1

[0044] In Table 1, id and xuanniu_name represent the number and name of the third knob image, respectively. num_scale represents the total number of available states for the power knob device. For example, in row id = 0, xuanniu1 represents the name, num_scale = 2 indicates that there are two power knob device states, and xuanniu_scale represents the angle corresponding to each state. Each angle corresponds to a power knob device state, and different power knob device states are marked with different state labels L. |-90|0| indicates that there are two power knob device states corresponding to angles of -90 degrees and 0 degrees, so num_scale = 2. In the second row, |-90|90|45| represents three power knob device states, so num_scale = 3 accordingly.

[0045] The first angle information is the angle of the knob in the first knob image. The second angle information is used to select the angle of the third knob image from the candidate images that is closest to the angle of the knob in the knob image. Based on the determined angle of the third knob image, the power-changing knob device state corresponding to that angle is obtained and used as the power-changing knob device state of the first knob image. For example, in Table 1, if the first angle information is -5 degrees, the closest state is the power-changing knob device state represented by angle 0 in row id=0.

[0046] The solution provided by the embodiments of the present invention addresses the problems of low recognition accuracy and poor adaptability in substation knob device status recognition. By constructing a knob image library and introducing an image feature extraction model, the robustness and generalization of knob type recognition are improved, adapting to diverse knob structures. In addition to the model's feature extraction, the system also incorporates retrieval and comparison of the knob image library, enabling more accurate determination of angle information in knob images, rather than relying solely on model-based feature extraction. This provides greater robustness and versatility.

[0047] The following embodiments illustrate the types of global features in step S102.

[0048] In one embodiment of the present invention, the diversified global features include: a first global feature, a second global feature and a third global feature; wherein, The first global feature is obtained by global average pooling calculation of each channel of the feature map; The second global feature is obtained by performing maximum pooling calculation on the feature map and summing the calculation results; The third global feature is obtained by performing vector residual encoding on the feature map.

[0049] Specifically, the global average pooling calculation of the first global feature is performed as follows:

[0050] in, represents the first global feature of the b-th sample knob image, Represents the height and width of the feature map of the b-th sample knob image of the feature map, represents the pixel value of the i-th row and j-th column in the b-th sample knob image, R represents the set of sample knob images in a batch, and C represents the number of output channels.

[0051] For the second global feature, first generate a multi-scale region on the feature map F of each sample knob image. Specifically, set the number of levels L=3, and the multi-scale region includes three sliding windows of different sizes generated on the feature map F. The square area is used as the RoI (Region of Interest).

[0052] For each region of interest, use the RoIAlign method to obtain the regional features of the region of interest :

[0053] in, represents the batch size; C represents the number of output channels; To fix the space size after alignment, you can set the size to 7*7.

[0054] The above maximum pooling calculation process includes Perform maximum pooling, and the pooling result is :

[0055] Among them, MaxPool() represents the maximum pooling, and the other parameters have the same meaning as the parameters represented by the same letters in the above formula. Do L2 normalization and get Finally, summing over all RoIs yields:

[0056] in, Represents the second global feature; the remaining parameters have the same meanings as the parameters represented by the same letters in the above formula.

[0057] For the third global feature, the process of vector residual encoding includes: determining the local feature corresponding to the feature map; By setting the cluster center, the residual vector of the local feature relative to the cluster center is calculated; The membership of the cluster center is weighted summed based on the residual vector and local features to obtain the sum feature; The and features are processed to obtain the third global feature.

[0058] Specifically, the feature map F is first expanded to expand the feature map of H×W spatial dimensions into a feature vector of dimension length N. , which is equivalent to converting the features of the two-dimensional space into the one-dimensional space features of the same representation, as follows:

[0059] in, Can be decomposed into each sample knob image indivual dimensional local features .

[0060] use The convolution map is Cluster Center:

[0061] Among them, S represents The corresponding similarity score with each cluster center, K is the number of cluster centers Represents a convolutional map.

[0062] 1x1 convolutions can change the number of channels without changing the spatial dimensions of the feature map (which includes both height and width). In clustering tasks, the input feature map may have multiple channels, each containing different aspects of feature information. 1x1 convolutions can linearly combine these multi-channel features, integrating information from different channels and making the features more compact and representative.

[0063] For example, if the input feature map is 28x28x192, a 1x1 convolution can be used to map its channel number to K (the number of cluster centers), resulting in an output of 28x28xK. In this way, the K values ​​at each spatial position can be regarded as the associated features of the position and the K cluster centers.

[0064] The membership matrix is ​​obtained through Softmax:

[0065] in, represents the membership of the nth local feature in the bth sample knob image to the kth cluster center; It represents the similarity score between the nth local feature in the bth sample knob image and the kth cluster center, which is obtained by S in the above embodiment.

[0066] Set learnable cluster centers , calculate the residual:

[0067] in, is the residual vector of the nth local feature in the bth sample knob image relative to the kth cluster center, represents the nth local feature in the bth sample knob image, Represents the characteristics of the k-th cluster center.

[0068] The cluster centers can be initially determined by random initialization and can be iterated a certain number of times by recalculating the means of all points in each cluster and using these means as the new cluster centers.

[0069] The residual vector is weighted and summed over the spatial dimension to obtain the sum feature:

[0070] in, is the sum feature of the b-th sample knob image relative to the k-th cluster center. Expanding it is equivalent to connecting K C-dimensional vectors end to end to obtain a long vector V of length K×C:

[0071] Perform PCA dimensionality reduction and L2 normalization on the long vector V to output the third global feature. The PCA projection matrix is ​​set to The third global feature is obtained through the above method, in which the K clustering method is used to establish a visual semantic dictionary to provide semantic anchors for local features, and the membership matrix is ​​calculated to provide a weighted basis for the calculation of the residual vector, thereby improving the discriminability of semantic expression.

[0072] According to the above embodiment, the diversified global features are spliced ​​to obtain the combined features, that is, the feature vectors of the first, second and third global features are uniformly L2 normalized and then spliced ​​to obtain the combined descriptor :

[0073] in, represents the first global feature after L2 normalization, represents the second global feature after L2 normalization, Represents the third global feature after L2 normalization.

[0074] In one embodiment of the present invention, the total loss is determined based on the ranking loss of the main module and the classification loss of the auxiliary module; The main module uses the concatenated combined descriptor features to calculate the ranking loss and learn the relative distance between samples. Here, the hard sample triple loss is used:

[0075] in, represents the ranking loss; Indicates the batch size; represents the i-th sample knob image, represents the j-th sample knob image, Indicates searching for the same type of knob image as the i-th sample in the batch ( ) is the positive sample with the largest distance among the samples; represents the combined features of the i-th sample knob image, Represents the combined features of the j-th sample knob image; represents the kth sample knob image, Indicates finding the image of knob that is different from the i-th sample in the batch ( ) is the negative sample with the smallest distance among the samples; Represents the combined features of the k-th sample knob image; represents the margin hyperparameter; express , calculate the loss for cases where the boundary conditions are not met.

[0076] The margin hyperparameter has a preset value, for example, it can be 0.2~0.5.

[0077] The auxiliary module determines the classification loss as follows, specifically taking the output of the first global feature as input and calculating the weighted cross entropy loss:

[0078] in, Represents classification loss; B represents batch size; C1 represents the total number of categories, and each category corresponds to a category label; represents the weight of the cth category; Represents the one-hot encoding of the true label of the i-th sample, where the true label can be the true category of the pre-labeled sample knob image; Represents the c-th column of the classifier weight matrix, corresponding to the linear transformation parameter of category c, represents the kth column of the classifier weight matrix; represents the first global feature of the i-th sample knob image. The classifier weight matrix is ​​initialized to a C1×d parameter matrix using the function Linear(d, C1). This matrix is ​​subsequently updated during training using backpropagation and gradient descent.

[0079] The calculation of the above classification loss is an auxiliary training strategy, which is used to: 1) Guide the model to extract representations that are sensitive to category discrimination; 2) Imposing a more stable supervision signal on the backbone network; 3) Avoid the main task, i.e., ranking loss, from over-reliance on difficult samples, which may lead to feature collapse or unstable training.

[0080] The first global feature is selected as input because, of the three types of global features mentioned above, it has the simplest structure, the most stable information, and the least position-sensitive feature representation. This makes it suitable as input for training the classifier in the auxiliary module, guiding the main module to learn discriminative semantic representations.

[0081] From a general perspective, the auxiliary module can improve the backbone network's ability to extract semantic representations through a lightweight but strongly supervised classification auxiliary task, thereby enhancing the matching retrieval effect of the main task.

[0082] The total loss is the weighted sum of the ranking loss and the classification loss:

[0083] in is the balancing factor, for example, You can take 0.5. Figure 2 An embodiment of model training is described in FIG.

[0084] Among them, the candidate box, positive sample and negative sample are first input into the backbone network. In the sample knob image, the candidate box is the annotation box in the pre-specified image, and the positive sample indicates that the input is a real and correct sample knob image of the target category in the current task. Negative samples indicate that the input is not a sample of the current target category, but a sample knob image of a different category or an error. Here, samples that match the candidate box knob are positive samples, and those that do not match the candidate box knob are negative samples. You can enter one or more positive or negative samples.

[0085] The first global feature is generated in the global average pooling module, the second global feature is generated in the multi-scale region max pooling module, and the third global feature is generated in the local feature vector aggregation module. The first global feature contributes to the weighted cross-entropy loss of the auxiliary module. The three global features are concatenated to generate the hard sample triplet loss. The model parameters are adjusted based on the weighted sum of the two losses. By extracting diverse global features, the trained model improves the robustness and generalization ability of knob type recognition and adapts to diverse knob structures.

[0086] In one embodiment, the knob image library is constructed as follows: Obtain the second knob images collected at different substations and different equipment models; detecting a knob image region of the second knob image, and cropping the knob image region to obtain a third knob image; The third knob image is classified, and a knob image library with category labels and angle information is constructed based on the classification results.

[0087] The method of detecting the knob image region can be similar to the aforementioned step S101 and implemented using an object detection model, etc. The third knob image is an image formed only by the knob image region.

[0088] The angle information can be manually marked, and the states of the power-changing knob device corresponding to different angles can be noted, see the embodiment of Table 1 above.

[0089] In one embodiment of the present invention, the first angle information is obtained in the following manner: Inputting the first knob image into the trained segmentation model to obtain the pointer segmentation area output by the segmentation model; The segmentation model is a model obtained by training a preset second neural network model using a second sample knob image with a preset annotation frame, and is used to perform image segmentation on the knob image; First angle information is determined based on the pointer segmentation area.

[0090] The second sample knob image is a manually annotated knob image. During the annotation process, the pointer mask coordinates of the second sample knob image are converted into normalized polygon vertex coordinates as labels to generate an annotated mask area. The image is input into the second neural network model, which is an improved YOLO11-seg model, for model training.

[0091] The main structure of YOLO11-seg remains unchanged, consisting of the Backbone network, the Neck layer, and the Head. The second sample knob image is preprocessed and passed to the Backbone layer to extract multi-scale features. The features are then fused through the Neck's Feature Pyramid Network and Path Aggregation Network to generate fused features. Finally, the Head layer detects and segments the branches based on the fused features. During training, the parameters of the second network model are adjusted based on the segmentation loss, similar to the method used for training the image feature extraction model.

[0092] In the path aggregation network, multi-scale features are upsampled as follows, and feature fusion is performed based on the upsampled features: Since the upsampling process of the nearest neighbor interpolation has a large information loss, the recovery of small areas is not clear. Therefore, the standard upsampling in the YOLO11-seg decoder is replaced by learnable upsampling, and the low-resolution feature map of the second sample knob image is input to Conv2d, and the number of output channels is set to . Use PixelShuffle(r) The image is restored to r times the resolution of the C channel, and feature fusion is performed based on the image features of the r times the resolution image.

[0093] In the training of the segmentation head, Focal Tversky Loss is used as the main segmentation loss, and background suppression loss is added. The overall loss function is defined as:

[0094] in For Focal Tversky Loss, is the background suppression loss; 、 is a preset weight coefficient. Focal Tversky Loss is calculated based on the difference between the segmented area output by the second neural model and the manually labeled segmented area; background suppression loss is calculated based on the difference between the background area outside the segmented area output by the second neural model and the background area outside the manually labeled segmented area.

[0095] In one embodiment of the present invention, the original first knob image may be preprocessed before being input into the segmentation model, including: Calculate the scaling ratio and target size of the segmentation model input ( ):

[0096] Calculate the horizontal offset of the fill offset and longitudinal offset :

[0097] Then perform proportional scaling and grayscale filling

[0098] in , Represents the first knob image after preprocessing, ratio represents the scaling ratio, Indicates proportional scaling. () represents the grayscale filling function, and the parameters in the function represent the range of grayscale filling. Represents the original first knob image. , Indicates the original image height and width.

[0099] The segmented region output by the second neural network model is obtained in the following manner.

[0100] The preprocessed first knob image Input the second neural network model, where the model is processed by the forward function f:

[0101] Among them, the candidate bounding box ; Confidence score ; Category Index , where the category index corresponds to the category with the maximum probability after Softmax classification of each candidate bounding box; the mask area .

[0102] The output of the forward function is processed using the NMS operator, where the NMS (Non-Maximum Suppression) operator is defined as , It represents the confidence threshold, so that among all the candidate bounding boxes output by the model, the confidence scores are first filtered out of the candidate bounding boxes below the confidence threshold. Is the intersection-over-union threshold. For each category, each time the candidate bounding box with the highest confidence is selected as the reserved box, and then the candidate bounding box with an IoU greater than The other boxes are suppressed.

[0103] After filtering by the NMS operator, we get:

[0104] in, represents the candidate bounding box after screening, S, L, and M are the confidence score, category index, and mask area corresponding to B.

[0105] Restore the original image scale and crop the mask area, including: Set the fill vector , for each mask area , and process it according to the following formula:

[0106] in, is the candidate bounding box corresponding to the i-th mask area and restored in size; is the candidate bounding box before restoring the size. is the ratio calculated in the previous embodiment.

[0107] Group by category and select the best result of segmentation area, predefined category set ,for definition

[0108] like ,Pick , and output .

[0109] represents the category index of the i-th candidate bounding box, Equivalent to selection The highest confidence score for each category represented The corresponding candidate bounding box is , so it is, yes The mask area, is the highest confidence score.

[0110] The final result is for each category The triplet .in, is the segmented region output by the second neural network model. Figure 3 , A, B, and C correspond to three states of the power knob device. In the current embodiment, the knob pointer 3 points to state C. Area 1 is , region 2 is .

[0111] In one embodiment of the present invention, determining the first angle information based on the pointer segmentation area includes: Get the maximum connected area of ​​each category in the pointer segmentation area; The first angle information is determined based on the centroid of each maximum connected region.

[0112] Specifically, in the pointer segmentation area, Perform binarization processing and The pixel values ​​in are normalized to the interval [0, 1] and binarized with a threshold of 0.5. The connected domains (Contours) of each category are extracted from the binarized results and sorted in descending order by area.

[0113] After sorting in descending order by area, the connected area with the largest area is taken from each category, which is also the maximum connected area.

[0114] The maximum connected area of ​​each class , calculate the center of mass:

[0115] in is the number of pixels in the region, is the centroid of the largest connected region of category i.

[0116] Let the centroid of the first region be , the second centroid is , then the angle between the connecting line and the vertical direction is

[0117]

[0118] Calculated That is, after matching the first angle information with the second angle information, the state of the power-changing knob device represented by the pre-stored label L corresponding to the angle and scale is output.

[0119] In terms of state recognition, the knob pointer area is accurately segmented by training the segmentation model, combined with OpenCV's closed operation denoising processing, and the pointer direction angle is obtained using contour fitting and center of mass calculation. The knob state is judged based on the preset angle threshold and scale information, thereby achieving high-precision state recognition in complex backgrounds and small angle differences.

[0120] The following Figure 4 The illustrated embodiment illustrates the overall process of the knob device state identification method provided by the present invention.

[0121] The captured knob image is input into the first-level device switch detector to detect the knob area and perform image cropping of the original image area. The first-level device switch detector refers to the target detection model in S101.

[0122] The cropped knob images are classified to construct a knob image library with category labels and scale ranges. The scale range is the scale corresponding to different angles of the second angle information.

[0123] Build and train an image feature extraction model.

[0124] The knob image is fed into the image feature extraction model, where the distance between the knob image image and the third image feature in the knob image library is calculated to obtain the category retrieval and scale range. The category retrieval is represented by the state label L, and the scale range is represented by the second angle information.

[0125] Build and train a segmentation model.

[0126] The knob image is input into the segmentation model to obtain the mask area for pointer segmentation.

[0127] The mask area is subjected to morphological denoising and binarization processing, the contour of the largest connected area is extracted, the centroid between regions is calculated and the angle is calculated, and the state of the output knob is matched, that is, the state of the transformer knob device is determined.

[0128] In one embodiment of the present invention, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method for identifying the state of a power-changing knob device of any of the above embodiments is implemented.

[0129] In one embodiment of the present invention, an electronic device is provided, comprising: a memory and a processor; a computer program is stored in the memory, and when the computer program is executed by the processor, the state identification method of the power-changing knob device of any of the above embodiments is implemented.

[0130] Figure 5 It is a structural block diagram of an electronic device according to an embodiment of the present invention.

[0131] like Figure 5 As shown, electronic device 500 includes: a processor 501 and a memory 503. Processor 501 and memory 503 are connected, for example, via a bus 502. Optionally, electronic device 500 may further include a transceiver 504. It should be noted that in actual applications, the number of transceivers 504 is not limited to one, and the structure of electronic device 500 does not constitute a limitation on the embodiments of the present invention.

[0132] Processor 501 can be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic device, transistor logic device, hardware component, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the present disclosure. Processor 501 can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0133] Bus 502 may include a path for transmitting information between the above components. Bus 502 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. Bus 502 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0134] The memory 503 is used to store a computer program corresponding to the state identification method of the power-changing knob device according to the above embodiment of the present invention, and the computer program is controlled and executed by the processor 501. The processor 501 is used to execute the computer program stored in the memory 503 to implement the content shown in the above method embodiment.

[0135] Among them, the electronic device 500 includes but is not limited to: mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 5 The electronic device 500 shown is merely an example and should not limit the functions and scope of use of the embodiments of the present invention.

[0136] It should be noted that the logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic device), a portable computer disk cartridge (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and a portable compact disc read-only memory (CDROM). Furthermore, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing it in another suitable manner if necessary, and then storing it in a computer memory.

[0137] It should be understood that various components of the present invention may be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods may be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof may be used: a discrete logic circuit having logic gate circuits for implementing logic functions on data signals, an application-specific integrated circuit having suitable combinational logic gate circuits, a programmable gate array (PGA), a field-programmable gate array (FPGA), etc.

[0138] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" means that a specific feature, structure, material, or characteristic described in conjunction with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0139] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential" and the like to indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.

[0140] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.

[0141] In the present invention, unless otherwise specified or limited, the terms "installed," "connected," "connect," "fixed," etc. should be understood in a broad sense. For example, they can refer to fixed connection, detachable connection, or integration; mechanical connection, electrical connection; direct connection, or indirect connection through an intermediate medium; internal communication between two components, or interaction between two components, unless otherwise specified. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0142] In the present invention, unless otherwise expressly specified or limited, when a first feature is "above" or "below" a second feature, it may mean that the first and second features are in direct contact, or that the first and second features are in indirect contact through an intermediary. Furthermore, when a first feature is "above," "above," or "above" a second feature, it may mean that the first feature is directly above or diagonally above the second feature, or simply means that the first feature is at a higher level than the second feature. When a first feature is "below," "below," or "below" a second feature, it may mean that the first feature is directly below or diagonally below the second feature, or simply means that the first feature is at a lower level than the second feature.

[0143] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A method for identifying the state of a transformer knob device, characterized in that: The method comprises: Obtaining a first knob image of a power-changing knob device; Inputting the first knob image into an image feature extraction model to obtain query features output by the image feature extraction model; wherein the image feature extraction model is a model obtained by training a preset neural network model using sample knob images with preset category labels, and is used to extract features from the knob image; comparing the query feature with image features of each third knob image in a pre-built knob image library, and determining a candidate image from the comparison results; The state of the power knob device is determined based on the first angle information of the first knob image and the second angle information of the candidate image.

2. The method according to claim 1, characterized in that The image feature extraction model is trained as follows: Obtaining a feature map of the sample knob image; Extracting diverse global features from the feature map; Splicing the diverse global features to obtain combined features; A total loss is determined according to the combined features, and model parameters of the neural network model are adjusted based on the total loss.

3. The method according to claim 2, characterized in that The diversified global features include: a first global feature, a second global feature and a third global feature; wherein, The first global feature is obtained based on the global average pooling calculation of each channel of the feature map; The second global feature is obtained by performing a maximum pooling calculation on the feature map and summing the calculation results; The third global feature is obtained based on vector residual encoding of the feature map.

4. The method according to claim 3, characterized in that The third global feature is obtained based on performing vector residual coding on the feature map, including: Determining a local feature corresponding to the feature map; By setting a cluster center, calculating the residual vector of the local feature relative to the cluster center; Performing weighted summation on the membership of the cluster center based on the residual vector and the local feature to obtain a sum feature; The sum feature is processed to obtain the third global feature.

5. The method according to claim 3, characterized in that The total loss is determined based on the ranking loss of the main module and the classification loss of the auxiliary module; The main module determines the sorting loss in the following manner: in, represents the ranking loss; Indicates the batch size; represents the i-th sample knob image, represents the j-th sample knob image, Indicates finding the anchor point within the batch The positive sample with the largest distance among the samples of the same type; represents the combined features of the i-th sample knob image, Represents the combined features of the j-th sample knob image; represents the kth sample knob image, Indicates finding the anchor point within the batch The negative sample with the smallest distance among heterogeneous samples; Represents the combined features of the k-th sample knob image; represents the margin hyperparameter; express , calculate the loss for cases where the boundary conditions are not met; The auxiliary module determines the classification loss as follows: in, represents the classification loss; B represents the batch size; C1 represents the total number of categories; represents the weight of the cth category; Represents the one-hot encoding of the true label of the i-th sample; Represents the c-th column of the classifier weight matrix, corresponding to the linear transformation parameter of category c, represents the kth column of the classifier weight matrix; Represents the first global feature of the i-th sample knob image.

6. The method according to claim 1, wherein The knob image library is constructed in the following manner: Obtain the second knob images collected at different substations and different equipment models; detecting a knob image region of the second knob image, and cropping the knob image region to obtain a third knob image; The third knob image is classified, and a knob image library with category labels and angle information is constructed based on the classification result.

7. The method according to claim 1, characterized in that The first angle information is obtained in the following manner: Inputting the first knob image into the trained segmentation model to obtain the pointer segmentation area output by the segmentation model; The segmentation model is a model obtained by training a preset second neural network model using a second sample knob image with a preset annotation frame, and is used to perform image segmentation on the knob image; The first angle information is determined based on the pointer segmentation area.

8. The method according to claim 7, characterized in that The determining the first angle information based on the pointer segmentation area includes: Obtain the maximum connected area of ​​each category in the pointer segmentation area; The first angle information is determined based on the centroid of each maximum connected area.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for identifying the state of a power-changing knob device according to any one of claims 1 to 8 is implemented.

10. An electronic device, characterized in that: include: Memory, processor; The memory stores a computer program, and when the processor executes the computer program, the method for identifying the state of a power-changing knob device according to any one of claims 1 to 8 is implemented.

Images