Electric power operation safety supervision target detection method, system and device based on semantic clustering and medium

Abstract

The invention discloses an electric power operation safety supervision target detection method, system and device based on semantic clustering and a medium, and belongs to the technical field of target detection and semantic analysis, and the method comprises the steps: extracting the image features of a target picture, determining candidate boxes needed by target detection through a region generation network, and unifying the sizes of the candidate boxes; extracting category semantic features of the target picture, and generating category semantic feature vectors; according to the candidate box, generating a clustering feature vector through a multi-layer sensing mechanism; calculating class decorrelation loss based on the clustering feature vector; calculating semantic clustering loss; and according to the candidate box, a target detection result is obtained through a multi-layer sensing mechanism, and classification loss and regression loss are calculated. According to the method, the wearing state of safety protection equipment and abnormal equipment can be accurately detected in a complex power grid environment, the problems of misjudgment and missing detection of unknown samples by a traditional detection model are effectively solved through semantic feature constraint and feature space reconstruction, and the intelligent safety supervision efficiency in scenes such as electric power inspection and high-altitude operation is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of target detection and semantic analysis, and in particular to a method, system, equipment and medium for detecting safety and supervision targets in power operations based on semantic clustering. Background Art

[0002] With the rapid development of smart grid construction, the demand for safety supervision in power operation scenarios (such as substation inspections, high-altitude operations, and live operations) is increasing. In complex power grid environments, target detection tasks must simultaneously address the precise identification of known target objects (such as safety equipment such as helmets, insulating gloves, and goggles) and the effective differentiation of dynamically emerging unknown risk targets (such as foreign objects on lines, equipment sparks, and objects thrown from high altitudes).

[0003] However, traditional target detection methods still have the following key defects: (1) Insufficient generalization ability:

[0004] The power scene environment is complex and the interference of non-operating targets is significant. The existing traditional end-to-end target detection model is highly dependent on the stacking of training data. The semantic feature space depends on the distribution of training data, has incomplete coverage and is sensitive to data distribution. When environmental interference or unknown risk targets occur, the feature distribution offset will cause the detection performance to be significantly reduced; (2) Poor robustness of semantic association and clustering: There are common features in the power scene, such as small target body, multiple targets at the same time, and partial occlusion. These directly lead to problems such as fuzzy feature space distribution and weak category semantic association in conventional detection algorithms, which in turn lead to false detection and missed detection, causing the safety warning system to fail. Although some studies have attempted to introduce zero-shot learning or contrastive learning mechanisms to improve the generalization ability of the model, there are generally defects such as insufficient semantic feature alignment accuracy and fuzzy discrimination boundaries. (3) Lack of normal cognition and unknown risk identification capabilities: Existing technologies mostly focus on optimizing the detection accuracy of known categories, lack the ability to decouple features and dynamically identify unknown categories, and are difficult to meet the real-time perception needs of "a priori unknown risks" in power grid operation scenarios. Therefore, there is an urgent need for a power operation safety and supervision target detection method that can integrate semantic knowledge constraints and enhance the structured expression of feature space, so as to improve the recognition accuracy of conventional protective equipment while achieving effective discovery and feature modeling of potential dangerous objects. Summary of the Invention

[0005] In view of the above-mentioned problems, the present invention is proposed.

[0006] Therefore, the technical problem solved by the present invention is: how to enhance the generalization ability of unknown risk objects with semantics close to known categories through image feature extraction, category semantic generation, clustering feature optimization and multi-loss calculation, while achieving efficient dynamic distinction between known protective equipment and unknown dangerous targets, and solving the problems of false detection and missed detection caused by feature space overlap and fuzzy semantic association in traditional methods.

[0007] In order to solve the above technical problems, the present invention provides the following technical solutions: a method for detecting power operation safety and supervision targets based on semantic clustering, which comprises the following steps:

[0008] Extracting image features of the target image, determining candidate frames required for target detection based on the image features, and unifying the sizes of the candidate frames;

[0009] Extracting the category semantic features of the target image and generating a semantic feature vector of the category semantic features;

[0010] Generate a cluster feature vector based on the candidate box;

[0011] Calculating class decorrelation loss based on the cluster feature vector;

[0012] Inputting the semantic feature vector and the clustering feature vector into a semantic clustering module to calculate the semantic clustering loss;

[0013] Based on the candidate boxes, the target detection results are obtained, and the classification loss and regression loss are calculated.

[0014] As a preferred solution of the method for detecting power operation safety and supervision targets based on semantic clustering described in the present invention, extracting image features of the target image includes:

[0015] Build a backbone network;

[0016] The target image is input into the backbone network to obtain the deep features of the target image.

[0017] As a preferred solution of the method for detecting power operation safety targets based on semantic clustering described in the present invention, extracting the category semantic features of the target image includes:

[0018] Build a model that includes a text encoder and an image encoder;

[0019] Constructing text-image pair data, and using the text-image pair data to train the model including the text encoder and the image encoder;

[0020] The class names of the known classes in the data set are input into the trained model including the text encoder and the image encoder to extract the semantic text features of the class semantic features.

[0021] As a preferred solution of the method for detecting power operation safety and supervision targets based on semantic clustering described in the present invention, determining the candidate boxes required for target detection includes:

[0022] Presetting anchor points according to image features of the target image;

[0023] Classify and regress the anchor points to obtain classification results and coordinate offsets of the anchor points;

[0024] Generating a candidate frame required for target detection according to the classification result of the anchor point and the coordinate offset of the anchor point;

[0025] The features of the candidate boxes are processed into feature maps of the same size.

[0026] As a preferred solution of the method for detecting power operation safety and supervision targets based on semantic clustering described in the present invention, calculating the semantic clustering loss includes:

[0027] Extract semantic features of known classes;

[0028] Obtaining clustering features of the candidate box;

[0029] Calculating the similarity between the cluster feature vector and the semantic feature vector of the category semantic feature;

[0030] The semantic clustering loss is calculated according to the similarity.

[0031] As a preferred solution of the method for detecting power operation safety and supervision targets based on semantic clustering described in the present invention, the calculation of class decorrelation loss includes:

[0032] Collecting a category feature subset according to the cluster feature vector of the candidate box;

[0033] constructing a category identity matrix based on the category feature subset;

[0034] Calculating the matching correlation of the cluster feature vectors of the candidate boxes;

[0035] A class decorrelation loss of the class identity matrix and the matching correlation is calculated.

[0036] As a preferred solution of the method for detecting electric power operation safety and supervision targets based on semantic clustering described in the present invention, the cosine similarity formula is used to calculate the similarity between the cluster feature vector and the semantic feature vector of the category semantic feature.

[0037] The beneficial effects of this preferred technical solution are: especially for complex power grid operation scenarios, it can accurately detect common typical targets such as safety protection equipment. It can also achieve normal awareness and risk identification of potential unknown risks (such as foreign objects on the lines and objects thrown from high places) in power operation safety supervision.

[0038] Another object of the present invention is to provide a system for detecting safety and supervision targets in power operations based on semantic clustering.

[0039] To solve the above technical problems, the present invention provides the following technical solutions: a system for detecting power operation safety and supervision targets based on semantic clustering, comprising: an image feature extraction module for extracting image features of a target image, and determining candidate boxes required for target detection based on the image features through a region generation network, while unifying the sizes of the candidate boxes;

[0040] A category semantic feature extraction module is used to extract the category semantic features of the target image and generate a semantic feature vector of the category through a text encoder;

[0041] A cluster feature generation module, configured to generate a cluster feature vector based on the candidate box;

[0042] A class decorrelation loss calculation module, configured to calculate the class decorrelation loss based on the clustering feature vector;

[0043] A semantic clustering loss calculation module, configured to input the semantic feature vector and the clustering feature vector into a semantic clustering module to calculate the semantic clustering loss;

[0044] The target detection and loss calculation module is used to obtain the target detection result based on the candidate box and calculate the classification loss and regression loss.

[0045] The present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program and is characterized in that when the processor executes the computer program, the steps of the method for detecting safety and supervision targets of electric power operations based on semantic clustering are implemented.

[0046] The present invention provides a computer-readable storage medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, the steps of the method for detecting power operation safety and supervision targets based on semantic clustering are implemented.

[0047] The beneficial effects of the present invention are as follows: through the coordinated optimization of the region generation network and the multi-layer perception mechanism, efficient distinction between known categories (such as safety helmets, insulating gloves) and unknown risk targets (such as charged foreign objects, insulation defects) is achieved; the semantic feature vectors of the categories are generated by the text encoder, which reduces the sensitivity to the distribution of training data and improves the accuracy of unknown category recognition; the use of a unified candidate box size and a multi-layer perception mechanism further improves the algorithm's operating efficiency, enabling it to meet the needs of real-time application scenarios; through the joint analysis of semantic feature vectors and clustering feature vectors, combined with the comprehensive calculation of classification loss and regression loss, accurate detection of known category targets and effective recognition of unknown category targets are achieved, breaking through the obstacle that the existing technology cannot effectively identify unknown categories with semantics similar to known categories, and providing reliable technical support for target detection in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0049] Fig. 1 A schematic diagram of the overall architecture of a method for detecting safety and supervision targets in power operations based on semantic clustering is provided as an embodiment of the present invention.

[0050] Fig. 2 A schematic diagram of the architecture of a semantic clustering module of a method for detecting safety and supervision targets in power operations based on semantic clustering is provided in accordance with an embodiment of the present invention.

[0051] Fig. 3 A schematic diagram of the architecture of a class decorrelation module of a method for detecting power operation safety and supervision targets based on semantic clustering is provided in accordance with an embodiment of the present invention. DETAILED DESCRIPTION

[0052] To make the above-mentioned objects, features, and advantages of the present invention more clearly understood, the following detailed description of the specific embodiments of the present invention is given in conjunction with the accompanying drawings. It is obvious that the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in this field without creative work should fall within the scope of protection of the present invention.

[0053] Example 1, with reference to Figs. 1-3 , which is the first embodiment of the present invention, provides a method for detecting power operation safety and supervision targets based on semantic clustering, comprising:

[0054] S1: Extract the image features of the target image, determine the candidate boxes required for target detection based on the image features, and unify the sizes of the candidate boxes;

[0055] S2: Extract the category semantic features of the target image and generate a semantic feature vector of the category semantic features;

[0056] S3: Generate cluster feature vector based on candidate boxes;

[0057] S4: Calculate class decorrelation loss based on cluster feature vectors;

[0058] S5: Input the semantic feature vector and clustering feature vector into the semantic clustering module to calculate the semantic clustering loss;

[0059] S6: Based on the candidate box, obtain the target detection result and calculate the classification loss and regression loss.

[0060] It should be noted that when extracting image features from target images and determining candidate boxes through region generation networks, traditional methods only use semantic clustering methods, which causes similar classes to be close in feature space, making it impossible to effectively distinguish unknown classes, causing the model to misclassify semantically similar unknown objects as known categories. In addition, when generating category semantic feature vectors through text encoders, if there are few samples of certain categories in the training data or there is annotation noise, the generated semantic feature vectors may not fully reflect the semantic information in the real scene. At the same time, after extracting image features and generating candidate boxes, traditional methods often only focus on the feature distribution of known categories, while ignoring the potential characteristics of unknown categories, resulting in poor performance of the model when processing unknown categories. Therefore, it is very important to improve the generalization ability, robustness and computational efficiency of open set object detection.

[0061] Therefore, through the technical solutions of S1 to S6, the present invention proposes a method for detecting power operation safety targets based on semantic clustering. By extracting image features and determining candidate boxes through a region generation network, and combining a multi-layer perception mechanism to generate cluster feature vectors, it is possible to solve the problem that unknown categories and known categories are difficult to distinguish when the feature space is highly overlapped, and to achieve alignment of the representation of known classes, thereby accurately detecting unknown class objects. By generating category semantic feature vectors through a text encoder, and combining the class decorrelation module and the semantic clustering module, clear cluster boundaries are divided between different clusters, and a more comprehensive and stable semantic feature space is constructed, which reduces the sensitivity to the distribution of training data, thereby improving the accuracy of unknown category recognition. Finally, by optimizing the loss calculation strategy to meet real-time requirements, efficient detection and differentiation of known and unknown categories are achieved, providing reliable technical support for open set target detection.

[0062] Example 2, reference Fig. 2 and Fig. 3 , which is the second embodiment of the present invention, provides a method for detecting power operation safety and supervision targets based on semantic clustering.

[0063] In the embodiment of the present application, extracting the image features of the target image includes the following steps A1 and A2:

[0064] A1: Build a backbone network, which consists of convolutional layers, activation layers, and pooling layers.

[0065] A2: According to the backbone network, the target image is input into it to obtain the deep features of the target image.

[0066] Specifically, the backbone network consists of 13 convolutional layers, 13 activation layers and 4 pooling layers; the convolution kernel size of the convolutional layer is 3*3 and the stride is 1.

[0067] In an optional embodiment, the backbone network can also be constructed by introducing an attention mechanism to optimize the feature extraction process, adding a spatial attention module or a channel attention module after the convolution layer to dynamically adjust the importance weights of different regions or channels in the feature map; and through a multi-scale feature fusion module, weighted fusion of features of different receptive fields.

[0068] In another alternative implementation, the backbone network can be constructed by replacing traditional convolutional neural networks with a Transformer architecture. This architecture leverages the self-attention mechanism to capture global contextual information and incorporates a multi-head attention module to model long-range dependencies within the target image. Furthermore, positional encoding introduces spatial structural information, ultimately generating deep features of the target image by stacking multiple Transformer encoder layers.

[0069] Extracting the category semantic features of the target image and generating a semantic feature vector of the category semantic features includes the following steps B1-B3:

[0070] B1: Build a model that includes a text encoder and an image encoder;

[0071] B2: Construct text-image pair data and use it to train the model that includes the text encoder and image encoder;

[0072] B3: Input the class names of known classes in the dataset into the trained model containing text encoder and image encoder to extract semantic text features of category semantic features.

[0073] Specifically, in B1, the model including a text encoder and an image encoder includes a text encoder and an image encoder.

[0074] In B2, for each training batch, a dataset containing N text-image pairs is constructed. The model containing the text encoder and the image encoder is trained using the text-image pair data, including the following steps B21-B25:

[0075] B21: Input N text-image pair data into the model containing the text encoder and the image encoder to obtain N text features and N image features.

[0076] B22: Construct positive and negative sample labels for text-image pairs using N text features and N image features. That is, N sets of text and images that truly belong to a pair are positive samples, and N 2 -N non-identical text-image pairs are negative samples.

[0077] B23: Combine N text features and N image features two by two, and predict the similarity of N 2 possible text-image pair data by calculating the cosine similarity of text features and image features.

[0078] B24: Calculate the similarity loss according to the similarity of the text-image pair data and the positive and negative sample labels of the text-image pair data, and the loss function is:

[0079]

[0080] where I i represents the image feature, T j represents the text feature, and N represents the total number of text-image pair data.

[0081] It should be noted that the above loss function effectively aligns the text features and image features in the semantic space by minimizing the sum of the similarities of the unpaired text-image pairs minus the sum of the similarities of the paired text-image pairs. This contrast learning method enables the model to learn more rich semantic information, laying a solid foundation for subsequent open set object detection. In practical applications, this loss function design enables the model to better understand the semantic association between different modalities and improve the generalization ability for unknown classes.

[0082] B25: According to the loss function obtained in step B24, train the model containing the text encoder and the image encoder by minimizing it.

[0083] In an optional embodiment, the way to generate the semantic feature vector of the class semantic feature can also be through adding a multi-head self-attention module to dynamically adjust the importance weight of different words in the text feature representation, that is, after tokenizing the input class name, the relevance of each word to the overall class is calculated using the attention mechanism.

[0084] In another optional embodiment, the way to generate the semantic feature vector of the class semantic feature can also use an external knowledge graph. For the class name of each known class, the related superordinate and subordinate concepts, synonyms or attributes are retrieved from the knowledge graph, and these additional semantic information is concatenated with the original class name and input into the model containing the text encoder and the image encoder.

[0085] Determine the candidate box required for object detection, including the following steps C1-C4:

[0086] C1: According to the image feature of the target picture, preset k anchor points;

[0087] C2: Classify and regress the k anchor points through a convolutional layer to obtain the classification result of the anchor points and the coordinate offset of the anchor points;

[0088] C3: Generate the candidate box required for target detection based on the classification results of the anchor points and the coordinate offsets of the anchor points;

[0089] C4: The features of the candidate boxes are processed into feature maps of the same size through the pooling layer.

[0090] Specifically, in step C1, an image with an original size of M*N is used to obtain image features of a target image with a size of (M / 16)*(N / 16) according to step S1. For each feature point, k anchor points are preset.

[0091] In step C2, the k anchor points are classified through a convolutional layer to determine whether the anchor points represent the target or the background, and then the k anchor points are regressed through a convolutional layer to generate the coordinate offset of each k anchor point.

[0092] In step C3, the candidate box required for target detection is obtained based on the classification results of the k anchor points and the coordinate offsets of the k anchor points generated in step C2.

[0093] In step C4, based on the image features of the candidate box and the target image obtained in step 1, the features of the candidate box are processed into feature maps of the same size through a pooling layer.

[0094] In an optional embodiment, the cluster feature vector generated based on the candidate boxes can also be generated by adding a spatial attention module or a channel attention module in step C2 to dynamically adjust the importance weights of different regions or channels in the k anchor feature maps. For example, by calculating the correlation between each k anchor point and the global feature, the feature capture capability of the key areas can be enhanced.

[0095] In another optional implementation, after generating cluster feature vectors based on candidate boxes, a context information fusion module can be introduced after step C4 to supplement the features of the candidate boxes using semantic information from the candidate box's neighboring regions. For example, a self-attention mechanism can be used to capture long-range dependencies between the candidate box and other regions, further enriching the feature representation of the candidate boxes.

[0096] Based on the candidate boxes, a clustering feature vector is generated through a multi-layer perceptron. The multi-layer perceptron consists of a two-layer fully connected network.

[0097] Calculating semantic clustering loss includes the following steps D1-D3:

[0098] D1: Extract semantic features of known classes;

[0099] D2: Get the clustering features of the candidate box;

[0100] D3: Calculate the similarity between the semantic feature vector of the class semantic feature and the clustering feature vector of the candidate box by the cosine similarity formula.

[0101] Specifically, in steps D1-D2, the semantic features {T1, T2, T3,..., T k} of the k known classes in the training data set are extracted, and the clustering features {F1, F2, F3,..., F m} of the m candidate boxes are obtained according to the foregoing steps.

[0102] In step D3, the semantic feature vector of the class semantic feature and the clustering feature of the candidate box are calculated by the cosine similarity formula:

[0103]

[0104] The cosine similarity between each clustering feature vector and the semantic feature vector of the class semantic feature is calculated, where F i and T j represent the semantic feature vectors of the clustering feature of the candidate box and the class semantic feature, respectively.

[0105] In step D4, according to the cosine similarity between the semantic feature vector of the class semantic feature and the clustering feature of the candidate box obtained in step D3, the formula is used according to the real matching condition:

[0106]

[0107] The semantic clustering loss is calculated, where r ij is the one-hot encoding of the semantic feature vector of the class semantic feature and the real matching condition of the clustering feature of the candidate box, and n represents the total number of all possible matching conditions.

[0108] It should be noted that the semantic clustering loss formula calculates the cosine similarity between the clustering feature of the candidate box and the semantic feature vector of the class semantic feature, and combines the softmax operation to convert the similarity into a probability distribution, and finally calculates the loss by cross-entropy. This design enables the model to form clear clustering boundaries in the semantic space, effectively distinguishing targets that are similar in semantics but belong to different classes. In practical applications, this loss function significantly improves the model's ability to recognize unknown classes that are similar in semantics, making the open set target detection more accurate and robust in complex scenarios.

[0109] Based on the clustering feature vector, the class decorrelation loss is calculated, including the following steps E1-E4:

[0110] E1: According to the clustering feature vector of the candidate box, a subset of class features is collected;

[0111] E2: According to the subset of class features, a class unit matrix is constructed;

[0112] E3: Use the cosine similarity formula to calculate the matching correlation of the cluster feature vector of the candidate box;

[0113] E4: Use the loss function to calculate the class identity matrix and the class decorrelation loss of the matching correlation.

[0114] Specifically, in step E1, the cluster feature vectors {F1, F2, F3, ..., F m}. Downsample the cluster feature vector of the candidate box, collect a cluster feature for each unique category, and obtain a subset of s features {F 1′ ,F 2′ ,F 3′ ,...,F s′}, where s≤k.

[0115] In step E2, a category identity matrix is ​​constructed according to the s categories corresponding to the subset of s features.

[0116] In step E3, the formula is applied to the subset of s features:

[0117]

[0118] Calculate the cosine similarity, where F i′ With F j′ is a subset of the cluster feature vector.

[0119] In step E3, based on the cosine similarity, use the formula:

[0120]

[0121] Calculate the matching correlation of the cluster feature vector of the candidate box, where sim i,l is the cosine similarity, and k represents the number of known categories.

[0122] In step E4, the loss function is used based on the matching correlation matrix of the cluster feature vector of the candidate box and the category unit matrix:

[0123]

[0124] Calculate the class decorrelation loss, where I i,j Represents the elements of the class identity matrix, Corr i,j Represents the matching correlation of the cluster feature vector of the candidate box.

[0125] It should be noted that the calculation of the class decorrelation loss is achieved by modeling the similarity between the subsets of clustered feature vectors and introducing an identity matrix as the target, which forces the features of different classes to be orthogonal to each other in the semantic space. This design effectively increases the inter-class clustering distance and solves the class confusion problem caused by semantic similarity in traditional methods. In practical applications, the class decorrelation loss greatly improves the model's ability to distinguish unknown classes, so that even in the case of high semantic overlap, the model can still accurately distinguish known classes from unknown classes. This is particularly important for security monitoring, autonomous driving and other scenarios that require high-precision target detection.

[0126] According to the candidate frame, the target detection result is obtained, and the classification loss and the regression loss are calculated, including the following steps F1-F6:

[0127] F1: According to the candidate frame, the regression result is obtained through a multilayer perceptron, and the regression loss is calculated.

[0128] F2: According to the candidate frame, the classification scores {s1, s2, s3,..., sk} of all classes of each candidate frame are obtained through a multilayer perceptron. k}。

[0129] F3: According to the classification scores, the classification probability p is obtained using the softmax function.

[0130] F4: The cross-entropy loss can be calculated according to the classification probability:

[0131]

[0132] Where C represents a set of k classes, c * represents the real class of the candidate frame, and y is the one-hot encoding corresponding to the real class of the candidate frame.

[0133] F5: According to the classification scores, the conditional probability of the unknown class is defined as:

[0134]

[0135] Where s u is the classification score of the k+1 class, i.e., the classification score of the unknown class.

[0136] F6: According to the conditional probability of the unknown class, the cross-entropy loss of the unknown class is calculated:

[0137]

[0138] Where p c is the classification probability of class c, y c is the one-hot encoding corresponding to the real class of the candidate frame, and u represents the unknown class.

[0139] It should be noted that traditional target detection methods typically focus solely on the classification accuracy of known classes. However, this invention explicitly models the conditional probabilities of unknown classes, enabling the model to proactively learn and identify them. Specifically, the conditional probabilities of unknown classes are calculated using a softmax operation after excluding the true class. This design enables the model to learn the differences between unknown and known classes during the training phase. In practical applications, this mechanism significantly improves the model's sensitivity to unknown classes, reduces the risk of misclassifying unknown classes as known classes, and provides more reliable detection results for safety-critical applications.

[0140] Example 3 is an application example of the present invention. This example provides a method for detecting safety and supervision targets for electric power operations based on semantic clustering. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through experiments.

[0141] To address the needs of identifying safety equipment and detecting unknown risk objects in power grid operation scenarios, a semantic clustering module and a class decorrelation module were designed. To verify the effectiveness of this method in power grid scenarios, experiments were conducted using a self-developed grid safety inspection dataset, GridSafe. This dataset contains annotated images from typical scenarios such as substation inspections, aerial work, and live operations. It covers eight known types of protective equipment, such as hard hats, insulating gloves, and goggles, as well as 20 potential risk objects, such as wire entanglement, insulator damage, and equipment sparks, as unknown categories.

[0142] The experimental dataset is set up as follows: the 8 types of protective equipment in the power grid dataset are used as the training set for closed-set target detection, and the 20 types of risk targets are divided into three groups (including 5 types, 10 types, and 20 types) according to their complexity as power operation safety supervision test sets of different difficulty levels.

[0143] The proposed model configuration is as follows: 1. The backbone network uses 13 convolutional layers; 2. The multilayer perceptron uses a 2-layer fully connected network; 3. The CLIP model consists of a text encoder and an image encoder; The training details of the model are as follows: During the training phase, the embodiment of the present invention uses the AdamW optimizer with a learning rate of 1×10 -4 , and performed 50,000 iterations. The training process of the embodiment of the present invention involves using a single GPU and a batch size of 6.

[0144] This paper uses multiple indicators to measure and compare the performance of the model in many aspects:

[0145] First use the basic mAP k ,AP u The experimental results are shown in Table 1. The method proposed in this paper compares the AP of OpenDet in all experimental settings. uImproved mAP by 1-3% k It also comprehensively outperforms other comparison methods.

[0146] Table 1

[0147]

[0148] In addition, this paper uses the Wilderness Impact (WI) indicator and Absolute Open-Set Error (AOSE) to measure the prediction effect of the model on unknown classes. WI is used to measure the degree of confusion when the model predicts unknown instances as known categories. The formula is as follows:

[0149]

[0150] It should be noted that the Wilderness Impact (WI) indicator is an important metric for evaluating the performance of open set target detection. It quantifies the degree of confusion of the model by calculating the ratio of the probability that the model predicts all detection samples (including known classes and unknown classes) as known classes to the probability that only known class samples are predicted as known classes, minus 1. The lower the WI value, the stronger the model's ability to distinguish unknown classes. AOSE measures the total number of unknown instances detected as known classes, that is, the absolute number of unknown targets misclassified into known categories. The experimental results are shown in Table 2. Compared with OpenDet, the method proposed in this invention effectively reduces the AOSE of the detection results and obtains an 11-18% gain in the WI indicator, indicating that the false detection rate of unknown risk targets is significantly reduced.

[0151] Compared to current mainstream open-source detection algorithms (such as Faster R-CNN, DS, and OpenDet), this method demonstrates significant advantages under complex grid conditions: In scenarios with strong and fluctuating illumination, it achieves an 89.7% recall rate for detecting broken insulators; and under conditions of multiple obstructions, it maintains a false detection rate of less than 2.1% for conductor entanglement. Experimental results demonstrate the effectiveness of this method in grid safety monitoring—it accurately identifies the wearing status of protective equipment and dynamically discovers unlabeled risk targets, addressing the blind spot in traditional models' perception of "unknown hidden dangers."

[0152] Table 2

[0153]

[0154] In summary, compared with the most advanced open-set target detection algorithms, the semantic clustering-based electric power operation safety and supervision target detection algorithm proposed in this invention achieves better open-set target detection results, especially achieving a significant lead in unknown class target detection. This performance improvement is due to the effective combination of the semantic clustering module and the class decorrelation module proposed in this invention. The former establishes a clear semantic decision boundary through clustering in the semantic space, and the latter increases the inter-class clustering distance through feature decorrelation. Together, they significantly improve the separation effect of unknown classes.

[0155] Example 4 is the fourth embodiment of the present invention, which differs from the first three embodiments in that:

[0156] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0157] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0158] More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering, or processing in another suitable manner as necessary, and then stored in a computer memory.

[0159] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0160] Example 5 provides a system for detecting power operation safety targets based on semantic clustering, including an image feature extraction module for extracting image features of a target image, and determining candidate boxes required for target detection using a region generation network based on the image features, while also unifying the sizes of the candidate boxes.

[0161] A category semantic feature extraction module is used to extract the category semantic features of the target image and generate a semantic feature vector of the category through a text encoder;

[0162] A cluster feature generation module, configured to generate a cluster feature vector based on the candidate box;

[0163] A class decorrelation loss calculation module, configured to calculate the class decorrelation loss based on the clustering feature vector;

[0164] A semantic clustering loss calculation module, configured to input the semantic feature vector and the clustering feature vector into a semantic clustering module to calculate the semantic clustering loss;

[0165] The target detection and loss calculation module is used to obtain the target detection result based on the candidate box and calculate the classification loss and regression loss.

[0166] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for detecting power operation safety and supervision targets based on semantic clustering, characterized by: include, Extracting image features of the target image, determining candidate frames required for target detection based on the image features, and unifying the sizes of the candidate frames; Extracting the category semantic features of the target image and generating a semantic feature vector of the category semantic features; Generate a cluster feature vector based on the candidate box; Calculating class decorrelation loss based on the cluster feature vector; Inputting the semantic feature vector and the clustering feature vector into a semantic clustering module to calculate the semantic clustering loss; Based on the candidate boxes, the target detection results are obtained, and the classification loss and regression loss are calculated.

2. The method for detecting power operation safety and supervision targets based on semantic clustering according to claim 1, characterized in that: Extract the image features of the target image, including: Build a backbone network; The target image is input into the backbone network to obtain the deep features of the target image.

3. The method for detecting power operation safety and supervision targets based on semantic clustering according to claim 2, characterized in that: Extracting the category semantic features of the target image includes: Build a model that includes a text encoder and an image encoder; Constructing text-image pair data, and using the text-image pair data to train the model including the text encoder and the image encoder; The class names of the known classes in the data set are input into the trained model including the text encoder and the image encoder to extract the semantic text features of the class semantic features.

4. The method for detecting power operation safety and supervision targets based on semantic clustering according to claim 3, characterized in that: Determine the candidate boxes required for target detection, including: Presetting anchor points according to image features of the target image; Classify and regress the anchor points to obtain classification results and coordinate offsets of the anchor points; Generating a candidate frame required for target detection according to the classification result of the anchor point and the coordinate offset of the anchor point; The features of the candidate boxes are processed into feature maps of the same size.

5. The method for detecting power operation safety and supervision targets based on semantic clustering according to claim 4, characterized in that: Calculate semantic clustering loss, including: Extract semantic features of known classes; Obtaining clustering features of the candidate box; Calculating the similarity between the cluster feature vector and the semantic feature vector of the category semantic feature; The semantic clustering loss is calculated according to the similarity.

6. The method for detecting power operation safety and supervision targets based on semantic clustering according to claim 5, characterized in that: Calculate class decorrelation loss, including: Collecting a category feature subset according to the cluster feature vector of the candidate box; constructing a category identity matrix based on the category feature subset; Calculating the matching correlation of the cluster feature vectors of the candidate boxes; A class decorrelation loss of the class identity matrix and the matching correlation is calculated.

7. The method for detecting power operation safety and supervision targets based on semantic clustering according to claim 6, characterized in that: The cosine similarity formula is used to calculate the similarity between the cluster feature vector and the semantic feature vector of the category semantic feature.

8. A system for detecting safety and supervision targets in electric power operations based on semantic clustering, applying the method for detecting safety and supervision targets in electric power operations based on semantic clustering according to any one of claims 1 to 7, characterized in that: include: An image feature extraction module is used to extract the image features of the target image and determine the candidate boxes required for target detection through a region generation network based on the image features, while unifying the sizes of the candidate boxes. A category semantic feature extraction module is used to extract the category semantic features of the target image and generate a semantic feature vector of the category through a text encoder; A cluster feature generation module, configured to generate a cluster feature vector based on the candidate box; A class decorrelation loss calculation module, configured to calculate the class decorrelation loss based on the clustering feature vector; A semantic clustering loss calculation module, configured to input the semantic feature vector and the clustering feature vector into a semantic clustering module to calculate the semantic clustering loss; The target detection and loss calculation module is used to obtain the target detection result based on the candidate box and calculate the classification loss and regression loss.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the processor implements the steps of a method for detecting safety and supervision targets of electric power operations based on semantic clustering according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of a method for detecting safety and supervision targets of electric power operations based on semantic clustering according to any one of claims 1 to 7 are implemented.

Images