Power transmission channel foreign matter-based identification method and device
Foreign object segmentation and feature extraction were performed through YOLOv7-seg and fine-tuned ConvNeXt model to generate a standard feature database, which solved the accuracy and robustness of differential foreign object recognition in high-class categories in transmission channels, and achieved fast and accurate foreign object recognition.
Patent Information
- Application Number
- CN202510647703.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-20
- Publication Date
- 2025-08-12
AI Technical Summary
The existing detection models cannot effectively deal with foreign object types that differ in high-class categories in transmission channels, resulting in insufficient recognition accuracy and robustness.
The pre-trained YOLOv7-seg image segmentation model is used for foreign object segmentation, and feature extraction is performed through the fine-tuned ConvNeXt model to generate a standard feature database, and a brute force search algorithm is used to match foreign object categories with cosine similarity.
It improves the accuracy and robustness of foreign object recognition, can quickly compare new features, has good scalability and traceability, reduces inference time, and enhances the reliability of the classification system.
Smart Images
Figure CN120472147A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of foreign body intrusion into power transmission channels, and in particular to a method and device for identifying foreign body intrusion into power transmission channels. Background Art
[0002] The power industry is increasingly facing challenges from various unexpected foreign object intrusions (FOIs), including drones, construction machinery, windblown debris, etc.; in order to cope with the increasing number of foreign object types, it is crucial to implement robust detection strategies; however, some foreign object types have high intra-class variation or visual similarity, such as dust screens, greenhouse films, and plastic packaging; this poses a huge challenge to conventional convolutional neural networks (CNNs), single-shot detectors (SSDs), and YOLO-based detection models; these models are designed to optimize target positioning and basic classification, but when the dataset contains categories with high intra-class variation, they may not be able to extract the most discriminative features of such targets, resulting in a decrease in recognition accuracy; in view of the above situation, it is necessary to improve the existing foreign object type location and extraction methods so that they can adapt to the current needs of foreign object extraction, thereby improving foreign object recognition accuracy. Summary of the Invention
[0003] The purpose of the present invention is to overcome the shortcomings of the existing technology and provide a method and device for identifying foreign objects in power transmission channels, so as to solve the technical problem that the existing technical solutions use a single detection model, which is unable to cope with categories with high intra-class differences, affecting the accuracy and robustness of the detection results.
[0004] To achieve the above object, the present invention is implemented by adopting the following technical solutions:
[0005] In a first aspect, the present invention provides a method for identifying foreign objects in a power transmission channel, comprising:
[0006] Acquire a scene image of the power transmission channel;
[0007] Using a pre-trained YOLOv7-seg image segmentation model to segment the scene image into foreign objects, and obtain images of each foreign object area;
[0008] Using a fine-tuned and pre-trained ConvNeXt model to extract features from each of the foreign body area images;
[0009] Comparing and matching the feature extraction result with a standard feature database to obtain the foreign body category;
[0010] The fine-tuning of the ConvNeXt model includes replacing its last classification layer with a fully connected layer of the required feature dimension, and the standard feature database is generated by the fine-tuned and pre-trained ConvNeXt model.
[0011] Optionally, the loss function used by the YOLOv7-seg image segmentation model during pre-training is:
[0012]
[0013] Where, is the total loss, are the classification loss and bounding box loss, is the pixel-level binary cross entropy loss and mask region loss, is the weight factor.
[0014] Optionally, the YOLOv7-seg image segmentation model aggregates similar foreign object categories during pre-training.
[0015] Optionally, the ConvNeXt model uses a triplet loss function during pre-training:
[0016]
[0017] Where, is the triplet loss, is a constant term, are the features of anchor samples, positive samples, and negative samples respectively;
[0018] When the distance between the anchor sample and the positive sample is less than the distance between the anchor sample and the negative sample by at least margin, the triplet loss To minimize:
[0019] .
[0020] Optionally, the comparing and matching the feature extraction result with a standard feature database to obtain the foreign body category includes:
[0021] A brute force search algorithm is used in combination with cosine similarity to find the standard feature that is most similar to the feature extraction result, and the foreign body category corresponding to the standard feature is used as the foreign body category of the feature extraction result.
[0022] In a second aspect, the present invention provides a device for identifying foreign matter in a power transmission channel, comprising:
[0023] An image acquisition module is configured to acquire a scene image of the power transmission channel;
[0024] A foreign body segmentation module is configured to use a pre-trained YOLOv7-seg image segmentation model to perform foreign body segmentation on the scene image to obtain images of various foreign body regions;
[0025] A feature extraction module is configured to extract features from each of the foreign body area images using a fine-tuned and pre-trained ConvNeXt model;
[0026] A matching and identification module is configured to compare the feature extraction result with a standard feature database to obtain a foreign body category;
[0027] The fine-tuning of the ConvNeXt model includes replacing its last classification layer with a fully connected layer of the required feature dimension, and the standard feature database is generated by the fine-tuned and pre-trained ConvNeXt model.
[0028] Optionally, the comparing and matching the feature extraction result with a standard feature database to obtain the foreign body category includes:
[0029] A brute force search algorithm is used in combination with cosine similarity to find the standard feature that is most similar to the feature extraction result, and the foreign body category corresponding to the standard feature is used as the foreign body category of the feature extraction result.
[0030] In a third aspect, the present invention provides an electronic device, including a processor and a storage medium;
[0031] The storage medium is used to store instructions;
[0032] The processor is configured to operate according to the instructions to execute the steps of the above method.
[0033] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the above method when executed by a processor.
[0034] In a fifth aspect, the present invention provides a computer program product, comprising a computer program / instruction, which implements the steps of the above method when executed by a processor.
[0035] Compared with the prior art, the present invention has the following beneficial effects:
[0036] The present invention provides a method and device for identifying foreign objects in power transmission channels. After fine-tuning and training the ConvNeXt model, a standard feature database is generated for comparison and classification of new feature inputs. A large and highly generalizable standard feature database can bring many unique advantages to image classification tasks: 1) Fast comparison: Digital features are quickly compared using simple distance metrics (such as Euclidean distance or cosine similarity); therefore, this feature comparison-based technique improves classification accuracy without significantly increasing inference time; 2) Strong generalization: Classification using feature datasets can be easily generalized to new, untrained data categories by simply adding new target category sample features generated by the same model to the database, thereby improving the scalability and robustness of the classification system; 3) Traceability: Each standard feature can be traced back to its source image. By comparing features with the input image, we can trace back to the source image closest to the input image, thereby achieving accurate fine-tuning of the standard database and denoising of data within the same category; all images used to generate standard features need to be pre-labeled, just like the data used to train the feature extraction model, so that the feature comparison results can match the correct category. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] Figure 1 1 is a flow chart of a method for identifying foreign matter in a power transmission channel according to an embodiment of the present invention;
[0038] Figure 2 Schematic diagram of the operating principle of the YOLOv7-seg image segmentation model provided by an embodiment of the present invention;
[0039] Figure 3 Schematic diagram of the fine-tuning structure of the ConvNeXt model provided by an embodiment of the present invention;
[0040] Figure 4 Schematic diagram of the data structure of data points in the standard feature database provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0041] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and are not intended to limit the scope of protection of the present invention.
[0042] Example 1:
[0043] like Figure 1 As shown, an embodiment of the present invention provides a method for identifying foreign objects in a power transmission channel, comprising the following steps:
[0044] Step S1: Acquire a scene image of a power transmission channel.
[0045] The scene images of the transmission channel can be collected by inspection drones or fixed cameras installed on the transmission channel.
[0046] Step S2: Use the pre-trained YOLOv7-seg image segmentation model to segment the scene image into foreign objects and obtain images of each foreign object area.
[0047] The YOLOv7-seg image segmentation model is a real-time instance segmentation model based on the YOLOv7 object detection framework. It combines the dual capabilities of object detection and pixel-level segmentation, making it suitable for high-precision segmentation tasks in complex scenarios. The YOLOv7-seg image segmentation model uses a three-segment network design: Backbone (backbone network) - Neck (feature fusion network) - Head (prediction head).
[0048] like Figure 2 As shown in Figure 1, the trained YOLOv7-seg image segmentation model found four object cropping regions AD from the original image, namely a dust net, a tower crane, an excavator and a cement mixer.
[0049] Specifically in this embodiment, the loss function used by the YOLOv7-seg image segmentation model during pre-training is:
[0050]
[0051] Where, is the total loss, are the classification loss and bounding box loss, is the pixel-level binary cross entropy loss and mask region loss, is the weight factor.
[0052] Total loss Find the gradient about the weight W to analyze how reducing the number of categories helps the YOLOv7-seg image segmentation model perform better in positioning and clipping boundary generation; gradient It consists of the inversion of each loss term to the model weight W:
[0053]
[0054] Gradients in the expanded classification task Before, we first give The formula is:
[0055]
[0056] Where, is the sample size, is the binary true label representation (0 or 1), is the predicted probability of category i; using the chain rule, the gradient of the classification loss with respect to the weight can be derived as:
[0057]
[0058] Considering the probability output of each category is the output of the Sigmoid activation function, that is ,and is the input of the activation function, which comes from the neurons in the previous layer, are the weights and biases of the neurons, is the input of the neuron; therefore, the gradient can be further expanded as:
[0059]
[0060] Expanding by the same chain rule and , it can be further proved that in the gradient function, the number of categories only affects Parameters in Therefore, in a dataset containing a constant number of instances, the fewer categories involved, the smaller the parameter space that needs to be considered during the loss optimization process; in other words, reducing the number of categories N can reduce the risk of overfitting and enable the model to allocate more optimization power to learning bounding box locations and segmentation features.
[0061] Based on the above theory, in this implementation, the YOLOv7-seg image segmentation model aggregates similar object categories during pre-training. For example, greenhouse film, dust screens, and streamers are grouped into one category, called "non-rigid objects." Tower cranes, lifting vehicles, cement mixers, excavators, bulldozers, and cement pumps are grouped into one category, called "construction machinery." The metal roofing sheet category remains unchanged and is renamed "rigid objects." Ultimately, the number of categories is reduced from 10 to 3.
[0062] Step S3: Use the fine-tuned and pre-trained ConvNeXt model to extract features from the images of each foreign object area.
[0063] ConvNeXt is a state-of-the-art feature extraction model optimized for medium-sized datasets. These extracted features are presented as numerical arrays, which can be abstractly understood as a low-dimensional collection of information elements generated by sampling and compressing the image through the CNN network. In this study, the dimension is 1x1024.
[0064] Based on the object mask cropping regions generated by the YOLOv7-seg image segmentation model, Vision Transformers (ViT), EfficientNet-L2, and ConvNeXt are the three most popular image classification model architectures. They are also widely used as backbone networks for feature extraction, primarily by removing the classification head of the model and adding fully connected layers at the end. Vision Transformers (ViT) uses the Transformer architecture suitable for large image datasets, EfficientNet is known for its high efficiency on small datasets and ease of deployment with limited computing resources, and ConvNeXt is a CNN-based model that modernizes traditional ConvNets by borrowing techniques from the Transformer, such as larger convolution kernels and LayerNorm. Table 1 compares the characteristics of these models in detail, including the required dataset size, computing resources, and inference speed.
[0065] Table 1 Performance comparison of main architectures of image feature extraction models
[0066]
[0067] ConvNeXt is more practical in most real-world applications, achieving the best balance between accuracy, speed, and training difficulty. In our case, the proposed framework is expected to be deployed on edge devices for real-time video inference, so ConvNeXt is the best choice because it provides good accuracy for medium-sized datasets and has fast inference speed under moderate computing resources.
[0068] The ConvNeXt model is trained using a triplet loss function. This function takes three sets of samples in each training iteration: anchors, positive samples, and negative samples. During training, the loss function pulls the features of similar samples closer together and pushes the embedded features of different classes further apart. This results in the feature data being clustered in geometric space based on mathematical distance similarity.
[0069] Triplet loss function:
[0070]
[0071] Where, is the triplet loss, is a constant term, are the features of anchor samples, positive samples, and negative samples respectively;
[0072] When the distance between the anchor sample and the positive sample is less than the distance between the anchor sample and the negative sample by at least margin, the triplet loss To minimize:
[0073] .
[0074] The distance metric includes Euclidean distance or cosine similarity. The feature dimension is selected as 1024, and cosine similarity is used as the distance metric.
[0075] In order to modify the ConvNeXt model to use the triplet loss function, such as Figure 3 As shown in
[15] , fine-tuning of a ConvNeXt model consists of replacing its last classification layer with a fully connected layer of the desired feature dimension.
[0076] When preparing the training dataset, the data needs to be labeled and grouped by category. Then, for each data (anchor data) a, a positive sample p is selected from the same category, and a negative sample n is selected from a different category to form a triplet [a, p, n]. We need to pre-calculate all possible triple combinations and ensure that the triplets are balanced between categories to avoid data imbalance. Unlike the cross-entropy loss function, the triplet loss function requires the model to perform three forward propagations on the anchor sample, positive sample, and negative sample in each iteration. The training process is as follows:
[0077] In each training iteration, the model processes the anchor input, positive input, and negative input, and obtains output features for each input.
[0078] The triplet loss function compares the features of anchor points, positive samples, and negative samples and uses the triplet loss Calculate the loss at each iteration.
[0079] The gradient of the loss function with respect to the weights is calculated and backpropagated via the chain rule to update the model weights.
[0080] After the ConvNeXt feature extraction model is trained, it is used to generate a standard feature database. This standard feature database is then used to compare and classify new feature inputs.
[0081] All images used to generate standard features need to be pre-labeled like the data for training the feature extraction model so that the feature comparison results can match the correct category. Each data point in the dataset has Figure 4 The data structure shown in Figure 1 is as follows, where the index refers to the unique ID of the data, the feature (embedding) is the numerical feature extracted from each image, and the path refers to the storage path of the source image that generates the feature.
[0082] Step S4: Compare and match the feature extraction result with the standard feature database to obtain the foreign body category.
[0083] The comparison matching algorithm is another important factor affecting the performance of feature-based classification. The choice of algorithm will vary depending on specific requirements, such as the size of the dataset, the dimensionality of the numerical features, and the need for speed and accuracy. For small or medium-sized datasets, brute force search and k-nearest neighbor (KNN) are the best choices. These algorithms can provide accurate and interpretable results, but the search speed will slow down significantly as the dataset increases. For larger datasets, approximate nearest neighbor (ANN) and clustering-based search algorithms can provide fast search speeds at the expense of a small amount of accuracy. The brute force search algorithm is chosen in combination with cosine similarity to provide the best performance balance between speed and accuracy. The cosine similarity measures the cosine of the angle between two vectors and is commonly used for text or high-dimensional data.
[0084] By using brute force search and cosine similarity comparison, the query data Q is calculated with each data point in the database D. The distance between , and select the category of the data with the smallest distance, that is, use the brute force search algorithm combined with cosine similarity to find the standard feature that is most similar to the feature extraction result, and use the standard feature corresponding to the foreign body category as the foreign body category of the feature extraction result.
[0085] In the foreign object tracking task, we can collect n (n>1) images of the object along its moving trajectory , then by The majority of the categories in the classification are used to determine the final classification prediction results.
[0086] Example 2:
[0087] An embodiment of the present invention provides a device for identifying foreign objects in a power transmission channel, comprising:
[0088] An image acquisition module is configured to acquire a scene image of the power transmission channel;
[0089] The foreign body segmentation module is configured to use the pre-trained YOLOv7-seg image segmentation model to segment the scene image into foreign bodies and obtain images of each foreign body area;
[0090] A feature extraction module is configured to extract features from each foreign object region image using a fine-tuned and pre-trained ConvNeXt model;
[0091] The matching recognition module is configured to compare the feature extraction results with the standard feature database to obtain the foreign body category. Specifically, the module uses a brute force search algorithm combined with cosine similarity to find the standard feature that is most similar to the feature extraction result, and uses the foreign body category corresponding to the standard feature as the foreign body category of the feature extraction result.
[0092] Among them, fine-tuning of the ConvNeXt model involves replacing its last classification layer with a fully connected layer of the required feature dimension.
[0093] Example 3:
[0094] Based on the identification method provided in the first embodiment, the embodiment of the present invention provides an electronic device, including a processor and a storage medium;
[0095] The storage medium is used to store instructions;
[0096] The processor is configured to operate according to the instructions to execute the steps of the above method.
[0097] Example 4:
[0098] Based on the identification method provided in the first embodiment, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the above method when executed by a processor.
[0099] Embodiment 5:
[0100] Based on the identification method provided in the first embodiment, an embodiment of the present invention provides a computer program product, including a computer program / instruction, which implements the steps of the above method when executed by a processor.
[0101] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0102] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0103] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.
[0104] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.
[0105] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.
Claims
1. A method for identifying foreign matter in a power transmission channel, characterized in that: include: Acquire a scene image of the power transmission channel; Using a pre-trained YOLOv7-seg image segmentation model to segment the scene image into foreign objects, and obtain images of each foreign object area; Using a fine-tuned and pre-trained ConvNeXt model to extract features from each of the foreign body area images; Comparing and matching the feature extraction result with a standard feature database to obtain the foreign body category; The fine-tuning of the ConvNeXt model includes replacing its last classification layer with a fully connected layer of the required feature dimension, and the standard feature database is generated by the fine-tuned and pre-trained ConvNeXt model.
2. The method for identifying foreign matter in a power transmission channel according to claim 1, characterized in that: The loss function used by the YOLOv7-seg image segmentation model during pre-training is: Where, is the total loss, are the classification loss and bounding box loss, is the pixel-level binary cross entropy loss and mask region loss, is the weight factor.
3. The method for identifying foreign matter in a power transmission channel according to claim 1, characterized in that: The YOLOv7-seg image segmentation model aggregates similar foreign object categories during pre-training.
4. The method for identifying foreign matter in a power transmission channel according to claim 1, characterized in that: The ConvNeXt model uses a triplet loss function during pre-training: Where, is the triplet loss, is a constant term, are the features of anchor samples, positive samples, and negative samples respectively; When the distance between the anchor sample and the positive sample is less than the distance between the anchor sample and the negative sample by at least margin, the triplet loss To minimize: 。 5. The method for identifying foreign matter in a power transmission channel according to claim 1, characterized in that: The comparing and matching the feature extraction result with the standard feature database to obtain the foreign body category includes: A brute force search algorithm is used in combination with cosine similarity to find the standard feature that is most similar to the feature extraction result, and the foreign body category corresponding to the standard feature is used as the foreign body category of the feature extraction result.
6. A device for identifying foreign matter in a power transmission channel, characterized in that: include: An image acquisition module is configured to acquire a scene image of the power transmission channel; A foreign body segmentation module is configured to use a pre-trained YOLOv7-seg image segmentation model to perform foreign body segmentation on the scene image to obtain images of various foreign body regions; A feature extraction module is configured to extract features from each of the foreign body area images using a fine-tuned and pre-trained ConvNeXt model; A matching and identification module is configured to compare the feature extraction result with a standard feature database to obtain a foreign body category; The fine-tuning of the ConvNeXt model includes replacing its last classification layer with a fully connected layer of the required feature dimension, and the standard feature database is generated by the fine-tuned and pre-trained ConvNeXt model.
7. The device for identifying foreign matter in power transmission channels according to claim 6, characterized in that: The comparing and matching the feature extraction result with the standard feature database to obtain the foreign body category includes: A brute force search algorithm is used in combination with cosine similarity to find the standard feature that is most similar to the feature extraction result, and the foreign body category corresponding to the standard feature is used as the foreign body category of the feature extraction result.
8. An electronic device, characterized in that: including processors and storage media; The storage medium is used to store instructions; The processor is configured to operate according to the instructions to execute the steps of the method according to any one of claims 1 to 5.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.
10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.