Hardware fitting detection model construction method, hardware fitting identification method, hardware fitting detection model construction equipment, hardware fitting identification equipment and storage medium

Abstract

The invention discloses a fitting detection model construction method and device, a fitting identification method and device and a storage medium, and the method comprises the steps: collecting image data of an overhead transmission line, and marking a sample detection frame where a fitting on the overhead transmission line is located and a sample type representing the corrosion degree of the fitting in the sample detection frame for the image data; inputting the image data into a backbone network, and extracting a plurality of backbone image features under self-attention; inputting the plurality of trunk image features into a neck network, and performing feature fusion in an independent channel to obtain a plurality of fused image features; inputting the plurality of fused image features into a head network, mapping the fused image features into a target detection frame on the image data and a target category representing the corrosion degree of the hardware in the target detection frame, and calculating a total loss value according to the target detection frame, the target category, the sample detection frame and the sample category; and the hardware fitting detection model is updated according to the total loss value, so that the accuracy and efficiency of identifying the hardware fitting by the hardware fitting detection model are improved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to hardware detection model construction, hardware identification method, equipment and storage medium. Background Art

[0002] With the expansion of the scale of power grids and the increase in the number of power equipment, the safety maintenance of power grids faces higher requirements. As one of the important infrastructures of the power grid, the safe and stable operation of overhead transmission lines is one of the important guarantees for power transmission. Hardware, as a widely used metal accessory on transmission lines, is responsible for supporting, fixing and connecting bare wires and other conductors. Since transmission lines are exposed to harsh outdoor environments for a long time, some hardware is usually susceptible to rust, damage or even falling off. Therefore, the detection of hardware is one of the factors that affect the stable operation of the power grid.

[0003] At present, the traditional hardware inspection method mainly relies on manual inspection. Workers manually take images of hardware of overhead power transmission lines through high-altitude operations or with the help of drones, inspection vehicles and other equipment, and manually analyze and evaluate the hardware and corrosion degree of the hardware in the hardware images. Although this method can complete the hardware inspection task to a certain extent, manual inspection is easily affected by environmental interference and subjective judgment of workers. Manual judgment may have large errors, resulting in low accuracy and efficiency of hardware inspection. Summary of the invention

[0004] The present invention provides a hardware detection model construction, a hardware identification method, a device and a storage medium, which are used to improve the accuracy and efficiency of hardware detection.

[0005] In a first aspect, an embodiment of the present invention provides a method for constructing a hardware detection model, wherein the hardware detection model has a backbone network, a neck network, and a head network, and the method includes:

[0006] Collecting image data of an overhead power transmission line, and marking the image data with a sample detection frame where hardware on the overhead power transmission line is located and a sample category representing the degree of corrosion of the hardware in the sample detection frame;

[0007] Inputting the image data into the backbone network, and extracting backbone image features of multiple scales under self-attention;

[0008] Inputting the trunk image features of multiple scales into the neck network, performing feature fusion in independent channels, and obtaining fused image features of multiple scales;

[0009] Input the fused image features at multiple scales into the head network and map them into a target detection frame on the image data and a target category representing the degree of corrosion of the metal fittings in the target detection frame;

[0010] If the hardware detection model is in the training stage, a total loss value is calculated according to the target detection frame, the target category, the sample detection frame and the sample category;

[0011] Updating the hardware detection model according to the total loss value;

[0012] If the update is completed, it is determined whether the hardware detection model meets the preset training conditions; if so, it is determined that the hardware detection model training is completed; if not, it returns to execute the input of the trunk image features of multiple scales into the neck network, and performs feature fusion in independent channels to obtain fused image features of multiple scales.

[0013] In a second aspect, an embodiment of the present invention further provides a hardware identification method, the method comprising:

[0014] Collect image data of overhead power transmission lines;

[0015] Loading a hardware detection model trained according to the method provided in the first aspect; the hardware detection model includes a backbone network, a neck network and a head network;

[0016] Inputting the image data into the backbone network, and extracting backbone image features of multiple scales under self-attention;

[0017] Inputting the trunk image features of multiple scales into the neck network, performing feature fusion in independent channels, and obtaining fused image features of multiple scales;

[0018] The fused image features at multiple scales are input into the head network and mapped into a target detection frame where the hardware on the overhead transmission line on the image data is located and a target category representing the degree of corrosion of the hardware in the target detection frame.

[0019] In a third aspect, an embodiment of the present invention further provides a device for constructing a hardware detection model, wherein the hardware detection model has a backbone network, a neck network, and a head network, and the device includes:

[0020] An image data acquisition module is used to collect image data of overhead power transmission lines, and to mark the image data with a sample detection frame where hardware on the overhead power transmission lines are located and a sample category representing the degree of corrosion of the hardware in the sample detection frame;

[0021] A backbone network training module, used for inputting the image data into the backbone network and extracting backbone image features of multiple scales under self-attention;

[0022] A neck network training module, used for inputting the trunk image features of multiple scales into the neck network, performing feature fusion in independent channels, and obtaining fused image features of multiple scales;

[0023] A head network training module, used for inputting the fused image features of multiple scales into the head network and mapping them into a target detection frame on the image data and a target category representing the degree of corrosion of the metal fittings in the target detection frame;

[0024] A total loss value calculation module, used for calculating the total loss value according to the target detection frame, the target category, the sample detection frame and the sample category if the hardware detection model is in the training stage;

[0025] A total loss value updating module, used for updating the hardware detection model according to the total loss value;

[0026] A condition judgment module, used to judge whether the hardware detection model meets the preset training conditions if the update is completed; if so, execute the model training completion module; if not, return to execute the neck network training module;

[0027] The model training completion module is used to determine whether the hardware detection model training is completed.

[0028] In a fourth aspect, an embodiment of the present invention further provides a hardware identification device, the device comprising:

[0029] A data acquisition module, used for acquiring image data of overhead power transmission lines;

[0030] A model loading module, used to load a hardware detection model trained according to the method provided in the first aspect; the hardware detection model includes a backbone network, a neck network and a head network;

[0031] A backbone image feature extraction module, used to input the image data into the backbone network and extract backbone image features of multiple scales under self-attention;

[0032] A fused image feature extraction module, used for inputting the trunk image features of multiple scales into the neck network, performing feature fusion in independent channels, and obtaining fused image features of multiple scales;

[0033] An image data mapping module is used to input the fused image features of multiple scales into the head network and map them into a target detection frame where the hardware on the overhead transmission line is located on the image data and a target category representing the degree of corrosion of the hardware in the target detection frame.

[0034] In a fifth aspect, an embodiment of the present invention further provides a computer device, the computer device comprising:

[0035] one or more processors;

[0036] A storage device for storing one or more programs;

[0037] When the one or more programs are executed by the one or more processors, the one or more processors implement the hardware detection model construction method provided by the first aspect of the present invention, or the hardware identification method provided by the second aspect of the present invention.

[0038] In a sixth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a hardware detection model building method as provided in the first aspect of the present invention, or a hardware identification method as provided in the second aspect of the present invention.

[0039] In the seventh aspect, an embodiment of the present invention further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the hardware detection model construction method provided in the first aspect of the present invention, or the hardware identification method provided in the second aspect of the present invention.

[0040] In an embodiment of the present invention, a hardware detection model has a backbone network, a neck network and a head network, collects image data of overhead transmission lines, and annotates the image data with a sample detection frame where the hardware on the overhead transmission line is located and a sample category representing the degree of corrosion of the hardware in the sample detection frame; inputs the image data into the backbone network, extracts backbone image features of multiple scales under self-attention; inputs the backbone image features of multiple scales into the neck network, performs feature fusion in an independent channel, and obtains fused image features of multiple scales; inputs the fused image features of multiple scales into the head network and maps them into a target detection frame on the image data and a target category representing the degree of corrosion of the hardware in the target detection frame; if the hardware detection model is in the training stage, the total loss value is calculated based on the target detection frame, the target category, the sample detection frame and the sample category; the hardware detection model is updated based on the total loss value; if the update is completed, it is determined whether the hardware detection model meets the preset training conditions; if so, it is determined that the training of the hardware detection model is completed; if not, it returns to execute the input of the backbone image features of multiple scales into the neck network, performs feature fusion in an independent channel, and obtains fused image features of multiple scales. Through multi-scale feature extraction and fusion, combined with the self-attention mechanism and independent channel feature fusion, it is possible to effectively extract and fuse image features of multiple scales from overhead transmission line images, accurately locate the position of hardware and evaluate its corrosion degree. By calculating the total loss value and dynamically updating the hardware detection model during the training phase, the hardware detection model can be gradually optimized to improve the accuracy and efficiency of hardware identification. On the other hand, the image data collected from the overhead transmission line can be input into the trained hardware detection model, and the target detection frame on the image data and the target category representing the degree of corrosion of the hardware in the target detection frame can be output. Based on the target detection frame, it is determined whether there is hardware in the image data and the degree of corrosion of the hardware, thereby improving the accuracy and efficiency of hardware detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 is a flow chart of a method for constructing a hardware detection model provided in accordance with the first embodiment of the present invention;

[0042] Figure 2 This is a process framework diagram of a hardware detection model network provided according to the first embodiment of the present invention;

[0043] Figure 3 This is a diagram showing an example of a parallel self-attention structure provided according to Embodiment 1 of the present invention;

[0044] Figure 4 is a structural example diagram of a compact inverted block structure provided according to Embodiment 1 of the present invention;

[0045] Figure 5 is a flow chart of a method for constructing a hardware detection model provided in accordance with Embodiment 2 of the present invention;

[0046] Figure 6 is a flow chart of a hardware identification method provided according to Embodiment 3 of the present invention;

[0047] Figure 7 This is an example diagram of a hardware identification result provided according to Embodiment 3 of the present invention;

[0048] Figure 8 It is a structural schematic diagram of a hardware detection model building device provided according to a fourth embodiment of the present invention;

[0049] Fig. 9 is a structural schematic diagram of a hardware identification device provided according to Embodiment 5 of the present invention;

[0050] Fig.10 It is a structural diagram of a computer device provided according to Embodiment 6 of the present invention. DETAILED DESCRIPTION

[0051] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0052] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can cover sequential implementations other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0053] Embodiment 1

[0054] See also Figure 1 , shows a flowchart of a hardware detection model construction method provided by the first embodiment of the present invention, the method can be executed by a hardware detection model construction device, the hardware detection model construction device can be implemented in the form of hardware and / or software, and the hardware detection model construction device can be configured in a computer device. Figure 1 As shown, the method includes:

[0055] Step 101: collect image data of an overhead power transmission line, and mark the image data with a sample detection frame where hardware on the overhead power transmission line is located and a sample category representing the degree of corrosion of the hardware in the sample detection frame.

[0056] Overhead transmission lines: These refer to power transmission lines erected in the air, usually including wires, hardware and other components. Hardware refers to the metal connection parts in power lines, such as tie rods, wire fixtures, etc. Usually these hardware are prone to corrosion due to long-term use.

[0057] The sample detection box represents the area selected in the collected image data, which specifically marks the location of the hardware in the image data to help the hardware detection model know which locations need special attention.

[0058] The sample categories with marked corrosion degrees provide a basis for judging the health status of hardware. By classifying the corrosion degree, the hardware detection model can identify the different health states of hardware (such as no corrosion, slight corrosion, severe corrosion, etc.), thereby assisting subsequent maintenance.

[0059] In this embodiment, image data of overhead power transmission lines are collected by image acquisition equipment (such as high-definition cameras, drones, etc.), and the collected image data are annotated, especially the hardware (such as metal connectors) on the overhead power transmission lines in the image data are annotated with sample detection frames and sample categories of hardware corrosion degrees, so that the subsequent hardware detection model training can accurately identify and judge the corrosion status of hardware, thereby providing a basis for the maintenance of overhead power transmission lines.

[0060] For example, a commonly used data annotation tool (labelimg) is usually used to annotate sample detection frames for the hardware in the image data. The annotation steps include: installing Anaconda, creating a suitable python virtual environment, and installing the matching version of labelimg; opening the image data to be annotated in labelimg; based on the different types of hardware contained in the image data, the hardware is surrounded as accurately as possible under the premise of clarifying the annotation strategy (such as whether to annotate when overlapping or partially occluded), and annotating the rectangular frame. The annotated results are saved in YOLO format.

[0061] Step 102: input the image data into the backbone network, and extract backbone image features of multiple scales under self-attention.

[0062] In this embodiment, if Figure 2As shown in the figure, the hardware detection model includes the backbone network Backbone, the neck network Neck and the head network Head. The collected image data is input into the backbone network Backbone to adapt to the detection needs of hardware of different sizes through multi-scale feature extraction. At the same time, the self-attention mechanism is used to focus on the key areas in the image data, extract the backbone image features of multiple scales, and use the improved CSPDarknet53. By introducing the cross-stage partial network to optimize the feature fusion and gradient flow, the sensitivity to important details is improved, thereby improving the accuracy of hardware detection and corrosion assessment.

[0063] Exemplarily, the backbone network Backbone is not limited to the partial structure of extracting features in artificially designed neural networks (such as VGG, YOLO, ResNet, etc.), but can also be a structure optimized by a model quantization method, a structure searched for image characteristics of overhead transmission lines by a NAS (Neural Architecture Search) method, and so on. This embodiment does not impose any restrictions on this.

[0064] In one embodiment of the present invention, Figure 2 As shown in the figure, the backbone network Backbone includes 5 convolution modules CBS, 4 C2f modules, 1 spatial pyramid pooling fast module SPPF and 1 parallel self-attention module PSA.

[0065] The convolution module CBS includes a convolution layer Conv2d, an activation function SiLU, and a batch normalization processing layer BatchNorm2d. The convolution module CBS first uses the two-dimensional convolution layer Conv2d to mine local features; secondly, it uses the batch normalization processing layer BatchNorm2d to standardize the feature map; thirdly, it uses the SiLU activation function for nonlinear activation to extract local features of the input image data.

[0066] The C2f module includes convolution layer Conv, segmentation Split, bottleneck layer Bottleneck and concatenation Concat. In the C2f module, the input features are first divided into two parts. One part directly skips the subsequent convolution operation and is connected through residual connection; the other part is processed through a series of bottleneck layers Bottleneck to mine deeper features; the features that have undergone convolution operations and the features that directly skip convolution are fused in the final stage, which optimizes feature extraction and improves computational efficiency.

[0067] The spatial pyramid pooling fast module SPPF includes a convolution layer Conv, a maximum pooling layer MaxPool(3) and a concatenation Concat. The SPPF module divides the input features into two parts of features, and concatenates one part of the features that has been processed by three maximum pooling layers MaxPool(3) of different sizes with the other part of the features that have not been processed. The spatial pyramid pooling fast module (SPPF) is used to improve the ability of the hardware detection model to capture features of different scales, simplify the operation process and improve the calculation efficiency, and enhance the multi-scale perception ability of the hardware detection model.

[0068] The parallel self-attention module PSA includes convolution layer Conv, split Split, multi-head self-attention module MHSA, feedforward network FFN and concatenation Concat. PSA can extract fine-grained features more effectively, enhance the detection ability of small hardware or details, and reduce computational complexity and memory usage by dividing the input features into two parts and performing self-attention calculation on one part. This design takes advantage of the redundancy of attention heads and reduces unnecessary calculations. PSA combines multi-head self-attention (MHSA) and feedforward network (FFN), and optimizes the dimension allocation. The query and key dimensions are reduced to half, and BatchNorm is used instead of LayerNorm. The combination of multi-head self-attention and feedforward network can more effectively fuse information between different feature layers and improve the richness of feature expression. The parallel self-attention module PSA captures global context information of different parts in parallel through the self-attention mechanism, and uses convolution layer Conv and feedforward network FFN to improve the processing ability of local features. PSA can better integrate local and global information, thereby enhancing the hardware detection model's ability to model complex patterns and long-distance dependencies, especially improving the effects of feature fusion and multi-scale perception in visual tasks.

[0069] In one example, if Figure 2 As shown, in the backbone network Backbone:

[0070] The image data is sequentially input into the convolution module CBS, the convolution module CBS, the C2f module, the convolution module CBS and the C2f module for feature extraction to obtain the first-order image features.

[0071] The first-order image features are sequentially input into the convolution module CBS and the C2f module for feature extraction to obtain the second-order image features.

[0072] The second-order image features are sequentially input into the convolution module CBS, the C2f module, and the spatial pyramid pooling fast module SPPF for feature extraction to obtain the first candidate image features.

[0073] The first candidate image features are input into the parallel self-attention module PSA to extract the third-order image features under self-attention.

[0074] In one embodiment of the present invention, Figure 3 As shown, the parallel self-attention module PSA includes a multi-head self-attention module MHSA, a feedforward network FFN, a first initial convolution layer Conv and a first ending convolution layer Conv; inputting the first candidate image feature into the parallel self-attention module PSA, and extracting the third-order image feature under self-attention may specifically include the following 1-8:

[0075] 1. Input the first candidate image feature into the first initial convolutional layer Conv to perform a convolution operation to obtain the first convolution image feature.

[0076] In this embodiment, low-level features (such as edges, textures, etc.) of the first candidate image are extracted through a convolution operation to obtain a first convolution image feature. The convolution operation helps to capture local spatial features.

[0077] 2. Split the first convolution image feature into a first training image feature and a first non-training image feature.

[0078] In this embodiment, the first convolution image is divided into a first training image feature and a first non-training image feature. The first training image feature is input into the multi-head self-attention module and the feedforward network to extract deeper features, while the first non-training image feature is connected to the output results of the first training image feature in the multi-head self-attention module and the feedforward network through a residual connection, which helps to alleviate the gradient vanishing problem and improve the expressiveness of the hardware detection model.

[0079] 3. Input the first training image features into the multi-head self-attention module (MHSA) to calculate the total attention image features.

[0080] In this embodiment, if Figure 3As shown, the first training image feature is input into the multi-head self-attention module MHSA. The input first training image feature (usually a matrix, representing the representation of certain words or pixels) will first undergo a linear transformation (Linear). The linear transformation is performed through a fully connected layer (i.e., matrix multiplication with a weight matrix) to map the input first training image feature to a higher or lower dimensional space. Specifically, the input first training image feature matrix will be mapped to three vector spaces: query (Q), key (K), and value (V), and the query (Q), key (K), and value (V) will be input into the scaled dot-product attention (Scaled Dot-Product Attention) to calculate the similarity between the query and the key, and the values ​​will be weighted and summed according to the similarity, and the weighted summation results will be input into the concatenation Concat and the linear transformation Linear in turn to generate the final total attention image feature.

[0081] 4. Add the first training image feature to the total attention image feature to obtain the first merged image feature.

[0082] In this embodiment, the total attention image calculated in the multi-head self-attention module is residually connected with the first training image feature to obtain the first merged image feature. The main function of the residual connection is to help optimize the training process of the hardware detection model by directly transmitting the input signal, avoid gradient disappearance, and improve the training efficiency and performance of the hardware detection model.

[0083] 5. Input the first merged image features into the feed-forward network (FFN) for linear transformation to obtain transformed image features.

[0084] In this embodiment, the feed-forward network (FFN) generally further processes the first merged image features through linear transformation and activation function to increase nonlinear expression capability, which helps to extract higher-level semantic information from the first merged image features.

[0085] 6. Add the first merged image feature and the transformed image feature to obtain the second merged image feature.

[0086] In this embodiment, the transformed image features obtained by the input feedforward network are residually connected with the first merged features to obtain second merged image features, which can further improve information fusion and ensure that the hardware detection model can comprehensively consider the initial features (such as the first merged image features) and the new features after transformation (such as the transformed image features), thereby improving the quality of the final representation.

[0087] 7. Concatenate the second merged image feature and the first non-training image feature to obtain a third merged image feature.

[0088] In this embodiment, the processed second merged image features are spliced ​​with the first non-training image features to ensure that the hardware detection model can use more information for richer feature expression. The splicing operation helps to maintain the original information of the first non-training feature part while maintaining the third merged image features after the combination transformation.

[0089] 8. Input the third merged image features into the first end convolution layer (Conv) to perform a convolution operation to obtain the third-order image features.

[0090] In this embodiment, the third merged image feature is finally input into the first end convolution layer for processing to obtain the final third-order image feature, integrating all the image features generated by 1-7, and further abstracted through convolution to generate the final third-order image feature for use by the subsequent neck network Neck.

[0091] Step 103: Input the trunk image features of multiple scales into the neck network, perform feature fusion in independent channels, and obtain fused image features of multiple scales.

[0092] In this embodiment, the backbone image features of multiple scales extracted by the backbone network are input into the neck network, and feature fusion is performed in independent channels to extract and fuse the fused image features of multiple scales, thereby enhancing the perception ability of the hardware detection model at different sizes and different levels of detail. When processing complex image scenes, it is able to simultaneously focus on the fused image features of different scales, thereby improving the accuracy and robustness of hardware corrosion detection.

[0093] In one embodiment of the present invention, Figure 2 As shown, the neck network Neck includes 2 upsampling modules, 3 C2f modules, 2 convolution modules and 1 C2f module combined with a compact inverted block structure (C2fCIB);

[0094] The C2f module combined with a compact inverted block structure (C2fCIB) includes a convolutional layer Conv, a segmentation Split, a compact inverted block structure CIB, and a concatenation Concat.

[0095] The C2f module combined with a compact inverted block structure (C2fCIB) is based on C2f coupled with a compact inverted block structure CIB. Compared with the bottleneck layer Bottleneck of C2f, CIB uses streamlined deep convolution for spatial mixing, and each input channel is convolved with a separate convolution kernel. This means that each input channel extracts spatial features independently, and there is no mixing between channels. This method greatly reduces the amount of convolution calculations because each convolution kernel only acts on one channel. After deep convolution, feature fusion between channels is performed through point convolution. This enables CIB to significantly reduce computational complexity and memory usage while maintaining similar performance. Through spatial mixing and channel mixing, CIB can better extract and fuse features at different levels. This feature fusion strategy improves the expressiveness of the hardware detection model, especially when dealing with complex overhead transmission line scenarios and small hardware.

[0096] The neck network Neck fuses the multi-scale image features extracted by the backbone network Backbone to improve the detection capability of the hardware detection model for image features of each scale. The neck network Neck includes the feature pyramid network (FPN) and the path aggregation network (PAN), which fuse high-resolution detail features and low-resolution semantic features. Starting from the third-order image features, up-sample layer by layer, and use 1×1 convolution to adjust the number of channels so that the first up-sampled features and the second up-sampled features are consistent with the number of channels of the multi-scale image features, and are fused by adding layer by layer to supplement the spatial information and enhance the semantic information.

[0097] In this embodiment, if Figure 2 As shown, in the neck network Neck:

[0098] 1. In the left branch of the Neck network:

[0099] The third-order image features are input into the upsampling module Upsample for feature extraction to obtain the first upsampling image features.

[0100] The first up-sampled image feature is concatenated with the second-order image feature to obtain a second candidate fused image feature.

[0101] The second candidate fused image features are sequentially input into the C2f module and the upsample module Upsample for feature extraction to obtain the second upsampled image features.

[0102] The second up-sampled image feature is concatenated with the first-order image feature to obtain a first candidate fused image feature.

[0103] 2. In the right branch of the Neck network:

[0104] The first candidate fused image feature is input into the C2f module for feature transformation to obtain the first-order fused image feature.

[0105] The first-order fused image features are input into the convolution module CBS to perform a convolution operation to obtain the third candidate fused image features.

[0106] The third candidate fused image feature and the second candidate fused image feature are concatenated to obtain a first concatenated image feature.

[0107] The first spliced ​​image features are input into the C2f module for feature transformation to obtain the second-order fused image features.

[0108] The second-order fused image features are input into the convolution module CBS to perform a convolution operation to obtain a fourth candidate fused image feature.

[0109] The fourth candidate fused image feature is concatenated with the third-order image feature to obtain a second concatenated image feature.

[0110] The second spliced ​​image features are input into the C2f module C2fCIB combined with the compact inverted block structure, and feature fusion is performed in an independent channel to obtain the third-order fused image features.

[0111] In one embodiment of the present invention, Figure 4 As shown, the C2f module of the compact inverted block structure includes a second initial convolution layer Conv, a second end convolution layer Conv and a compact inverted block structure CIB; the compact inverted block structure CIB includes 5 convolution layers, of which 3 are deep convolution layers 3×3DW and 2 are ordinary convolution layers Conv. The second spliced ​​image feature input is combined with the C2f module of the compact inverted block structure, and feature fusion is performed in an independent channel to obtain the third-order fused image feature, which specifically includes the following steps:

[0112] Input the second spliced ​​image feature into the second initial convolution layer Conv to perform a convolution operation to obtain a second convolution image feature; divide the second convolution image feature (Split) into a second training image feature and a second non-training image feature; set the number of executions N of the compact inverted block structure; input the second training image feature into a compact inverted block structure (CIB) to perform a convolution operation until the number of executions N is met to obtain a plurality of convolution training image features of different scales; the convolution training image features of a plurality of different scales are the execution results of each compact inverted block structure, and the execution result of each compact inverted block is used as the input of the next compact inverted block structure; concatenate the plurality of convolution training image features of different scales with the second non-training image feature to obtain a third spliced ​​image feature; input the third spliced ​​image feature into the second end convolution layer Conv to perform a convolution operation to obtain a third-order fused image feature.

[0113] Step 104: Input the fused image features of multiple scales into the head network and map them into a target detection frame on the image data and a target category representing the degree of corrosion of the metal fittings in the target detection frame.

[0114] In this embodiment, fused image features of different scales can capture hardware of different sizes in the image data, especially in a complex overhead transmission line environment, where hardware of different scales may appear in the image data at different angles and sizes. By inputting fused image features of multiple scales, the head network can more accurately locate and classify hardware in the image data, and more comprehensively determine the specific location of the hardware in the image data (i.e., the target detection frame) and the degree of corrosion of the hardware (i.e., the target category).

[0115] The head network adopts an anchor-free design. Instead of using a fixed anchor box, it directly predicts the center point, width, height and category of the image data to locate and classify hardware of different scales. A decoupled head is used to separate the classification branch and the regression branch, and each output branch is further subdivided into two parts, one for classification and the other for regression bounding boxes. The classification branch uses global average pooling to classify each feature, and outputs the probability distribution of each target category by reducing the dimension of the feature; the regression branch uses the DFL (Distribution FocalLoss) strategy to treat the coordinate prediction as a distribution rather than a definite single value.

[0116] For example, Figure 2 As shown, in the head network Head:

[0117] A decoupling operation is performed on the first fused image features, the second fused image features and the third fused image features. The decoupling operation includes two branches (i.e., a classification branch and a regression branch). The two branches have the same structure, which are both convolution modules CBS and convolution layers Conv2d. One branch generates a target detection box loss function Bbox Loss from the features (i.e., the first fused image features, the second fused image features and the third fused image features) to map the target detection box. The other branch generates a target category loss function Cls Loss from the features (i.e., the first fused image features, the second fused image features and the third fused image features) to map the target category, and non-maximum suppression (NMS) is used to remove overlapping target detection boxes to obtain the target detection boxes that are finally retained and the target category of the corrosion degree of the hardware in the target detection box.

[0118] Step 105: If the hardware detection model is in the training stage, the total loss value is calculated based on the target detection frame, target category, sample detection frame and sample category.

[0119] In this embodiment, if the hardware detection model is in the training stage, the total loss value is calculated based on the target detection frame, target category, sample detection frame and sample category. The total loss value is used to evaluate the gap between the execution result predicted by the hardware detection model and the actual labeled sample detection frame and sample category. Through the total loss value, the hardware detection model can identify the accuracy of the current prediction, so as to adjust the parameters during the training process to optimize the performance. The smaller the total loss value, the more accurate the prediction of the hardware detection model. Therefore, the total loss value is an important indicator to guide the learning and improvement of the hardware detection model.

[0120] Specifically, the total loss value includes:

[0121] L total =λ cls L CE +λ DFL L DFL +λ CIOU L CIOU ;

[0122]

[0123]

[0124]

[0125] Among them, L total represents the total loss value, λ cls , DFL , CIOU Both represent weights, L CE represents the difference between the target category and the sample category, C represents the number of target categories, and yi represents the sample category, q i represents the target category, L DFL represents the difference between the target detection frame and the sample detection frame, n represents the number of target detection frames, represents the target detection box, p i represents the sample detection box, KL represents With p i The relative entropy between CIOU represents the distance between the target detection box and the sample detection box, IoU represents the intersection-over-union ratio between the target detection box and the sample detection box, ρ 2 (b,b g ) represents the Euclidean distance between the target detection box and the sample detection box, c represents the diagonal distance between the target detection box and the minimum closure box of the sample detection box, α represents a parameter, and v represents a term that measures the consistency of the aspect ratio between the target detection box and the sample detection box.

[0126] Step 106: Update the hardware detection model according to the total loss value.

[0127] In this embodiment, by calculating the total loss value, the hardware detection model can identify which parameters need to be adjusted to reduce the gap between the prediction and the true value, and update the hardware detection model based on the total loss value using a back propagation algorithm such as stochastic gradient descent (SGD) or adaptive moment estimation (Adam).

[0128] The SGD algorithm is an optimization algorithm used to update the parameters of the hardware detection model during the training process. Unlike the traditional gradient descent algorithm, SGD uses only a small portion of the training data (one or several samples) to calculate the gradient each time, thereby speeding up the training and improving the adaptability to large data sets. This method helps to jump out of the local optimal solution when facing complex non-convex optimization problems, thereby finding a better global solution.

[0129] Adaptive moment estimation (Adam) is an optimization algorithm that improves the efficiency of gradient descent by combining momentum method and adaptive learning rate. It uses the first-order moment (mean) and second-order moment (variance) of the current gradient to adjust the learning rate of each parameter. It has strong adaptability and is particularly suitable for processing large-scale data and high-dimensional space. Adam can converge quickly during training and is relatively tolerant of the choice of hyperparameters.

[0130] Step 107 , if the update is completed, determine whether the hardware detection model meets the preset training conditions; if so, execute step 108 ; if not, return to execute step 103 .

[0131] In this embodiment, after completing a parameter update, it is determined whether the hardware detection model has met the set training conditions. These training conditions may include whether the reduction of the total loss value is within an acceptable range, whether the performance of the hardware detection model is significantly improved, or whether the number of training times has reached a predetermined upper limit. If it is detected that these criteria are met, it is confirmed that the training process has been completed.

[0132] On the contrary, if these preset training conditions are not met, it means that the hardware detection model still has not achieved the ideal learning effect. At this time, the backbone image features of multiple scales will be input into the neck network again, and feature fusion will be performed in independent channels to obtain fused image features of multiple scales, and then continue the subsequent processing. This cyclic design ensures that the hardware detection model is continuously optimized and iterated, so that it can accurately capture and extract key information when facing different overhead transmission line image data, thereby improving the effectiveness and reliability of the hardware detection model in practical applications. This feedback mechanism ensures the dynamics and adaptability of the entire training process, prompting the hardware detection model to gradually improve its performance in continuous training and ultimately achieve the desired effect.

[0133] Step 108: Determine that the hardware detection model training is completed.

[0134] In this embodiment, the conditions for continuing training can be preset, and these training conditions may include whether the reduction of the total loss value is within an acceptable range, whether the performance of the hardware detection model is significantly improved, or whether the number of training times has reached a predetermined upper limit, etc. The present invention does not limit this. If it is detected that these criteria are met, it is confirmed that the hardware detection model training is completed.

[0135] In an embodiment of the present invention, a hardware detection model has a backbone network, a neck network and a head network, collects image data of overhead transmission lines, and annotates the image data with a sample detection frame where the hardware on the overhead transmission line is located and a sample category representing the degree of corrosion of the hardware in the sample detection frame, inputs the image data into the backbone network, extracts backbone image features of multiple scales under self-attention, inputs the backbone image features of multiple scales into the neck network, performs feature fusion in an independent channel, obtains fused image features of multiple scales, inputs the fused image features of multiple scales into the head network and maps them into a target detection frame on the image data and a target category representing the degree of corrosion of the hardware in the target detection frame, if the hardware detection model is in the training stage, calculates the total loss value based on the target detection frame, the target category, the sample detection frame and the sample category; updates the hardware detection model based on the total loss value, and if the update is completed, determines whether the hardware detection model meets the preset training conditions, and if so, determines that the training of the hardware detection model is completed, and if not, returns to execute the input of the backbone image features of multiple scales into the neck network, performs feature fusion in an independent channel, and obtains fused image features of multiple scales. Through multi-scale feature extraction and fusion, combined with the self-attention mechanism and independent channel feature fusion, it is possible to effectively extract and fuse image features of multiple scales from overhead transmission line images, accurately locate the hardware position and evaluate its corrosion degree. By calculating the total loss value and dynamically updating the hardware detection model during the training phase, the hardware detection model can be gradually optimized to improve the accuracy and efficiency of hardware detection.

[0136] Embodiment 2

[0137] See also Figure 5 , shows a flowchart of a method for constructing a hardware detection model provided by the second embodiment of the present invention. This embodiment, based on the aforementioned first embodiment, describes the detailed process of the verification phase of the hardware detection model. Figure 5 As shown, the method includes:

[0138] Step 501: collect image data of overhead power transmission lines, and mark the image data with a sample detection frame where hardware on the overhead power transmission lines are located and a sample category representing the degree of corrosion of the hardware in the sample detection frame.

[0139] In this embodiment, real-time image data is collected for overhead power transmission lines, and the image data is annotated with a sample detection frame where the hardware on the overhead power transmission line is located and a sample category representing the degree of hardware corrosion in the sample detection frame. The collected real-time image data is used to verify the accuracy and robustness of the hardware detection model in practical applications. The hardware detection model has a backbone network, a neck network, and a head network.

[0140] Step 502: input the image data into the backbone network, and extract backbone image features of multiple scales under self-attention.

[0141] Step 503: Input the trunk image features of multiple scales into the neck network, perform feature fusion in independent channels, and obtain fused image features of multiple scales.

[0142] Step 504: Input the fused image features of multiple scales into the head network and map them into a target detection frame on the image data and a target category representing the degree of corrosion of the metal fittings in the target detection frame.

[0143] Step 505: If the hardware detection model is in the verification stage, the precision index, recall index, average accuracy index and average precision mean index are calculated based on the target detection frame, target category, sample detection frame and sample category.

[0144] In this embodiment, if the hardware detection model is in the verification stage, the performance of the hardware detection model is evaluated by calculating indicators such as precision, recall, average accuracy, and average precision. These indicators help to comprehensively measure the detection effect of the hardware detection model, especially in the image data of identifying overhead power transmission lines, to ensure that the hardware detection model can not only correctly detect the location of the hardware (precision), but also identify all hardware targets (recall). In addition, the average accuracy and average precision help quantify the performance of the hardware detection model in different image data.

[0145] Step 506: perform a verification operation on the hardware detection model according to the precision index, the recall index, the average accuracy index and the average precision mean index.

[0146] In this embodiment, the calculated precision index, recall index, average accuracy index and average precision mean index are used to verify the hardware detection model, and the hardware detection model is adjusted and optimized according to the actual evaluation results. Through these evaluation indicators, it can be judged whether the model meets the preset performance standards, and then decide whether to conduct further training or adjustment. It can achieve sufficiently high accuracy and reliability in practical applications, thereby improving the accuracy and efficiency of hardware identification and hardware corrosion assessment of overhead transmission lines.

[0147] Specifically, the accuracy is expressed as:

[0148]

[0149] The recall rate is expressed as:

[0150]

[0151] The average accuracy is expressed as:

[0152]

[0153] The mean average precision is expressed as:

[0154]

[0155] P represents the precision, P T It indicates the number of target detection boxes correctly predicted by the hardware detection model and the target categories in the target detection box, P F Represents the number of target detection boxes that are incorrectly predicted as correct by the hardware detection model and the target categories in the target detection box, N F It represents the number of target detection boxes that are incorrectly predicted by the hardware detection model as wrong and the number of target categories in the target detection box. R represents the recall rate, AP represents the average precision, and R n+1 and R n Represents different recall rates, ρ interp (R n+1 ) indicates that the recall rate R n+1 The accuracy of represents the maximum accuracy, mAP represents the average precision of the target category, N represents the number of target categories, AP i represents the average accuracy of the i target categories.

[0156] In an embodiment of the present invention, the hardware detection model has a backbone network, a neck network and a head network, collects image data of the overhead transmission line, and annotates the image data with a sample detection frame where the hardware on the overhead transmission line is located and a sample category representing the degree of corrosion of the hardware in the sample detection frame. The image data is input into the backbone network, and the backbone image features of multiple scales are extracted under self-attention. The backbone image features of multiple scales are input into the neck network, and feature fusion is performed in an independent channel to obtain fused image features of multiple scales. The fused image features of multiple scales are input into the head network and mapped into a target detection frame on the image data and a target category representing the degree of corrosion of the hardware in the target detection frame. If the hardware detection model is in the verification stage, the precision index, the recall index, the average accuracy index and the average precision mean index are calculated based on the target detection frame, the target category, the sample detection frame and the sample category. The hardware detection model is verified according to the precision index, the recall index, the average accuracy index and the average precision mean index. Through the steps of image data annotation, feature extraction, feature fusion and the like, the hardware detection model can not only accurately locate the position of the hardware, but also evaluate its degree of corrosion. In addition, during the verification phase, the model performance is comprehensively evaluated through indicators such as precision, recall, average accuracy and average precision, to ensure its reliability and efficiency in practical applications and improve the accuracy and efficiency of hardware detection.

[0157] Embodiment 3

[0158] See also Figure 6, shows a flowchart of a hardware detection model construction method provided by the third embodiment of the present invention, the method can be executed by a hardware detection model construction device, the hardware detection model construction device can be implemented in the form of hardware and / or software, and the hardware detection model construction device can be configured in a computer device. Figure 6 As shown, the method includes:

[0159] Step 601: Collect image data of overhead power transmission lines.

[0160] In this embodiment, image data is collected from overhead power transmission lines. The collection process can be performed by photographing or using drones, and the collected images are divided into data blocks of equal size to facilitate input into a hardware detection model for prediction.

[0161] Step 602: Load the hardware detection model.

[0162] In this embodiment, a hardware detection model trained according to the method provided in Embodiment 1 and / or Embodiment 2 is loaded to identify the collected image data, and the hardware detection model includes a backbone network, a neck network, and a head network.

[0163] Step 603: input the image data into the backbone network, and extract backbone image features of multiple scales under self-attention.

[0164] In one embodiment of the present invention, the backbone network includes 5 convolution modules, 4 C2f modules, 1 spatial pyramid pooling fast module and 1 parallel self-attention module; the backbone image features include first-order image features, second-order image features and third-order image features.

[0165] The image data is sequentially input into the convolution module, the convolution module, the C2f module, the convolution module and the C2f module for feature extraction to obtain the first-order image features.

[0166] The first-order image features are sequentially input into the convolution module and the C2f module for feature extraction to obtain the second-order image features.

[0167] The second-order image features are sequentially input into the convolution module, the C2f module, and the spatial pyramid pooling fast module for feature extraction to obtain the first candidate image features.

[0168] The first candidate image features are input into the parallel self-attention module, and the third-order image features are extracted under self-attention.

[0169] In one embodiment of the present invention, the parallel self-attention module includes a multi-head self-attention module, a feedforward network, a first initial convolution layer and a first end convolution layer; inputting the first candidate image feature into the parallel self-attention module, and extracting the third-order image feature under self-attention specifically includes the following steps:

[0170] The first candidate image feature is input into the first initial convolutional layer to perform a convolution operation to obtain a first convolution image feature.

[0171] The first convolution image features are divided into first training image features and first non-training image features.

[0172] The first training image features are input into the multi-head self-attention module to calculate the total attention image features.

[0173] The first training image feature is added to the total attention image feature to obtain the first merged image feature.

[0174] The first merged image features are input into the feedforward network for linear transformation to obtain transformed image features.

[0175] The first merged image feature is added to the transformed image feature to obtain a second merged image feature.

[0176] The second merged image feature and the first non-training image feature are concatenated to obtain a third merged image feature.

[0177] The third merged image feature is input into the first end convolution layer to perform a convolution operation to obtain a third-order image feature.

[0178] Step 604: Input the trunk image features of multiple scales into the neck network, perform feature fusion in independent channels, and obtain fused image features of multiple scales.

[0179] In one embodiment of the present invention, the neck network includes 2 upsampling modules, 3 C2f modules, 2 convolution modules and 1 C2f module combined with a compact inverted block structure; the fused image features include first-order fused image features, second-order fused image features and third-order fused image features.

[0180] The third-order image features are input into the upsampling module for feature extraction to obtain the first upsampling image features.

[0181] The first up-sampled image feature is concatenated with the second-order image feature to obtain a second candidate fused image feature.

[0182] The second candidate fused image features are sequentially input into the C2f module and the upsampling module for feature extraction to obtain the second upsampling image features.

[0183] The second up-sampled image feature is concatenated with the first-order image feature to obtain a first candidate fused image feature.

[0184] The first candidate fused image feature is input into the C2f module for feature transformation to obtain the first-order fused image feature.

[0185] The first-order fused image features are input into the convolution module to perform a convolution operation to obtain the third candidate fused image features.

[0186] The third candidate fused image feature and the second candidate fused image feature are spliced ​​together to obtain a first spliced ​​image feature.

[0187] The first spliced ​​image features are input into the C2f module for feature transformation to obtain the second-order fused image features.

[0188] The second-order fused image features are input into the convolution module to perform a convolution operation to obtain a fourth candidate fused image feature.

[0189] The fourth candidate fused image feature is concatenated with the third-order image feature to obtain a second concatenated image feature.

[0190] The second spliced ​​image features are input into the C2f module combined with a compact inverted block structure, and feature fusion is performed in independent channels to obtain the third-order fused image features.

[0191] In one embodiment of the present invention, the C2f module of the compact inverted block structure includes a second initial convolution layer, a second end convolution layer and a compact inverted block structure; the compact inverted block structure includes 5 convolution layers; the second spliced ​​image feature is input into the C2f module combined with the compact inverted block structure, and feature fusion is performed in an independent channel to obtain the third-order fused image feature, which specifically includes the following steps:

[0192] The second spliced ​​image feature is input into the second initial convolution layer to perform a convolution operation to obtain a second convolution image feature.

[0193] The second convolution image features are divided into second training image features and second non-training image features.

[0194] Sets the number of times the compact inverted block structure is executed.

[0195] The second training image feature is input into the compact inverted block structure to perform a convolution operation until the execution times are met, thereby obtaining a plurality of convolution training image features of different scales; the plurality of convolution training image features of different scales are the execution results of each compact inverted block structure.

[0196] The convolutional training image features of multiple different scales are concatenated with the second non-training image feature to obtain a third concatenated image feature.

[0197] The third concatenated image features are input into the second end convolution layer to perform a convolution operation to obtain the third-order fused image features.

[0198] Step 605: Input the fused image features of multiple scales into the head network and map them into a target detection frame where the hardware on the overhead transmission line on the image data is located and a target category representing the degree of corrosion of the hardware in the target detection frame.

[0199] In this embodiment, by inputting fused image features at multiple scales, the head network can more accurately locate and classify hardware in the image data, more comprehensively determine whether there are hardware in the image data, and if there are hardware, determine the specific location of the hardware (i.e., the target detection box) and the degree of corrosion of the hardware (i.e., the target category).

[0200] For example, Figure 7 The figure shows the result of inputting a certain image data into a hardware detection model. In the figure, "sector" and "board" represent different types of hardware. A target detection frame is generated for the identified hardware. The target category of the degree of corrosion of the hardware can be defined by the color of the target detection frame to represent the different degrees of corrosion of the hardware, or the degree of corrosion can be directly displayed in the target detection frame. The present invention is not limited here.

[0201] In an embodiment of the present invention, image data of an overhead transmission line is collected; a hardware detection model is loaded; the hardware detection model includes a backbone network, a neck network, and a head network; the image data is input into the backbone network, and backbone image features of multiple scales are extracted under self-attention; the backbone image features of multiple scales are input into the neck network, and feature fusion is performed in an independent channel to obtain fused image features of multiple scales; the fused image features of multiple scales are input into the head network and mapped into a target detection frame where the hardware on the overhead transmission line on the image data is located and a target category representing the degree of corrosion of the hardware in the target detection frame. By adopting a multi-scale feature extraction and fusion strategy, the hardware position in the image data can be more accurately identified and its degree of corrosion can be evaluated. By extracting image features through the backbone network, performing feature fusion through the neck network, and predicting the target detection frame and classifying the degree of corrosion through the head network, the accuracy and efficiency of hardware detection can be improved.

[0202] Embodiment 4

[0203] Figure 8 This is a schematic diagram of the structure of a hardware detection model construction device provided in Embodiment 4 of the present invention. The hardware detection model has a backbone network, a neck network and a head network. Figure 8 As shown, the device comprises:

[0204] The image data acquisition module 801 is used to collect image data of the overhead transmission line, and mark the image data with a sample detection frame where the hardware on the overhead transmission line is located and a sample category representing the corrosion degree of the hardware in the sample detection frame;

[0205] A backbone network training module 802, used to input the image data into the backbone network and extract backbone image features of multiple scales under self-attention;

[0206] A neck network training module 803 is used to input the trunk image features of multiple scales into the neck network, perform feature fusion in independent channels, and obtain fused image features of multiple scales;

[0207] A head network training module 804 is used to input the fused image features of multiple scales into the head network and map them into a target detection frame on the image data and a target category representing the degree of corrosion of the metal fittings in the target detection frame;

[0208] A total loss value calculation module 805 is used to calculate the total loss value according to the target detection frame, the target category, the sample detection frame and the sample category if the hardware detection model is in the training stage;

[0209] A total loss value updating module 806 is used to update the hardware detection model according to the total loss value;

[0210] The condition judgment module 807 is used to judge whether the hardware detection model meets the preset training conditions if the update is completed; if so, execute the model training completion module 808; if not, return to execute the neck network training module 803;

[0211] The model training completion module 808 is used to determine whether the hardware detection model training is completed.

[0212] In one embodiment of the present invention, the backbone network includes 5 convolution modules, 4 C2f modules, 1 spatial pyramid pooling fast module and 1 parallel self-attention module; the backbone image features include first-order image features, second-order image features and third-order image features; the backbone network training module 802 includes:

[0213] A first-order image feature extraction module, used for sequentially inputting the image data into the convolution module, the convolution module, the C2f module, the convolution module and the C2f module to extract features and obtain first-order image features;

[0214] A second-order image feature extraction module, used for sequentially inputting the first-order image features into the convolution module and the C2f module for feature extraction to obtain second-order image features;

[0215] A first candidate image feature acquisition module, used for sequentially inputting the second-order image features into the convolution module, the C2f module, and the spatial pyramid pooling fast module for feature extraction to obtain first candidate image features;

[0216] The third-order image feature extraction module is used to input the first candidate image feature into the parallel self-attention module and extract the third-order image feature under self-attention.

[0217] In one embodiment of the present invention, the parallel self-attention module includes a multi-head self-attention module, a feedforward network, a first initial convolutional layer and a first end convolutional layer; the third-order image feature extraction module includes:

[0218] A first convolution image feature acquisition module, used for inputting the first candidate image feature into the first initial convolution layer to perform a convolution operation to obtain a first convolution image feature;

[0219] A first convolution image feature division module, used for dividing the first convolution image feature into a first training image feature and a first non-training image feature;

[0220] A total attention image feature calculation module, used for inputting the first training image feature into the multi-head self-attention module to calculate the total attention image feature;

[0221] A first merged image feature acquisition module, configured to add the first training image feature to the total attention image feature to obtain a first merged image feature;

[0222] A transformed image feature acquisition module, used for inputting the first combined image feature into the feedforward network for linear transformation to obtain a transformed image feature;

[0223] A second merged image feature acquisition module, used for adding the first merged image feature and the transformed image feature to obtain a second merged image feature;

[0224] A third merged image feature acquisition module, used for concatenating the second merged image feature and the first non-training image feature to obtain a third merged image feature;

[0225] The third-order image feature acquisition module is used to input the third merged image feature into the first end convolution layer to perform a convolution operation to obtain the third-order image feature.

[0226] In one embodiment of the present invention, the neck network includes 2 upsampling modules, 3 C2f modules, 2 convolution modules and 1 C2f module combined with a compact inverted block structure; the fused image features include first-order fused image features, second-order fused image features and third-order fused image features; the neck network training module 803 includes:

[0227] A first up-sampled image feature acquisition module, used for inputting the third-order image feature into the up-sampled module for feature extraction to obtain a first up-sampled image feature;

[0228] A second candidate fused image feature acquisition module, configured to concatenate the first up-sampled image feature with the second-order image feature to obtain a second candidate fused image feature;

[0229] A second up-sampled image feature acquisition module, used for sequentially inputting the second candidate fused image feature into the C2f module and the up-sampled module for feature extraction to obtain a second up-sampled image feature;

[0230] A first candidate fused image feature acquisition module, configured to concatenate the second up-sampled image feature with the first-order image feature to obtain a first candidate fused image feature;

[0231] A first-order fusion image feature acquisition module, used for inputting the first candidate fusion image feature into the C2f module for feature transformation to obtain a first-order fusion image feature;

[0232] A third candidate fused image feature acquisition module, configured to input the first-order fused image feature into the convolution module to perform a convolution operation to obtain a third candidate fused image feature;

[0233] A first spliced ​​image feature acquisition module, configured to splice the third candidate fused image feature and the second candidate fused image feature to obtain a first spliced ​​image feature;

[0234] A second-order fusion image feature acquisition module, used for inputting the first spliced ​​image feature into the C2f module for feature transformation to obtain a second-order fusion image feature;

[0235] a fourth candidate fused image feature acquisition module, configured to input the second-order fused image feature into the convolution module to perform a convolution operation, thereby obtaining a fourth candidate fused image feature;

[0236] A second stitched image feature acquisition module, configured to stitch the fourth candidate fused image feature with the third-order image feature to obtain a second stitched image feature;

[0237] The third-order fused image feature extraction module is used to input the second spliced ​​image features into the C2f module combined with the compact inverted block structure, perform feature fusion in an independent channel, and obtain the third-order fused image features.

[0238] In one embodiment of the present invention, the C2f module of the compact inverted block structure includes a second initial convolution layer, a second end convolution layer and a compact inverted block structure; the compact inverted block structure includes 5 convolution layers; the third-order fusion image feature extraction module includes:

[0239] A second convolution image feature acquisition module, used for inputting the second spliced ​​image feature into the second initial convolution layer to perform a convolution operation to obtain a second convolution image feature;

[0240] A second convolution image feature division module, used for dividing the second convolution image feature into a second training image feature and a second non-training image feature;

[0241] An execution times setting module, used to set the execution times of the compact inverted block structure;

[0242] A convolution training image feature extraction module, used for inputting the second training image feature into the compact inverted block structure to perform a convolution operation until the execution times are met, thereby obtaining a plurality of convolution training image features of different scales; the plurality of convolution training image features of different scales are the execution results of the compact inverted block structure each time;

[0243] A third spliced ​​image feature acquisition module, configured to splice the convolutional training image features of the plurality of different scales with the second non-training image feature to obtain a third spliced ​​image feature;

[0244] The third-order fused image feature acquisition module is used to input the third spliced ​​image feature into the second end convolution layer to perform a convolution operation to obtain the third-order fused image feature.

[0245] In one embodiment of the present invention, the total loss value includes:

[0246] L total =λ cls L CE +λ DFL L DFL +λ CIOU L CIOU ;

[0247]

[0248]

[0249]

[0250] Among them, L total represents the total loss value, λ cls , DFL , CIOU Both represent weights, L CE represents the difference between the target category and the sample category, C represents the number of target categories, y i represents the sample category, q i represents the target category, L DFLrepresents the difference between the target detection frame and the sample detection frame, n represents the number of target detection frames, represents the target detection box, p i represents the sample detection box, KL represents With p i The relative entropy between CIOU represents the distance between the target detection box and the sample detection box, IoU represents the intersection-over-union ratio between the target detection box and the sample detection box, ρ 2 (b,b g ) represents the Euclidean distance between the target detection box and the sample detection box, c represents the diagonal distance of the minimum closure box between the target detection box and the sample detection box, α represents a parameter, and v represents a term for measuring the consistency of the aspect ratio between the target detection box and the sample detection box.

[0251] In one embodiment of the present invention, the device further comprises:

[0252] An index calculation module, used for calculating the precision index, the recall index, the average accuracy index and the average precision mean index according to the target detection frame, the target category, the sample detection frame and the sample category if the hardware detection model is in the verification stage;

[0253] A model verification module, used to perform a verification operation on the hardware detection model according to the precision rate index, the recall rate index, the average accuracy rate index and the average precision mean index;

[0254] Wherein, the accuracy is expressed as:

[0255]

[0256] The recall rate is expressed as:

[0257]

[0258] The average accuracy is expressed as:

[0259]

[0260] The average precision mean is expressed as:

[0261]

[0262] P represents the accuracy, P T represents the number of target categories between the target detection box and the target detection box correctly predicted by the hardware detection model, P Frepresents the number of target detection boxes and target categories in the target detection boxes that are incorrectly predicted as correct by the hardware detection model, N F represents the number of target detection boxes that are incorrectly predicted by the hardware detection model as wrong and the number of target categories in the target detection box, R represents the recall rate, AP represents the average precision, and R n+1 and R n Denotes different recall rates, ρ interp (R n+1 ) indicates that the recall rate R n+1 The accuracy of represents the maximum accuracy, mAP represents the mean average precision of the target category, N represents the number of target categories, AP i represents the average accuracy of the i target categories.

[0263] The hardware detection model construction device provided in the embodiment of the present invention can execute the hardware detection model construction method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the hardware detection model construction method.

[0264] Embodiment 5

[0265] Fig. 9 A schematic diagram of the structure of a hardware identification device provided in Embodiment 5 of the present invention is shown in FIG. Fig. 9 As shown, the device comprises:

[0266] The data acquisition module 901 is used to collect image data of the overhead transmission line;

[0267] The model loading module 902 is used to load the hardware detection model trained by the method provided in the first embodiment and / or the second embodiment; the hardware detection model includes a backbone network, a neck network and a head network;

[0268] A backbone image feature extraction module 903, used to input the image data into the backbone network and extract backbone image features of multiple scales under self-attention;

[0269] A fused image feature extraction module 904 is used to input the trunk image features of multiple scales into the neck network, perform feature fusion in independent channels, and obtain fused image features of multiple scales;

[0270] The image data mapping module 905 is used to input the fused image features of multiple scales into the head network and map them into a target detection frame where the hardware on the overhead transmission line on the image data is located and a target category representing the degree of corrosion of the hardware in the target detection frame.

[0271] In one embodiment of the present invention, the backbone network includes 5 convolution modules, 4 C2f modules, 1 spatial pyramid pooling fast module and 1 parallel self-attention module; the backbone image features include first-order image features, second-order image features and third-order image features; the backbone image feature extraction module 903 includes:

[0272] A first-order image feature extraction module, used for sequentially inputting the image data into the convolution module, the convolution module, the C2f module, the convolution module and the C2f module to extract features and obtain first-order image features;

[0273] A second-order image feature extraction module, used for sequentially inputting the first-order image features into the convolution module and the C2f module for feature extraction to obtain second-order image features;

[0274] A first candidate image feature acquisition module, used for sequentially inputting the second-order image features into the convolution module, the C2f module, and the spatial pyramid pooling fast module for feature extraction to obtain first candidate image features;

[0275] The third-order image feature extraction module is used to input the first candidate image feature into the parallel self-attention module and extract the third-order image feature under self-attention.

[0276] In one embodiment of the present invention, the parallel self-attention module includes a multi-head self-attention module, a feedforward network, a first initial convolutional layer and a first end convolutional layer; the third-order image feature extraction module includes:

[0277] A first convolution image feature acquisition module, used for inputting the first candidate image feature into the first initial convolution layer to perform a convolution operation to obtain a first convolution image feature;

[0278] A first convolution image feature division module, used for dividing the first convolution image feature into a first training image feature and a first non-training image feature;

[0279] A total attention image feature calculation module, used for inputting the first training image feature into the multi-head self-attention module to calculate the total attention image feature;

[0280] A first merged image feature acquisition module, configured to add the first training image feature to the total attention image feature to obtain a first merged image feature;

[0281] A transformed image feature acquisition module, used for inputting the first combined image feature into the feedforward network for linear transformation to obtain a transformed image feature;

[0282] A second merged image feature acquisition module, used for adding the first merged image feature and the transformed image feature to obtain a second merged image feature;

[0283] A third merged image feature acquisition module, used for concatenating the second merged image feature and the first non-training image feature to obtain a third merged image feature;

[0284] The third-order image feature acquisition module is used to input the third merged image feature into the first end convolution layer to perform a convolution operation to obtain the third-order image feature.

[0285] In one embodiment of the present invention, the neck network includes 2 upsampling modules, 3 C2f modules, 2 convolution modules and 1 C2f module combined with a compact inverted block structure; the fused image features include first-order fused image features, second-order fused image features and third-order fused image features; the fused image feature extraction module 904 includes:

[0286] A first up-sampled image feature acquisition module, used for inputting the third-order image feature into the up-sampled module for feature extraction to obtain a first up-sampled image feature;

[0287] A second candidate fused image feature acquisition module, configured to concatenate the first up-sampled image feature with the second-order image feature to obtain a second candidate fused image feature;

[0288] A second up-sampled image feature acquisition module, used for sequentially inputting the second candidate fused image feature into the C2f module and the up-sampled module for feature extraction to obtain a second up-sampled image feature;

[0289] A first candidate fused image feature acquisition module, configured to concatenate the second up-sampled image feature with the first-order image feature to obtain a first candidate fused image feature;

[0290] A first-order fusion image feature acquisition module, used for inputting the first candidate fusion image feature into the C2f module for feature transformation to obtain a first-order fusion image feature;

[0291] A third candidate fused image feature acquisition module, configured to input the first-order fused image feature into the convolution module to perform a convolution operation to obtain a third candidate fused image feature;

[0292] A first spliced ​​image feature acquisition module, configured to splice the third candidate fused image feature and the second candidate fused image feature to obtain a first spliced ​​image feature;

[0293] A second-order fusion image feature acquisition module, used for inputting the first spliced ​​image feature into the C2f module for feature transformation to obtain a second-order fusion image feature;

[0294] a fourth candidate fused image feature acquisition module, configured to input the second-order fused image feature into the convolution module to perform a convolution operation, thereby obtaining a fourth candidate fused image feature;

[0295] A second stitched image feature acquisition module, configured to stitch the fourth candidate fused image feature with the third-order image feature to obtain a second stitched image feature;

[0296] The third-order fused image feature extraction module is used to input the second spliced ​​image features into the C2f module combined with the compact inverted block structure, perform feature fusion in an independent channel, and obtain the third-order fused image features.

[0297] In one embodiment of the present invention, the C2f module of the compact inverted block structure includes a second initial convolution layer, a second end convolution layer and a compact inverted block structure; the compact inverted block structure includes 5 convolution layers; the third-order fusion image feature extraction module includes:

[0298] A second convolution image feature acquisition module, used for inputting the second spliced ​​image feature into the second initial convolution layer to perform a convolution operation to obtain a second convolution image feature;

[0299] A second convolution image feature division module, used for dividing the second convolution image feature into a second training image feature and a second non-training image feature;

[0300] An execution times setting module, used to set the execution times of the compact inverted block structure;

[0301] A convolution training image feature extraction module, used for inputting the second training image feature into the compact inverted block structure to perform a convolution operation until the execution times are met, thereby obtaining a plurality of convolution training image features of different scales; the plurality of convolution training image features of different scales are the execution results of the compact inverted block structure each time;

[0302] A third spliced ​​image feature acquisition module, configured to splice the convolutional training image features of the plurality of different scales with the second non-training image feature to obtain a third spliced ​​image feature;

[0303] The third-order fused image feature acquisition module is used to input the third spliced ​​image feature into the second end convolution layer to perform a convolution operation to obtain the third-order fused image feature.

[0304] The hardware identification device provided in the embodiment of the present invention can execute the hardware identification method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the hardware identification method.

[0305] Embodiment 6

[0306] See also Fig.10 , showing a schematic diagram of the structure of a computer device provided by an embodiment of the present invention. The computer device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, blade servers, mainframe computers, and other suitable computers. The computer device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0307] like Fig.10 As shown, the computer device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the computer device 10 can also be stored. The processor 11, ROM 12 and RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0308] A number of components in the computer device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the computer device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0309] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a hardware detection model construction method, or a hardware identification method.

[0310] In some embodiments, the hardware detection model building method, or the hardware identification method can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the computer device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the hardware detection model building method or the hardware identification method described above can be executed. Alternatively, in other embodiments, the processor 11 can be configured to execute the hardware detection model building method or the hardware identification method in any other appropriate manner (for example, by means of firmware).

[0311] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0312] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0313] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0314] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer device. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0315] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0316] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0317] Embodiment 7

[0318] An embodiment of the present invention further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the hardware detection model construction method or the hardware identification method provided in any embodiment of the present invention.

[0319] In the process of implementation, the computer program product can be written in one or more programming languages ​​or a combination thereof to perform the computer program code of the present invention, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0320] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0321] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for constructing a hardware detection model, characterized in that: The hardware detection model has a backbone network, a neck network and a head network, and the method includes: Collecting image data of an overhead power transmission line, and marking the image data with a sample detection frame where hardware on the overhead power transmission line is located and a sample category representing the degree of corrosion of the hardware in the sample detection frame; Inputting the image data into the backbone network, and extracting backbone image features of multiple scales under self-attention; Inputting the trunk image features of multiple scales into the neck network, performing feature fusion in independent channels, and obtaining fused image features of multiple scales; Input the fused image features at multiple scales into the head network and map them into a target detection frame on the image data and a target category representing the degree of corrosion of the metal fittings in the target detection frame; If the hardware detection model is in the training stage, a total loss value is calculated according to the target detection frame, the target category, the sample detection frame and the sample category; Updating the hardware detection model according to the total loss value; If the update is completed, it is determined whether the hardware detection model meets the preset training conditions; if so, it is determined that the hardware detection model training is completed; if not, it returns to execute the input of the trunk image features of multiple scales into the neck network, and performs feature fusion in independent channels to obtain fused image features of multiple scales.

2. The method according to claim 1, characterized in that The backbone network includes 5 convolution modules, 4 C2f modules, 1 spatial pyramid pooling fast module and 1 parallel self-attention module; the backbone image features include first-order image features, second-order image features and third-order image features; the image data is input into the backbone network, and the backbone image features of multiple scales are extracted under self-attention, including: Inputting the image data into the convolution module, the convolution module, the C2f module, the convolution module and the C2f module in sequence to extract features, thereby obtaining first-order image features; Inputting the first-order image features into the convolution module and the C2f module in sequence for feature extraction to obtain second-order image features; Inputting the second-order image features into the convolution module, the C2f module, and the spatial pyramid pooling fast module in sequence for feature extraction to obtain first candidate image features; The first candidate image feature is input into the parallel self-attention module, and the third-order image feature is extracted under self-attention.

3. The method according to claim 2, characterized in that The parallel self-attention module includes a multi-head self-attention module, a feedforward network, a first initial convolution layer and a first ending convolution layer; the step of inputting the first candidate image feature into the parallel self-attention module and extracting the third-order image feature under self-attention includes: Inputting the first candidate image feature into the first initial convolutional layer to perform a convolution operation to obtain a first convolutional image feature; Dividing the first convolution image feature into a first training image feature and a first non-training image feature; Inputting the first training image feature into the multi-head self-attention module to calculate the total attention image feature; Adding the first training image feature to the total attention image feature to obtain a first combined image feature; Inputting the first combined image feature into the feedforward network for linear transformation to obtain a transformed image feature; Adding the first merged image feature and the transformed image feature to obtain a second merged image feature; Concatenate the second merged image feature and the first non-training image feature to obtain a third merged image feature; The third merged image feature is input into the first end convolution layer to perform a convolution operation to obtain a third-order image feature.

4. The method according to claim 2, characterized in that: The neck network includes 2 upsampling modules, 3 C2f modules, 2 convolution modules and 1 C2f module combined with a compact inverted block structure; the fused image features include first-order fused image features, second-order fused image features and third-order fused image features; The step of inputting the trunk image features of multiple scales into the neck network and fusing the features in independent channels to obtain fused image features of multiple scales includes: Inputting the third-order image features into the upsampling module for feature extraction to obtain first upsampling image features; Concatenate the first up-sampled image feature and the second-order image feature to obtain a second candidate fused image feature; Inputting the second candidate fused image features into the C2f module and the upsampling module in sequence for feature extraction to obtain second upsampling image features; Concatenate the second up-sampled image feature with the first-order image feature to obtain a first candidate fused image feature; Inputting the first candidate fused image feature into the C2f module for feature transformation to obtain a first-order fused image feature; Inputting the first-order fused image features into the convolution module to perform a convolution operation to obtain a third candidate fused image feature; splicing the third candidate fused image feature and the second candidate fused image feature to obtain a first spliced ​​image feature; Inputting the first spliced ​​image features into the C2f module for feature transformation to obtain second-order fused image features; Inputting the second-order fused image features into the convolution module to perform a convolution operation to obtain a fourth candidate fused image feature; Splicing the fourth candidate fused image feature with the third-order image feature to obtain a second spliced ​​image feature; The second spliced ​​image features are input into the C2f module combined with the compact inverted block structure, and feature fusion is performed in an independent channel to obtain a third-order fused image feature.

5. The method according to claim 4, characterized in that The C2f module of the compact inverted block structure includes a second initial convolution layer, a second end convolution layer and a compact inverted block structure; the compact inverted block structure includes 5 convolution layers; the inputting the second spliced ​​image feature into the C2f module combined with the compact inverted block structure, performing feature fusion in an independent channel, and obtaining a third-order fused image feature, includes: Inputting the second spliced ​​image feature into the second initial convolution layer to perform a convolution operation to obtain a second convolution image feature; Dividing the second convolution image features into second training image features and second non-training image features; Setting the number of executions of the compact inverted block structure; Inputting the second training image feature into the compact inverted block structure to perform a convolution operation until the execution times are met, thereby obtaining a plurality of convolution training image features of different scales; the plurality of convolution training image features of different scales are the execution results of the compact inverted block structure each time; Splicing the multiple convolution training image features of different scales with the second non-training image feature to obtain a third spliced ​​image feature; The third spliced ​​image feature is input into the second end convolution layer to perform a convolution operation to obtain a third-order fused image feature.

6. The method according to claim 1, characterized in that The total loss value includes: L total =λ cls L CE +λ DFL L DFL +λ CIOU L CIOU ; Among them, L total represents the total loss value, λ cls , DFL , CIOU Both represent weights, L CE represents the difference between the target category and the sample category, C represents the number of target categories, y i represents the sample category, q i represents the target category, L DFL represents the difference between the target detection frame and the sample detection frame, n represents the number of target detection frames, represents the target detection box, p i represents the sample detection box, KL represents With p i The relative entropy between CIOU represents the distance between the target detection box and the sample detection box, IoU represents the intersection-over-union ratio between the target detection box and the sample detection box, ρ 2 (b,b g ) represents the Euclidean distance between the target detection box and the sample detection box, c represents the diagonal distance of the minimum closure box between the target detection box and the sample detection box, α represents a parameter, and v represents a term for measuring the consistency of the aspect ratio between the target detection box and the sample detection box.

7. The method according to any one of claims 1 to 6, characterized in that Also includes: If the hardware detection model is in the verification stage, the precision index, the recall index, the average accuracy index and the average precision mean index are calculated according to the target detection frame, the target category, the sample detection frame and the sample category; Performing a verification operation on the hardware detection model according to the precision rate index, the recall rate index, the average accuracy rate index and the average precision mean index; Wherein, the accuracy is expressed as: The recall rate is expressed as: The average accuracy is expressed as: The average precision mean is expressed as: P represents the accuracy, P T represents the number of target detection boxes and target categories in the target detection boxes that are correctly predicted by the hardware detection model, P F represents the number of target detection boxes and target categories in the target detection boxes that are incorrectly predicted as correct by the hardware detection model, N F represents the number of target detection boxes that are incorrectly predicted by the hardware detection model as wrong and the number of target categories in the target detection box, R represents the recall rate, AP represents the average precision, and R n+1 and R n Denotes different recall rates, ρ interp (R n+1 ) indicates that the recall rate R n+1 The accuracy of represents the maximum accuracy, mAP represents the mean average precision of the target category, N represents the number of target categories, AP i represents the average accuracy of the i target categories.

8. A method for identifying hardware, characterized in that: The method comprises: Collect image data of overhead power transmission lines; Loading a hardware detection model trained according to the method described in any one of claims 1 to 7; the hardware detection model includes a backbone network, a neck network and a head network; Inputting the image data into the backbone network, and extracting backbone image features of multiple scales under self-attention; Inputting the trunk image features of multiple scales into the neck network, performing feature fusion in independent channels, and obtaining fused image features of multiple scales; The fused image features at multiple scales are input into the head network and mapped into a target detection frame where the hardware on the overhead transmission line on the image data is located and a target category representing the degree of corrosion of the hardware in the target detection frame.

9. A computer device, characterized in that: The computer device comprises: one or more processors; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method for constructing a hardware detection model as described in any one of claims 1 to 7, or the method for identifying power transmission line hardware as described in claim 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method for constructing a hardware detection model as described in any one of claims 1 to 7 is implemented, or the method for identifying hardware as described in claim 8 is implemented.

Images