Identifying method for patrol flight defects of power transmission and transformation unmanned aerial vehicle based on isomorphic model

Abstract

The invention relates to a power transmission and transformation unmanned aerial vehicle patrol flight defect identification method based on an isomorphic model, and belongs to the field of power patrol. The problems that in the prior art, defect identification indexes of different equipment types are unbalanced, and model parameters are redundant are solved. According to the method, firstly, a detection model with a relatively large parameter quantity is used for extracting backbone and equipment identification capability, an equipment detection task is used for carrying out model pre-training, and a backbone part is used as a common baseline feature representation model for all defect identification; secondly, dividing candidate areas through an equipment area, and performing defect identification on different equipment objects by applying different tasks by using multi-expert routing according to identity information of the candidate areas; finally, providing a spatial adaptive classification head model for a classification task; for a detection task, a local and joint detection head model is provided; for a scene perception task, an environment perception head model is provided.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of power inspection, and provides a power transmission and transformation unmanned aerial vehicle patrol defect identification method based on an isomorphic model. BACKGROUND

[0002] Power transmission and transformation defect identification has been changed from manual inspection to unmanned aerial vehicle inspection. Unmanned aerial vehicles solve the gap in manpower demand for manual inspection caused by the rapid growth of line mileage, but at the same time, the demand for analyzing inspection images arises. Due to the massive shooting of unmanned aerial vehicles, this brings great challenges to the post-defect identification of operation and maintenance teams.

[0003] To solve this problem, defect identification schemes based on deep learning are gradually applied to various defects. For example, fittings, anti-vibration hammers, and iron towers, which have successively experienced pure CNN technical solutions (such as YOLOv8), DETR series model technical solutions (such as the CO-DETR model), and large model technical solutions (such as the Guangming large model). These models have very good recognition effects on some defects, but for some defects with few samples and discrete defect correlation features, the recognition effect is very poor, and serious imbalance occurs.

[0004] For different device defect identification, existing technologies commonly use two modes of device-specific defect identification and the same deep learning model for full-quantity identification of all defects. The former has a large amount of redundant calculation, with more than 70% of the calculation being invalid and repetitive. The latter is fused into a model, and difficult categories and categories that are not compatible with the current task type will cause recognition performance degradation of other categories. SUMMARY

[0005] The purpose of the present application is to solve the technical problems of calculation redundancy, aggregation recognition performance degradation, and huge differences in recognition effects of different categories caused by model solutions in power transmission and transformation unmanned aerial vehicle defect identification.

[0006] In order to achieve the above purpose, the present application adopts the following technical means:

[0007] The present application provides a power transmission and transformation unmanned aerial vehicle patrol defect identification method based on an isomorphic model, characterized in that it comprises:

[0008] Step 1: using a SwinL model as the backbone network of the isomorphic model, pre-training through a device detection task and a feature mask reconstruction task, and taking the pre-trained device identification model as a baseline model for all defect identification;

[0009] Step 2: for the baseline model, using a multi-expert routing technology to distribute the local feature representation output by the backbone network to the corresponding expert model;

[0010] Step 3: For component, instance-level defect and local feature representation, the multi-level features of the backbone network are fused using the enhanced FPN framework to obtain a spatial adaptive classification head model, and a spatial adaptive classification result is output;

[0011] Step 4: For device-level defects, a multi-dimensional auxiliary detection head model is generated by cascading the detection head and the parallel auxiliary task technology, and a local defect recognition result is output for rust, offset and shedding local defect recognition.

[0012] Step 5: For environment perception type defects, an environment perception head model is generated by a spatial scale adaptive attention mechanism, and a global defect recognition result is output, such as a vibration damper slip.

[0013] In the above scheme, the SwinL model is used as the backbone network of the isomorphic model, and the device detection task and the feature mask reconstruction task are used for pre-training, wherein the loss function of pre-training is:

[0014] L total =L d (g(f(X)),Y)+λ·||(h(f(X)⊙M)-x)⊙(1-M)||1

[0015] Where f is the backbone network encoder function, g is the detection head, h is the mask reconstruction model, M is a binary mask matrix representing the area to be masked. λ is the loss weighting weight of the two tasks.

[0016] In the above scheme, step 2 includes the following steps:

[0017] Step 2.1: According to the defect characteristics of the power transmission and transformation equipment, a device-route mapping table is obtained, wherein the device-route mapping table is a one-to-many mapping relationship.

[0018] Step 2.2: The device area output by the baseline model is distributed to the corresponding expert model according to the device-route mapping relationship to obtain a backbone network feature distributor.

[0019] In the above scheme, the feature distributor adaptively scales the task area size according to the feature area label and the routing type, and divides the recognition result into target device recognition result and non-target device recognition result.

[0020] For the target device recognition result, a predefined device aggregation method is used for fusion.

[0021] For the non-target device recognition result, it is summarized to a global non-target device set, and after excluding the target device, the device aggregation method is applied and the hyperparameters are set for filtering. The target device recognition result and the non-target device recognition result are combined to obtain the device recognition result.

[0022] In the above scheme, in step 3, the multi-level features of the backbone are fused using an enhanced FPN framework to obtain a spatial adaptive classification head model for component, instance-level defect and local feature representation, wherein the fusion process of the enhanced FPN framework includes:

[0023] fpn i =CNN(WindowAttn(b outi ))

[0024] fpn i-1 =fpn i-1 +Interprolate(fpn i ),i>1

[0025] FPN out =Concat([AP(LCNN(fpn0)),…,AP(LCNN(fpn cn ))])

[0026] Wherein, b outi represents the i-th level output feature of the backbone network, CNN represents a convolutional neural network, WindowAttn represents a window attention mechanism, Interprolate represents an interpolation function, AP represents an adaptive pooling operator, LCNN represents a layer convolution module, cn represents the number of intermediate layers of the backbone network, and fpn0 represents the output after the 0-th level feature of the backbone is processed by CNN and WindowAttn.

[0027] The classification model output is obtained by using full connection convolution technology, and the calculation process is as follows:

[0028] category=Sigmoid(FC(FPN out ))

[0029] Wherein, FC represents a full connection layer, and Sigmoid represents an activation function.

[0030] In the above scheme, step 4 includes:

[0031] Step 4.1: using cascade technology, a plurality of detection heads are connected in series on the detection head to obtain a cascade detection head prediction, wherein the processing of the first n-1 layers of the decoder is:

[0032] DeLn 0：n-1 =Transformer(last out ,Q,Vis)

[0033] The processing of the m prediction layers of the cascade part is:

[0034] DeLnn-1：m+n-1 = Transformer(last out , last out , Q, Vis)

[0035] wherein last out represents the output of the last layer of the main branch, Q represents the query array of the main branch, Vis represents the visual feature output by the backbone network, last’ out represents the gradient truncation object of the output of the last layer;

[0036] Step 4.2: using a parallel auxiliary task, sharing the weights of the decoder in the detector, using different types to calculate the target detection loss of the objective function, obtaining an optimized shared weight decoder, wherein the training target is:

[0037]

[0038] wherein X represents the input picture, Y represents the detection label, f represents the backbone network encoder function, MSA represents the multi-scale adapter, MT i represents the loss function of the corresponding task, λ i represents the weight parameter.

[0039] In the above scheme, step 5 comprises:

[0040] Step 5.1: for the dynamic attention mechanism, using a window attention and global attention hybrid technique, obtaining a spatial scale adaptive attention module, wherein the hidden variable of the image block is represented as:

[0041] h (patch) = FC (PC (DWConv (xpatch) ) )

[0042] The window attention calculation is:

[0043]

[0044] The number of regions is predicted as:

[0045] k = DF (h)

[0046] The spatial scale adaptive attention module is:

[0047]

[0048] wherein x patch represents the image block, DWConv represents the depth convolution, PC represents the point convolution, FC represents the full connection layer, FC q and FC kdenotes a fully connected layer for feature mining, d denotes a feature dimension, DF denotes a distributed prediction function, AG denotes a regional feature fusion operator, and k denotes a selected number of regions;

[0049] Step 5.2: replacing the spatial scale adaptive attention module with the shallow layer of the QWEN2.5-VL model encoder to obtain an encoder of the environment perception model;

[0050] Step 5.3: combining a semantic decoder to obtain an environment perception head model.

[0051] Because the application adopts the above technical means, the following beneficial effects are achieved:

[0052] 1. The application adopts the SwinL model as the backbone network backbone in step 1, pre-trains the device recognition model through the device detection task and the feature mask reconstruction task, and uses the pre-trained device recognition model as the baseline model for all defect recognition, which solves the technical problems of model parameter redundancy and repeated waste of computing resources in the prior art when different device types are recognized, and achieves the effect of eliminating redundant calculation in full-scene defect recognition by sharing the device recognition model backbone.

[0053] 2. The application uses the multi-expert routing technology in step 2 to distribute the local feature representation output by the backbone network backbone to the corresponding expert model, which solves the technical problem of index imbalance caused by the single task model in the prior art that cannot adapt to different defect type recognition requirements, and achieves the effect of dynamically allocating expert models for targeted defect recognition according to the characteristics of device defects.

[0054] 3. The application uses the enhanced FPN framework to fuse the multi-level features of the backbone network backbone in step 3 to obtain a spatially adaptive classification head model, which solves the technical problem of limited feature expression ability caused by the small number of learnable parameters of the classification model in the prior art, and achieves the effect of enhancing the feature mining ability of the classification model for part and instance level defects by fusing multi-level features. 4. The application uses the multi-dimensional auxiliary detection head model generated by cascading the detection head and the parallel auxiliary task technology in step 4, which solves the technical problem of insufficient detection rate and accuracy of the detection model caused by the single learning goal in the prior art, and achieves the effect of improving the robustness and recognition accuracy of the detection model for device level defects by cascading prediction and multi-task assistance. 5. The application uses the spatial scale adaptive attention mechanism to generate an environment perception head model in step 5, which solves the technical problem of low recognition accuracy of feature non-aggregated defects (such as environment perception defects) caused by weak semantic correlation in the prior art, and achieves the effect of enhancing the environmental semantic perception ability by dynamic attention mechanism to improve the overall defect recognition effect. Attached Figure Description

[0055] Figure 1 Multi-expert routing diagram;

[0056] Figure 2 A diagram illustrating classification experts;

[0057] Figure 3 Detection expert route diagram;

[0058] Figure 4 Schematic diagram of the spatial scale adaptive attention encoder structure. Detailed Implementation

[0059] The embodiments of the present invention will be described in detail below. Although the present invention will be described and illustrated in conjunction with some specific embodiments, it should be noted that the present invention is not limited to these embodiments. On the contrary, any modifications or equivalent substitutions made to the present invention should be covered within the scope of the claims of the present invention.

[0060] Furthermore, to better illustrate the present invention, numerous specific details are set forth in the following detailed embodiments. Those skilled in the art will understand that the present invention can be practiced without these specific details.

[0061] A method for defect identification during unmanned aerial vehicle (UAV) patrols in power transmission and transformation based on an isomorphic model includes:

[0062] Step 1: Use the SwinL model as the backbone of the isomorphic model, and pre-train it through the device detection task and feature mask reconstruction task. Use the pre-trained device recognition model as the baseline model for all defect recognition.

[0063] Step 2: For the baseline model, use multi-expert routing technology to distribute the local feature representations output by the backbone to the corresponding expert models;

[0064] Step 3: For component, instance-level defects and local feature representations, use the enhanced FPN framework to fuse multi-level features of the backbone to obtain a spatial adaptive classification head model and output the spatial adaptive classification result;

[0065] Step 4: Based on the equipment-level defects and regional feature representations, auxiliary branching techniques are used to obtain a multi-dimensional auxiliary detection head model for the identification of local defects such as corrosion, displacement, and detachment;

[0066] Step 5: For environmental perception defects and scene feature representations, use a dynamic attention mechanism to obtain an environmental perception head model for global defect identification, such as vibration damper slippage.

[0067] In the above scheme, step 1 includes the following steps:

[0068] Step 1.1, for the pre-training, uses industry data and multi-task fine-tuning technology to train a full-category equipment identification model for the power industry, which is used as a baseline model for all defect identification; (sharing the backbone of the equipment identification model, realizing the elimination of redundant calculation of full-scene defect identification)

[0069] In the baseline model, SwinL is used as the backbone of the model, and two different types of pre-training tasks are connected after the backbone. The first is a detection pre-training model, which uses the most optimal detection scheme in the neck and head part, such as the decoder and auxiliary head of CO-DETR. Using high-precision labeled equipment data in the industry as training data, the model learns to identify equipment. The second is a feature mask reconstruction task, which randomly masks some feature points on the feature representation of the backbone, and uses MIM-type models to reconstruct the image with the remaining feature points, so that the model can learn semantic features other than equipment to prevent overfitting on equipment, which leads to lack of defect identification transferability.

[0070] Where, for the input image X, after encoding by the backbone, the encoded features are given to the detection head for equipment detection; by sparse sampling of the encoded features, the mask reconstruction model such as MAGE is used to reconstruct the image. Its training follows the following formula:

[0071] L total =L d (g(f(X)),Y)+λ·||(h(f(X)⊙M)-x)⊙(1-M)||1

[0072] Where, f is the backbone encoder function, g is the detection head, h is the mask reconstruction model, M is a binary mask matrix representing the area to be masked. is the loss weighting weight of the two tasks.

[0073] In the above scheme, step 2 includes the following steps:

[0074] Step 2.1, for the equipment identification model and backbone feature representation, uses multi-expert routing technology to obtain a backbone feature distributor; (in power transmission defect identification, according to the equipment category, select the appropriate task type for routing distribution)

[0075] In the power industry, we can register the type of defect identification that the equipment needs to perform according to the category, function, and other dimensions of the equipment. When registering, the equipment can bind multiple expert models, and after defect identification, the final decision module is used to judge the defect report, and the schematic diagram is as follows Figure 1.

[0076] Step 2.1.1. According to the characteristics of the defects contained in the power transmission and transformation equipment, obtain the equipment-route mapping table for the equipment.

[0077] In the defect identification of power transmission and transformation equipment, one equipment often has multiple defects to be identified. For example, the suspension clamp device unit, we need to focus on ship body rust, bolt shedding, ship body offset, etc. Among them, the ship body rust defect is characterized by local area features, which can be mapped to the classification head model (classification expert route), and also can be mapped to the detection head model (detection expert route). The ship body offset defect has two cases of significant offset and suspected offset, the former can be processed by the classification expert route and the detection expert route, and the latter can only be processed by the perception head model (perception expert route). Similarly, we can process other equipment defects to obtain a one-to-many "equipment-route" mapping table. After obtaining the mapping table, configure the task identification result aggregation method for each equipment, such as taking intersection or union.

[0078] Step 2.1.2. For the baseline model equipment identification result, assign the corresponding equipment area to the expert model according to the pre-defined equipment-route mapping relationship, and obtain the backbone feature distributor.

[0079] The feature distributor, according to the obtained feature area label and route type, adaptively scales the task area size, and identifies the defects of the equipment in the area. The identification result is divided into target equipment and non-target equipment identification result. For the target equipment, use the pre-defined equipment aggregation method to fuse the identification result. For the non-target equipment, it is summarized into a global non-target equipment, and the target equipment is excluded to prevent duplication. Apply the equipment aggregation method to the global non-target equipment set, set the hyperparameters to filter the fusion target, and obtain the final identification supplement. Merge the target equipment identification result and the non-target equipment identification result to obtain the identification result of the equipment of the isomorphic model.

[0080] In the above scheme, step 3 includes the following steps:

[0081] Step 3.1. For the backbone feature representation, use the FPN technology to obtain the fused feature representation.

[0082] The backbone has cn intermediate layer result outputs, and the traditional classification model uses the result of the last layer for feature reprocessing. This mode will cause the loss of basic shallow features, and these features are very important for semantic discrimination. In the detection model, FPN, PAN and other structures are introduced to fuse features of different levels for dense prediction. Based on the enlightenment of the detection model, a simple FPN structure is used to fuse the features, which is the same as the traditional FPN, and a large kernel convolution is used in the connection operator to expand the receptive field and window attention of the corresponding layer to enhance the local feature representation. Using two kinds of operators is to excavate features from different dimensions on the one hand, and to increase the parameter quantity of the model on the other hand. Therefore, we increase the traditional FPN from 2cn modules to 3cn modules to ensure that the model has enough parameter quantity to learn classification features.

[0083] fpn i = CNN (WindowAttn (b outi ))

[0084] Wherein, fpn i is the direct processing of the i-th output of the backbone. The fusion process is:

[0085] fpn i-1 = fpn i-1 + Interprolate (fpn i ), i > 1

[0086] Interprolate is an interpolation function for the i-th fpn feature, which scales the spatial scale of the feature to be similar to the i-th fpn feature Figure 1 , and the fused feature is obtained by point-by-point addition. The final output will be processed as follows:

[0087] FPN out = Concat ([AP (LCNN (fpn0)), …, AP (LCNN (fpn cn ))])

[0088] Wherein, AP is an adaptive pooling operator, and LCNN is a layer convolution module.

[0089] Step 3.2 uses fully connected convolution technology to obtain the classification model output according to the fused feature representation;

[0090] After obtaining FPN out in FPN, the processing formula is:

[0091] category = Sigmoid (FC (FPN out ))

[0092] Where FC function is full connection processing to FPN output, Sigmoid function is a smooth processing to numerical value, and finally using softmax to category can get the prediction probability of each category.

[0093] The schematic diagram of the classification expert routing is as shown in Figure 2 .

[0094] In the above scheme, step 4 includes the following steps:

[0095] Step 4.1 uses cascade technology for the so-called auxiliary branch technology, and more detection heads are connected in series on the detection head to obtain a cascade detection head prediction.

[0096] Recently, the prediction performance of the DETR head has been greatly improved after a large number of improvements, and the target detection related list has always been in the top few. In order to improve the recall ability of one-to-one output, this patent uses cascade technology to stack the configuration of the last output layer m times on the n-layer decoder, so we will get a m+n-1 layer cascade decoder.

[0097] The first n-1 layer decoder is processed as:

[0098] DeLn 0：n-1 =Transformer(last out ,Q,Vis)

[0099] Where Q is the QUERY array of the main branch, last out is the output of the last layer of the main branch, Vis is the visual feature output by the backbone, and Transformer is the cross attention module. The m prediction layers of the cascade part are processed as:

[0100] DeLn n-1：m+n-1 =Transformer(last out ,last out ,Q,Vis)

[0101] Where last’ out is the gradient truncated object of the output of the last layer. Because the gradient is truncated, it is convenient to quickly stabilize the current layer during training. In order to let the model pass the gradient value between different layers, the loss of the current layer, one branch passes the gradient value in the current layer to update the parameters of the current layer; one branch is used to update the parameters of the last layer, that is, the last m prediction layers use look-forward-twice.

[0102] During training, all m+n-1 layers of the decoder need to align with the labels, but only the last m layers are involved in prediction during inference. The schematic diagram is as follows Figure 3 .

[0103] Step 4.2 uses parallel auxiliary tasks for the so-called auxiliary branch technology, shares the weights of the decoder in the detector, uses different types to calculate the target detection loss of the target function, and obtains an optimized shared weight decoder;

[0104] In step 4.1, the number of target alignment is mainly enhanced, and the representation potential of one-to-one prediction is enhanced, because in the detection field, dense prediction is an important means to preserve recall, which is complementary to the shortcomings of traditional one-to-one. In the CO-DETR paper, it has been proved that using parallel mixed one-to-many tasks can greatly improve the recognition ability of the model under the premise of sharing the weights of the encoder. Inspired by CO-DETR, we integrate ATSS detection head, FCOS detection head, FasterRCNN detection head, RetinaNet detection head, and CornerNet detection head. After multi-head alignment, the unstable problem of Hungarian matching in one-to-one prediction is effectively solved, ensuring fast convergence and improving learning ability of the model.

[0105] Let MT=(ATSS,FCOS,FasterRCNN,RetinaNet,CornerNet) represent different task sets, and use the weight parameter λ=(λ1,λ2,λ3,λ4,λ5) to harmonize different types of target alignment loss. The training target of the parallel task auxiliary task can be designed as:

[0106]

[0107] Where is the MT i loss function of the corresponding task, X represents the input picture, Y represents the detection label, f is the backbone encoder function, and MSA is a multi-scale adapter for adapting features of different scales for different tasks. After the assistance of this one-to-many detection head, a large number of target detection is only injected into the one-to-one branch through these parallel branches, and the implementation is only accumulation, which helps the DETR head to quickly and stably converge.

[0108] In the above scheme, step 5 includes the following steps:

[0109] Step 5.1 uses a window attention and global attention hybrid technology for the dynamic attention mechanism to obtain a spatial scale adaptive attention module;(joint global and local attention, joint self-attention and cross-attention, generate spatial scale adaptive attention structure)

[0110] With the transformer paper proposing self-attention, the attention module has undergone long-term development, and the QKV computing mechanism still occupies the dominant position of the deep learning basic module. In order to solve the problem of exponential growth of transformer module calculation, researchers recently developed Mamba, WindowAtten, DLformer and other structures. Mamba community introduces state space, reduces the operator complexity to linear complexity. WindowAtten technology performs transformer calculation in a small window in vision, achieving a balance between efficiency and performance. In the QWEN2.5-VL model, the model architecture uses a hybrid of Full Attention and Window Attention in the encoder stage to improve the ability of global perception and provide a shortcut for feature coding of sparse correlation semantics. DLformer is an architecture that predicts the remaining useful life from fine granularity to coarse granularity. Because the feature representation of coarse granularity needs to be fused with the feature of fine granularity, feature reuse is realized, and the contribution of different time steps is weighted to improve performance.

[0111] Inspired by the above three technologies, the basic module of the environmental perception model of the application first encodes each patch of the image into a hidden space, where the hidden variable of the patch is represented as:

[0112] h(patch)=FC(PC(DWConv(x patch )))

[0113] DWConv is a deep convolution, PC is a point convolution, and after two convolutions, a local feature map is obtained, the channel number is 1, and Flatten is used to expand the space. Finally, the hidden space feature representation is obtained by using full connection.

[0114] The global patchAtten is constructed using the hidden representation of the patch:

[0115]

[0116] Where FC is used for feature mining to form different feature representations, and patchAtten is obtained by using the self-attention calculation formula. In the sequence, another controllable parameter k is needed, which represents the number of patches collected for subsequent attention calculation. The calculation of k is as follows:

[0117] k=DF(h)

[0118] where DF is the distributed prediction function, and DFL is similar to the technique, we preset the bias term, and then predict the probability of these terms to achieve the prediction of the number of regions. Based on the output k and patchAtten, each patch selects the top K other patches to update the current region feature, so we get a cross-region spatial adaptive attention module:

[0119]

[0120] where AG is the operator that fuses the TopK region features, such as directly adding them in the channel, and then scaling. Thus we get the spatial scale adaptive attention module.

[0121] Step 5.2 replaces the encoder shallow layer of QWEN2.5-VL for the spatial scale adaptive attention module to obtain the encoder of the environment perception model;

[0122] The apAtten layer is used in the first half of the encoder, and the local attention encoding layer in the QWEN2.5-VL model is used in the subsequent layers, and the global attention encoding layer is used in the last layer. Thus we get an encoder that enhances the correlation of shallow features and environmental semantic perception, and its structure is shown in Appendix 4.

[0123] Step 5.3 combines a semantic decoder for the encoder of the environment perception model to obtain the environment perception head model;(normal operation)

[0124] The encoder has been mined several times in the modeling of cross-space region features, and then the self-attention mechanism is used to fully mine the visual semantics, and finally the global attention is used to optimize and adjust the representation of these features. In the decoding stage, we use a mature visual decoder to obtain the discriminant analysis of devices and their attributes in the environment.

Claims

1. A method for identifying defects during unmanned aerial vehicle (UAV) patrols in power transmission and transformation based on an isomorphic model, characterized in that, include: Step 1: The SwinL model is used as the backbone network of the isomorphic model. It is pre-trained through the device detection task and the feature mask reconstruction task. The pre-trained device recognition model is used as the baseline model for all defect recognition. Step 2: For the baseline model, use multi-expert routing technology to distribute the local feature representations output by the backbone network to the corresponding expert models; Step 3: For component, instance-level defects and local feature representations, use the enhanced FPN framework to fuse multi-level features of the backbone network to obtain a spatial adaptive classification head model and output the spatial adaptive classification result; Step 4: For equipment-level defects, a multi-dimensional auxiliary detection head model is generated using cascaded detection heads and parallel auxiliary task technology, and the local defect identification results are output. Step 5: For environmental perception defects, generate an environmental perception head model through a spatial scale adaptive attention mechanism and output global defect identification results.

2. The method according to claim 1, characterized in that, include: The SwinL model is used as the backbone network of the isomorphic model. It is pre-trained using device detection and feature mask reconstruction tasks. The loss function for pre-training is: L total =L d (g(f(X)),Y)+λ·||(h(f(X)⊙M)-x)⊙(1-M)||1 Where f is the backbone network encoder function, g is the detector head, h is the mask reconstruction model, M is the binary mask matrix, representing the region to be masked, and λ is the weighted weight of the losses from the two tasks.

3. The method according to claim 1, characterized in that, Step 2 includes: Step 2.1: Based on the defect characteristics of the power transmission and transformation equipment, obtain the equipment-route mapping relationship table, wherein the equipment-route mapping relationship table is a one-to-many mapping relationship; Step 2.2: Assign the device regions output by the baseline model to the corresponding expert models according to the device-routing mapping relationship to obtain the backbone network feature distributor.

4. The method according to claim 1, characterized in that, The feature distributor adaptively scales the task area size based on the feature region label and routing type, and divides the identification results into target device identification results and non-target device identification results; The target device identification results are then fused using a predefined device aggregation method. The non-target device identification results are summarized into a global non-target device set. After excluding target devices, a device aggregation method is applied and hyperparameters are set for filtering. The target device identification results and non-target device identification results are merged to obtain the device identification results.

5. The method according to claim 1, characterized in that, In step 3, for component-level defects and local feature representations, the enhanced FPN framework is used to fuse multi-level features of the backbone to obtain a spatially adaptive classification head model. The fusion process of the enhanced FPN framework includes: fpn i =CNN(WindowAttn(b outi )) fpn i-1 =fpn i-1 +Interprolate(fpn i ),i>1 FPN out =Concat([AP(LCNN(fpn0)),…,AP(LCNN(fpn cn ))]) Among them, b outi Let represent the i-th level output feature of the backbone network, CNN represent a convolutional neural network, WindowAttn represent a window attention mechanism, Interprolate represent an interpolation function, AP represent an adaptive pooling operator, LCNN represent a layer convolutional module, and cn represent the number of intermediate layers in the backbone network; and The classification model output is obtained using fully connected convolutional techniques, and the calculation process is as follows: category=Sigmoid(FC(FPN out )) Here, FC represents a fully connected layer, and Sigmoid represents an activation function.

6. The method according to claim 1, characterized in that, Step 4 includes: Step 4.1: Using cascade technology, multiple detectors are connected in series on the detector head to obtain cascaded detector predictions. The processing of the first n-1 layers of the decoder is as follows: Part 0：n-1 =Transformer(load out ,Q,Vis) The processing of the m prediction layers in the cascade part is as follows: Part n-1：m+n-1 =Transformer(load out ,load out ,Q,Vis) Among them, last out The last element represents the output of the layer above the main branch, Q represents the query array of the main branch, Vis represents the visual features output by the backbone network, and last represents the output of the previous layer. out This represents the gradient cutoff object of the previous layer's output; Step 4.2: Using a parallel auxiliary task, the weights of the decoder in the shared detector are shared, and the object detection loss is calculated using different types of objective functions to obtain the optimized shared weight decoder. The training objective is: Where X represents the input image, Y represents the detection label, f represents the backbone network encoder function, MSA represents the multi-scale adapter, and MT... i Let λ represent the loss function for the corresponding task. i This represents the weighting parameter.

7. The method according to claim 1, characterized in that, Step 5 includes: Step 5.1: For the dynamic attention mechanism, a hybrid technique of window attention and global attention is used to obtain a spatial scale adaptive attention module, where the latent variables of the image patch are represented as: h(patch)=FC(PC(DWConv(x patch ))) Window attention is calculated as follows: The number of regions is predicted to be: k = DF(h) The spatial adaptive attention module is: Where, x patch Represents an image patch, DWConv represents depthwise convolution, PC represents pointwise convolution, FC represents a fully connected layer, and FC... q and FC k denoted as the fully connected layer used for feature mining, d represents the feature dimension, DF represents the distributed prediction function, AG represents the region feature fusion operator, and k represents the number of regions selected. Step 5.2: Replace the shallow layer of the QWEN2.5-VL model encoder with the spatial scale adaptive attention module to obtain the encoder of the environment perception model; Step 5.3: Combine the semantic decoder to obtain the environment-aware head model.

Images