Power transmission channel hidden danger identification method and system fusing optical and radar remote sensing

Abstract

The invention belongs to the technical field of power transmission channel hidden danger recognition, and provides a power transmission channel hidden danger recognition method and system fusing optical and radar remote sensing, and the method comprises the steps: constructing a power transmission channel typical hidden danger feature extraction model based on an improved UNet, and carrying out the feature extraction of optical remote sensing data, and obtaining an optical image feature vector; constructing a power transmission channel typical hidden danger feature extraction model based on the improved TransUNet, and performing feature extraction on the radar remote sensing data to obtain a radar image feature vector; performing feature fusion on the optical image feature vector and the radar image feature vector to obtain a fusion feature; the fusion features are sorted and selected based on feature importance, and a classification probability result is obtained in combination with a mixed loss function. According to the method, image reconstruction from low resolution to high resolution is realized through the multi-channel feature map, and the high-resolution feature map is generated, so that more detail information is reserved, and the feature extraction precision is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of power transmission channel hidden danger identification, and in particular relates to a power transmission channel hidden danger identification method and system integrating optical and radar remote sensing. Background Art

[0002] The characteristics of the transmission line network, such as large span, wide coverage, and complex terrain, have undoubtedly increased the difficulty of safety inspections of transmission channels, especially in recent years with the rapid increase in the scale of the transmission line network. In extreme weather such as strong winds and illegal operations, typical hidden dangers in transmission channels include floating objects, construction operations, and offline buildings (structures), which may cause foreign objects to cover the transmission lines and damage the transmission lines, thereby causing power failures. Therefore, intelligent identification of typical hidden dangers in transmission channels is particularly important.

[0003] At present, the traditional on-site field inspection mode can no longer meet the needs of large-scale transmission channel hidden danger inspection work, and a new inspection mode is urgently needed. As a type of high-tech, remote sensing technology has the advantages of large range, less environmental constraints, objectivity and accuracy. The rapid development of this technology provides a new solution for the inspection of typical hidden dangers in transmission channels. However, due to the limitation of satellite transit time, a single satellite remote sensing data source cannot achieve all-day monitoring. For some short-term strong wind extreme weather, if it cannot be captured in time, it may cause sudden transmission line tripping events; at the same time, for small-scale external damage hidden danger information, the monitoring effect of low spatial resolution satellite remote sensing data is unstable. In addition, optical satellite remote sensing data is greatly affected by cloud cover. In cloudy or rainy weather, it is difficult to achieve effective monitoring and early warning of transmission line external damage hidden danger information and foreign objects in the conductor. Summary of the invention

[0004] The purpose of the present invention is to provide a method and system for identifying hidden dangers in power transmission channels by integrating optical and radar remote sensing, so as to solve the problem that single data monitoring is greatly affected by the external environment and it is difficult to achieve effective early warning monitoring.

[0005] To achieve the above object, the present invention adopts the following technical solutions: In a first aspect, the present invention provides a method for identifying hidden dangers in a power transmission channel by integrating optical and radar remote sensing, comprising: A typical hidden danger feature extraction model for power transmission channels based on improved UNet is constructed to extract features from optical remote sensing data and obtain optical image feature vectors. A typical hidden danger feature extraction model for power transmission channels based on improved TransUNet is constructed to extract features from radar remote sensing data and obtain radar image feature vectors. Perform feature fusion on the optical image feature vector and the radar image feature vector to obtain fusion features; The fusion features are selected based on the feature importance ranking and combined with the hybrid loss function to obtain the classification probability result.

[0006] Optionally, the construction of a typical hidden danger feature extraction model for a power transmission channel based on an improved UNet specifically includes: replacing the encoder of the original UNet with a multi-scale deformable convolution module, and replacing the deconvolution in the original UNet decoder with sub-pixel convolution.

[0007] Optionally, feature extraction is performed on the optical remote sensing data to obtain an optical image feature vector, specifically including: The deformable convolution used is calculated as follows: (1) Where y represents the feature map after convolution calculation, x represents the input feature map, and m 0 represents the position on the feature map, m i represents a specific sampling point in the sampling grid G, Δm i Indicates the offset that needs to be learned; by adding additional offsets, the deformable convolution can learn the shape characteristics of the hidden danger objects in the power transmission channel; In the upsampling process, the sub-pixel convolution strategy is selected to achieve image reconstruction from low resolution to high resolution through multi-channel feature maps. The input of the sub-pixel convolution is the low-resolution feature map H×W. First, the image features are extracted to obtain n 1 feature maps, after multiple operations, are adjusted to r in the last convolution. 2 channel, so that the total number of pixels is consistent with the high-resolution image to be generated, and finally r 2 The feature maps of the channels are recombined into the upsampling results of W×r and H×r to generate a high-resolution feature map rH×rW×C, where r is the upsampling scaling factor; sub-pixel convolution is used to convert the features from N×(C×r×r)×W×H to the form of N×C×(H×r)×(W×r).

[0008] Optionally, the construction of a typical hidden danger feature extraction model for power transmission channels based on improved TransUNet, performing feature extraction on radar remote sensing data, and obtaining radar image feature vectors specifically includes: The TransUNet network model is used as the basic network structure. The overall structure retains the encoder and decoder structure of the semantic segmentation model to form a "U"-shaped network. The left half of the model is the encoder part, which adopts the 2D convolution form and increases the number of filters layer by layer. The number of filter channels is 64, 128, 256, and 512 respectively. The image features are extracted through a four-layer downsampling structure. A feature extraction residual module is designed in the encoder part for learning spectral, spatial and other features. The module includes a residual connection part and 2D convolutions of 1×1, 3×3, and 1×1 convolution kernels, and the convolution is performed separately during the convolution. The activation layer and batch normalization layer are operated; then the 12 Transformer layers of the original TransUNet model are reduced to 6 Transformer layers; after that, the spatial-channel attention mechanism module is introduced to improve the accuracy of feature learning; the right half of the model is the decoder part, which uses the form of transposed convolution to reduce the number of filters layer by layer, restore the image resolution to achieve pixel-by-pixel classification, and use high- and low-level feature fusion to fuse with the feature layer obtained by the encoding part, and replace the bilinear interpolation upsampling layer of the TransUNet decoder part with sub-pixel convolution for upsampling.

[0009] Optionally, the step of fusing the optical image feature vector and the radar image feature vector to obtain a fused feature specifically includes: The optical image feature vector and the radar image feature vector are stacked, convolved, separated and then multiplied with the original features to achieve adaptive selection of different branch features on a three-dimensional scale. Finally, the branch feature maps assigned with attention weights are stacked and passed through a convolution layer to obtain the final fusion feature.

[0010] Optionally, the fusion features are selected based on feature importance sorting to obtain a classification probability result, specifically including: From the two types of features, features with large information content and conducive to classification are selected. After fusion, the features are input into the fully connected layer to obtain the feature vector, and then input into the softmax classifier to obtain the final classification probability result.

[0011] Optionally, the hybrid loss function is specifically: Firstly, an improved loss function Log-Cosh Dice Loss for multi-classification problems is introduced. The region-based Log-Cosh Dice Loss loss function improves the non-convexity of Dice Loss. The calculation formula is as follows, where Set to 0.5; (2) Among them, Loss represents the improved hybrid loss function of the present invention, CELoss represents the cross entropy loss function, Llc-dce represents the Log-Cosh Dice Loss loss function, and α represents the fusion coefficient.

[0012] in, Loss represents the improved hybrid loss function of the present invention, CELoss represents the cross entropy loss function, L lc-dce represents the Log-Cosh Dice Loss function, α Represents the fusion coefficient.

[0013] In a second aspect, the present invention provides a power transmission channel hidden danger identification system integrating optical and radar remote sensing, comprising: The optical image feature extraction module is used to build a typical hidden danger feature extraction model for power transmission channels based on the improved UNet, extract features from optical remote sensing data, and obtain optical image feature vectors; The radar image feature extraction module is used to build a typical hidden danger feature extraction model for power transmission channels based on the improved TransUNet, extract features from radar remote sensing data, and obtain radar image feature vectors; A feature fusion module is used to fuse the optical image feature vector and the radar image feature vector to obtain a fusion feature; The sorting output module is used to select the fusion features based on the feature importance sorting, combined with the hybrid loss function, to obtain the classification probability result.

[0014] In a third aspect, the present invention provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method for identifying hidden dangers in power transmission channels by integrating optical and radar remote sensing when executing the computer program.

[0015] In a fourth aspect, the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the method for identifying hidden dangers in power transmission channels by integrating optical and radar remote sensing.

[0016] Compared with the prior art, the present invention has the following technical effects: The present invention integrates multi-scale deformable convolution module and sub-pixel convolution, and the model significantly improves the feature extraction and reconstruction capabilities of optical remote sensing images. The multi-scale deformable convolution module adds an additional offset so that the convolution can learn the shape characteristics of hidden danger objects in the transmission channel, which is crucial for identifying hidden dangers in complex terrain and backgrounds. The sub-pixel convolution strategy achieves image reconstruction from low resolution to high resolution through multi-channel feature maps during the upsampling process, generating high-resolution feature maps, thereby retaining more detail information and improving the accuracy of feature extraction.

[0017] The model is based on the improved TransUNet network. By reducing the number of Transformer layers, introducing a spatial-channel attention mechanism module, and replacing the upsampling layer with a sub-pixel convolution, the feature extraction capability of radar remote sensing data is significantly improved. The introduction of residual connections and attention mechanisms enhances the model's learning ability for spectral, spatial and other features, while improving the model's robustness to variations in the shape and scale of hidden dangers. These improvements enable the model to more accurately identify transmission channel hazards in radar remote sensing data.

[0018] The present invention designs a data fusion model for the identification of typical hidden dangers in power transmission channels based on multi-source multi-modal optical and radar remote sensing features. The model realizes the feature-level fusion of optical image feature vectors and radar image feature vectors through steps such as feature stacking, convolution calculation, and feature separation. This fusion method not only retains the unique information of each modality, but also realizes the complementarity between different modalities, thereby improving the identification accuracy of typical hidden dangers in power transmission channels.

[0019] In order to accurately identify typical hidden dangers of power transmission channels, the present invention designs a hybrid loss function that integrates the cross entropy loss function and the regression Dice loss. Among them, the introduction of Log-Cosh Dice Loss improves the non-convexity of Dice Loss, making the model more stable during the training process. At the same time, by setting a suitable α value (such as 0.5), the classification accuracy and regression accuracy can be balanced, thereby further improving the recognition performance of the model.

[0020] In summary, the present invention improves recognition accuracy: through improvements such as multi-scale deformable convolution, sub-pixel convolution, attention mechanism, and the fusion of multi-source remote sensing features, the recognition accuracy of typical hidden dangers in power transmission channels is significantly improved.

[0021] Enhanced robustness: The model’s robustness to variations in hidden danger shape and scale has been significantly enhanced, enabling more accurate identification of hidden dangers in complex terrain and backgrounds.

[0022] Improved stability: The improved loss function makes the model more stable and converges faster during training, thereby improving the practicality and reliability of the model.

[0023] Expand the scope of application: This technical solution is not only suitable for the identification of hidden dangers in power transmission channels, but can also be extended to remote sensing image recognition and analysis in other fields, and has broad application prospects.

[0024] This technical solution realizes the feature extraction, fusion and classification of multi-source and multi-modal remote sensing data by integrating the transmission channel hidden danger identification methods of optical and radar remote sensing, significantly improving the identification accuracy and robustness of typical hidden dangers in transmission channels, and providing strong guarantee for the safe operation of power systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 The overall technical route of the present invention.

[0026] Figure 2 Schematic diagram of the multi-scale deformable convolution module.

[0027] Figure 3 Schematic diagram of the typical hidden danger feature extraction model for power transmission channels based on radar remote sensing.

[0028] Figure 4 Schematic diagram of the multimodal optical and radar image feature fusion model.

[0029] Figure 5 Schematic diagram of the prediction results of the present invention on a small area.

[0030] Figure 6 Schematic diagram of the prediction results of the present invention over a large area.

[0031] Figure 7 Flowchart of the present invention. DETAILED DESCRIPTION

[0032] The present invention is further described below in conjunction with the accompanying drawings: Example 1, please refer to Figure 7 The present invention provides a method for identifying hidden dangers in power transmission channels by integrating optical and radar remote sensing, comprising: A typical hidden danger feature extraction model for power transmission channels based on improved UNet is constructed to extract features from optical remote sensing data and obtain optical image feature vectors. A typical hidden danger feature extraction model for power transmission channels based on improved TransUNet is constructed to extract features from radar remote sensing data and obtain radar image feature vectors. Perform feature fusion on the optical image feature vector and the radar image feature vector to obtain fusion features; The fusion features are selected based on the feature importance ranking and combined with the hybrid loss function to obtain the classification probability result.

[0033] The present invention is designed for feature extraction model of optical remote sensing, which realizes feature extraction and reconstruction of optical remote sensing images from low resolution to high resolution through multi-channel feature maps by fusing multi-scale deformable convolution modules, sub-pixel convolution, etc.

[0034] Design of feature extraction model for radar remote sensing. A radar remote sensing feature extraction model based on residual connection and attention mechanism is constructed to improve the robustness of the model to hidden danger shape and scale variations.

[0035] According to the characteristics of multi-source and multi-modal optical and radar remote sensing, a data fusion model for the identification of typical hidden dangers in power transmission channels is designed to realize the feature-level fusion of multi-source heterogeneous data, so as to improve the identification accuracy of typical hidden dangers in power transmission channels.

[0036] Embodiment 2, the present invention provides a method for identifying hidden dangers in power transmission channels by integrating optical and radar remote sensing, which specifically includes: Design of multi-source remote sensing feature extraction model 1. Typical hidden danger feature extraction model for power transmission channels based on optical remote sensing For optical remote sensing images, the present invention constructs a typical hidden danger feature extraction model for power transmission channels based on improved UNet. The specific improvements include: replacing the encoder of the original UNet with a multi-scale deformable convolution module, and replacing the deconvolution in the original UNet decoder with sub-pixel convolution.

[0037] Specifically, the multi-scale deformable convolution module constructed by the present invention is a multi-level parallel structure ( Figure 2 ), which can effectively learn the multi-scale characteristics of hidden dangers, thereby improving the robustness of the model to scale changes.

[0038] Compared with ordinary convolution, deformable convolution adds a spatial position sampling module, and improves its adaptability to hidden danger shape and scale variations by learning additional offsets. The calculation method of the deformable convolution used in the present invention is as follows: (1) Where y represents the feature map after convolution calculation, x represents the input feature map, and m 0 represents the position on the feature map, m i represents a specific sampling point in the sampling grid G, Δm i Represents the offset that needs to be learned. By adding additional offsets, the deformable convolution can learn the shape characteristics of hidden danger objects in the transmission channel, thereby enhancing the robustness of spatial feature extraction in complex environments.

[0039] At the same time, the present invention selects to use the sub-pixel convolution strategy in the upsampling process, and realizes image reconstruction from low resolution to high resolution through multi-channel feature maps. Compared with ordinary deconvolution for signal recovery, the operation of the present invention can reduce information loss and improve the image recovery effect in the upsampling process. Specifically, the input of the sub-pixel convolution is a low-resolution feature map H×W. First, the image features are extracted to obtain n 1 feature maps, after multiple operations, are adjusted to r in the last convolution. 2 channel, so that the total number of pixels is consistent with the high-resolution image to be generated, and finally r 2 The feature maps of the channels are recombined into the upsampling results of W×r and H×r to generate a high-resolution feature map rH×rW×C, where r is the upsampling scaling factor. Sub-pixel convolution is used to convert the features from N×(C×r×r)×W×H to N×C×(H×r)×(W×r).

[0040] 2. Typical hidden danger feature extraction model for power transmission channels based on radar remote sensing Based on radar remote sensing images, this paper constructs a typical hidden danger feature extraction model for power transmission channels based on improved TransUNet ( Figure 3 ). The model uses the TransUNet network model as the basic network structure. The overall structure retains the encoder and decoder structure of the semantic segmentation model to form a "U"-shaped network. The left half of the model is the encoder part, which uses 2D convolution to increase the number of filters layer by layer. The number of filter channels is 64, 128, 256, and 512 respectively. The image features are extracted through a four-layer downsampling structure. A feature extraction residual module is designed in the encoder part for learning spectral, spatial and other features. The module includes a residual connection part and 2D convolutions with 1×1, 3×3, and 1×1 convolution kernels, and the activation layer and batch normalization layer operations are performed during the convolution. Then the 12 Transformer layers of the original TransUNet model are The model is reduced to 6 Transformer layers to optimize the gradient vanishing problem in the training process; then, the spatial-channel attention mechanism module is introduced to improve the accuracy of feature learning; the right half of the model is the decoder part, which uses the form of transposed convolution to reduce the number of filters layer by layer, restore the image resolution to achieve pixel-by-pixel classification, and use high- and low-level feature fusion to fuse with the feature layer obtained by the encoding part. In order to avoid the checkerboard effect in the semantic segmentation results, the bilinear interpolation upsampling layer of the TransUNet decoder part is replaced by sub-pixel convolution for upsampling.

[0041] Design of multi-modal optical and radar feature fusion model After using the above-mentioned deep learning model to extract the features of optical and radar images respectively, it is necessary to effectively fuse these features in order to give full play to the advantages of multi-source and multi-modal data in remote sensing intelligent identification of typical hidden dangers of power transmission lines. The present invention realizes adaptive learning of feature weights through a feature importance ranking model, thereby showing the contribution of each dimension of feature vector in modeling to classification and realizing feature importance ranking. Specifically, after the optical remote sensing image and the radar remote sensing image are respectively subjected to the designed feature extraction model, multi-source feature vectors are obtained. The above-mentioned multi-source features are multiplied with the original features after feature stacking, convolution calculation, and feature separation, and then adaptively selected with different branch features on a three-dimensional scale. Finally, the feature maps of each branch with attention weights are stacked and passed through the convolution layer to obtain the final fusion feature. The main purpose of the gating unit is to realize feature importance selection, select features with large information content and conducive to classification from the two types of features, and reduce the influence of features with small information content. After fusion, the features are input into the fully connected layer to obtain feature vectors, and then input into the softmax classifier to obtain the final classification probability result.

[0042] Improvement of loss function The loss function is used to evaluate the degree of error between the predicted target box and the actual target box. It measures the quality of a model and affects the performance of a model. Therefore, the choice of loss function is crucial.

[0043] The present invention designs a new hybrid loss function. First, an improved loss function Log-CoshDiceLoss for multi-classification problems is introduced. It is a region-based loss function that can accurately extract shapes and boundaries and can well balance the relationship between precision and recall. The region-based Log-CoshDiceLoss loss function improves the non-convexity of DiceLoss and can remain continuous and finite after the first-order differential. Therefore, this loss can make the loss and gradient value of a pixel point not only related to the label value and predicted value of the point, but also to the label values ​​and predicted values ​​of other points. The present invention combines the advantages of CE Loss and Log-CoshDiceLoss and designs a hybrid loss function. The calculation formula is as follows, where The hybrid loss function of the present invention can realize pixel and area loss calculation, and can better extract the boundary information of each target object.

[0044] (2) Effectiveness of Multi-source Remote Sensing Data Fusion To further demonstrate the performance improvement of the transmission channel typical hidden danger identification model by the multi-source multi-modal optical and radar remote sensing image fusion proposed in the present invention, the optical remote sensing image branch and the radar remote sensing image branch of the model are used as a separate model for model training and comparison with the fused model. The separated independent branches are only used for model training and classification with single-source data. To ensure data consistency, the same data set is used for training and testing.

[0045] Table 1 Comparison of accuracy between multi-source data and single-source data

[0046] As shown in Table 1, it can be seen that the overall accuracy of typical hidden danger identification in power transmission channels is 86.74% and 79.25% respectively when only optical remote sensing or radar remote sensing images are used. By combining the two branches and using multi-source data for model training, the overall accuracy can be improved to 91.42%, an increase of 4.68% and 12.17% respectively, proving the effectiveness of optical remote sensing and radar remote sensing image fusion in the identification of typical hidden dangers in power transmission channels.

[0047] Effectiveness of Loss Function Improvement In order to further demonstrate the effectiveness of the hybrid loss function proposed in the present invention in identifying typical hidden dangers in power transmission channels, a comparative experiment was conducted between it and the traditional loss function. At the same time, to ensure data consistency, the same data set was still used for training and testing.

[0048] Table 2 Accuracy comparison before and after loss function improvement

[0049] As shown in Table 2, it can be seen that if the cross entropy loss is used as the original loss function, the overall accuracy and Kappa coefficient of typical hidden danger identification in the transmission channel are 88.56% and 0.8485 respectively. If the hybrid loss function of the present invention is used, the overall accuracy and Kappa coefficient are 91.42% and 0.8724 respectively, which are increased by 3.86% and 3.39% respectively, proving the effectiveness of the loss function of the present invention in the identification of typical hidden dangers in the transmission channel.

[0050] In yet another embodiment of the present invention, a transmission channel hidden danger identification system integrating optics and radar remote sensing is provided, which can be used to implement the above-mentioned transmission channel hidden danger identification method integrating optics and radar remote sensing. Specifically, the system includes: The optical image feature extraction module is used to build a typical hidden danger feature extraction model for power transmission channels based on the improved UNet, extract features from optical remote sensing data, and obtain optical image feature vectors; The radar image feature extraction module is used to build a typical hidden danger feature extraction model for power transmission channels based on the improved TransUNet, extract features from radar remote sensing data, and obtain radar image feature vectors; A feature fusion module is used to fuse the optical image feature vector and the radar image feature vector to obtain a fusion feature; The sorting output module is used to select the fusion features based on the feature importance sorting, combined with the hybrid loss function, to obtain the classification probability result.

[0051] The division of modules in the embodiments of the present invention is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional module in each embodiment of the present invention may be integrated into one processor, or may exist physically separately, or two or more modules may be integrated into one module. The above-mentioned integrated modules may be implemented in the form of hardware or in the form of software functional modules.

[0052] In another embodiment of the present invention, a computer device is provided, the computer device including a processor and a memory, the memory is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, which is suitable for implementing one or more instructions, and is specifically suitable for loading and executing one or more instructions in the computer storage medium to implement the corresponding method flow or corresponding function; the processor described in the embodiment of the present invention can be used for the operation of the transmission channel hidden danger identification method integrating optical and radar remote sensing.

[0053] In another embodiment of the present invention, the present invention further provides a storage medium, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device for storing programs and data. It is understandable that the computer-readable storage medium here can include both built-in storage media in a computer device and, of course, extended storage media supported by the computer device. The computer-readable storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by a processor are also stored in the storage space, and these instructions can be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the transmission channel hidden danger identification method for integrating optical and radar remote sensing in the above embodiment.

[0054] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0055] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0056] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0057] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0058] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for identifying hidden dangers in power transmission channels by integrating optical and radar remote sensing, characterized in that: include: A typical hidden danger feature extraction model for power transmission channels based on improved UNet is constructed to extract features from optical remote sensing data and obtain optical image feature vectors. A typical hidden danger feature extraction model for power transmission channels based on improved TransUNet is constructed to extract features from radar remote sensing data and obtain radar image feature vectors. Perform feature fusion on the optical image feature vector and the radar image feature vector to obtain fusion features; The fusion features are selected based on the feature importance ranking and combined with the hybrid loss function to obtain the classification probability result.

2. The method for identifying hidden dangers in power transmission channels by integrating optical and radar remote sensing according to claim 1 is characterized in that: The construction of a typical hidden danger feature extraction model for a power transmission channel based on an improved UNet specifically includes: replacing the encoder of the original UNet with a multi-scale deformable convolution module, and replacing the deconvolution in the original UNet decoder with sub-pixel convolution.

3. The method for identifying hidden dangers in power transmission channels by integrating optical and radar remote sensing according to claim 2 is characterized in that: Feature extraction is performed on optical remote sensing data to obtain optical image feature vectors, including: The deformable convolution used is calculated as follows: （1） Where y represents the feature map after convolution calculation, x represents the input feature map, m0 represents the position on the feature map, and m i represents a specific sampling point in the sampling grid G, Δm i Indicates the offset that needs to be learned; by adding additional offsets, the deformable convolution can learn the shape characteristics of the hidden danger objects in the power transmission channel; In the upsampling process, we choose to use the sub-pixel convolution strategy to achieve image reconstruction from low resolution to high resolution through multi-channel feature maps. The input of the sub-pixel convolution is the low-resolution feature map H×W. First, we extract the image features to obtain n1 feature maps. After multiple operations, the last convolution is adjusted to r 2 channel, so that the total number of pixels is consistent with the high-resolution image to be generated, and finally r 2 The feature maps of the channels are recombined into the upsampling results of W×r and H×r to generate a high-resolution feature map rH×rW×C, where r is the upsampling scaling factor; sub-pixel convolution is used to convert the features from N×(C×r×r)×W×H to the form of N×C×(H×r)×(W×r).

4. The method for identifying hidden dangers in power transmission channels by integrating optical and radar remote sensing according to claim 1 is characterized in that: The construction of a typical hidden danger feature extraction model for power transmission channels based on the improved TransUNet extracts features from radar remote sensing data to obtain radar image feature vectors, specifically including: The TransUNet network model is used as the basic network structure. The overall structure retains the encoder and decoder structure of the semantic segmentation model to form a "U"-shaped network. The left half of the model is the encoder part, which adopts the 2D convolution form and increases the number of filters layer by layer. The number of filter channels is 64, 128, 256, and 512 respectively. The image features are extracted through a four-layer downsampling structure. A feature extraction residual module is designed in the encoder part for learning spectral, spatial and other features. The module includes a residual connection part and 2D convolutions of 1×1, 3×3, and 1×1 convolution kernels, and the convolution is performed separately during the convolution. The activation layer and batch normalization layer are operated; then the 12 Transformer layers of the original TransUNet model are reduced to 6 Transformer layers; after that, the spatial-channel attention mechanism module is introduced to improve the accuracy of feature learning; the right half of the model is the decoder part, which uses the form of transposed convolution to reduce the number of filters layer by layer, restore the image resolution to achieve pixel-by-pixel classification, and use high- and low-level feature fusion to fuse with the feature layer obtained by the encoding part, and replace the bilinear interpolation upsampling layer of the TransUNet decoder part with sub-pixel convolution for upsampling.

5. The method for identifying hidden dangers in power transmission channels by integrating optical and radar remote sensing according to claim 1, characterized in that: The feature fusion of the optical image feature vector and the radar image feature vector to obtain the fusion feature specifically includes: The optical image feature vector and the radar image feature vector are stacked, convolved, separated and then multiplied with the original features to achieve adaptive selection of different branch features on a three-dimensional scale. Finally, the branch feature maps assigned with attention weights are stacked and passed through a convolution layer to obtain the final fusion feature.

6. The method for identifying hidden dangers in power transmission channels by integrating optical and radar remote sensing according to claim 1 is characterized in that: The fusion features are selected based on the feature importance ranking to obtain the classification probability results, which specifically include: From the two types of features, features with large information content and conducive to classification are selected. After fusion, the features are input into the fully connected layer to obtain the feature vector, and then input into the softmax classifier to obtain the final classification probability result.

7. The method for identifying hidden dangers in power transmission channels by integrating optical and radar remote sensing according to claim 6 is characterized in that: The hybrid loss function is specifically: Firstly, an improved loss function Log-Cosh Dice Loss for multi-classification problems is introduced. The region-based Log-Cosh Dice Loss loss function improves the non-convexity of Dice Loss. The calculation formula is as follows, where Set to 0.5; （2） in, Loss represents the improved hybrid loss function of the present invention, CELoss represents the cross entropy loss function, L lc-dce represents the Log-Cosh Dice Loss function, α Represents the fusion coefficient.

8. A power transmission channel hidden danger identification system integrating optical and radar remote sensing, characterized in that: include: The optical image feature extraction module is used to build a typical hidden danger feature extraction model for power transmission channels based on the improved UNet, extract features from optical remote sensing data, and obtain optical image feature vectors; The radar image feature extraction module is used to build a typical hidden danger feature extraction model for power transmission channels based on the improved TransUNet, extract features from radar remote sensing data, and obtain radar image feature vectors; A feature fusion module is used to fuse the optical image feature vector and the radar image feature vector to obtain a fusion feature; The sorting output module is used to select the fusion features based on the feature importance sorting, combined with the hybrid loss function, to obtain the classification probability result.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method for identifying hidden dangers in a power transmission channel by integrating optical and radar remote sensing as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method for identifying hidden dangers in a power transmission channel by integrating optical and radar remote sensing as described in any one of claims 1 to 7 are implemented.

Images