Power transmission channel hidden danger prediction method and device, computer equipment, readable storage medium and program product

Abstract

The invention relates to a power transmission channel hidden danger prediction method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: acquiring optical remote sensing information and synthetic aperture radar information collected for an environment where a power transmission channel is located; performing convolution feature extraction on the synthetic aperture radar information and the optical remote sensing information for multiple times to obtain multiple pieces of radar feature information of different sizes and multiple pieces of optical remote sensing feature information of different sizes; for each size, fusing the radar feature information and the optical remote sensing feature information under the size to obtain fused feature information corresponding to the size; performing multi-level feature extraction on the fusion feature information corresponding to each size to obtain target feature information corresponding to each size; and performing hidden danger prediction based on the target feature information to obtain a hidden danger prediction result of the power transmission channel in the environment. The method can improve the accuracy of hidden danger prediction.

Description

Technical Field

[0001] The present application relates to the field of electric power technology, and in particular to a method, device, computer equipment, computer-readable storage medium and computer program product for predicting hidden dangers in a power transmission channel. Background Art

[0002] As society's demand for electricity continues to grow, the safe operation of transmission lines has become critical. However, transmission channels often pass through complex terrains, such as forests, mountains, and farmlands. Foreign objects in the external environment (such as fallen trees, construction, and overcrowded vegetation) may pose hidden dangers to power facilities, leading to power outages or even major accidents. Therefore, monitoring and intelligent identification of hidden dangers in transmission channels has become a key research direction for the power sector.

[0003] In traditional technology, the inspection of transmission lines mainly relies on manual inspection. However, due to the complex environment of some transmission lines, especially at high altitudes or remote areas, it is difficult to carry out inspections. Therefore, the accuracy of the inspection results cannot be ensured. Summary of the invention

[0004] Based on this, it is necessary to provide a method, device, computer equipment, computer-readable storage medium and computer program product for predicting hidden dangers in power transmission channels that can improve the accuracy of inspections in response to the above-mentioned technical problems.

[0005] In a first aspect, the present application provides a method for predicting hidden dangers in a power transmission channel, comprising:

[0006] Obtain optical remote sensing information and synthetic aperture radar information collected for the environment where the transmission channel is located;

[0007] Performing multiple convolution feature extractions on the synthetic aperture radar information and the optical remote sensing information respectively to obtain multiple radar feature information of different sizes and multiple optical remote sensing feature information of different sizes;

[0008] For each of the sizes, the radar feature information and the optical remote sensing feature information under the size are fused to obtain fused feature information corresponding to the size;

[0009] Performing multi-level feature extraction on the fused feature information corresponding to each of the sizes to obtain target feature information corresponding to each of the sizes;

[0010] Hidden danger prediction is performed based on each of the target characteristic information to obtain a hidden danger prediction result of the power transmission channel in the environment.

[0011] In one embodiment, multiple convolution feature extractions are performed on the synthetic aperture radar information to obtain multiple radar feature information of different sizes, including:

[0012] Performing a convolution operation on the synthetic aperture radar information to obtain convolved radar information;

[0013] Performing an activation operation on the convolution radar information to obtain initial radar feature information corresponding to the convolution radar information;

[0014] The initial radar characteristic information is subjected to multiple rounds of downsampling processing, and the processing results of each round are determined as multiple radar characteristic information of different sizes; wherein each round of downsampling processing includes a convolution operation and an activation operation performed in sequence; the convolution object of the first round is the initial radar characteristic information; and the convolution object of any round other than the first round is the radar characteristic information obtained in the previous round.

[0015] In one of the embodiments, the activating operation on the convolution radar information to obtain initial radar feature information corresponding to the convolution radar information includes:

[0016] Get the activation function consisting of hyperparameters and adaptive parameters;

[0017] An activation operation is performed on the convolution radar information based on the activation function to obtain initial radar feature information corresponding to the convolution radar information.

[0018] In one embodiment, multiple convolution feature extractions are performed on the optical remote sensing information to obtain multiple optical remote sensing feature information of different sizes, including:

[0019] Performing a convolution operation on the optical remote sensing information to obtain convolution remote sensing information;

[0020] Performing multi-scale analysis on the convolution remote sensing information to obtain initial remote sensing feature information corresponding to the convolution remote sensing information;

[0021] The initial remote sensing feature information is subjected to multiple rounds of downsampling processing, and the processing results of each round are determined as multiple optical remote sensing feature information of different sizes; wherein each round of downsampling processing includes sequential convolution operations and multi-scale analysis; the convolution object of the first round is the initial remote sensing feature information; and the convolution object of any round other than the first round is the optical remote sensing feature information obtained in the previous round.

[0022] In one embodiment, performing multi-scale analysis on the convolution remote sensing information to obtain initial remote sensing feature information corresponding to the convolution remote sensing information includes:

[0023] Performing a convolution operation on the convolution remote sensing information based on convolution kernels of different sizes to obtain a plurality of convolution remote sensing feature information;

[0024] Merging each of the convolution remote sensing feature information with the optical remote sensing information to obtain merged feature information;

[0025] An activation operation, a convolution operation, and a channel weighting operation are performed on the combined feature information to obtain initial remote sensing feature information.

[0026] In one embodiment, performing multi-level feature extraction on the fused feature information corresponding to each of the sizes to obtain target feature information corresponding to each of the sizes includes:

[0027] Determine the maximum fusion feature information corresponding to the maximum size from the fusion feature information corresponding to each of the sizes;

[0028] Performing feature enhancement on the maximum fused feature information to obtain enhanced fused feature information;

[0029] Multi-level feature extraction is performed on the enhanced fused feature information and the remaining fused feature information except the enhanced fused feature information to obtain target feature information corresponding to each of the sizes.

[0030] In one embodiment, the step of performing feature enhancement on the maximum fused feature information to obtain enhanced fused feature information includes:

[0031] Performing global average pooling on the maximum fusion feature information to obtain a maximum fusion feature matrix;

[0032] Performing a convolution operation, a batch normalization operation, an activation operation, and a secondary convolution operation on the maximum fusion feature matrix in sequence to obtain an attention weight corresponding to the maximum fusion feature matrix;

[0033] The maximum fused feature information is calculated based on the attention weight to obtain enhanced fused feature information.

[0034] In a second aspect, the present application also provides a power transmission channel hidden danger prediction device, comprising:

[0035] An information acquisition module, used to acquire optical remote sensing information and synthetic aperture radar information collected from the environment where the power transmission channel is located;

[0036] A convolution feature extraction module, used to perform multiple convolution feature extractions on the synthetic aperture radar information and the optical remote sensing information, respectively, to obtain multiple radar feature information of different sizes and multiple optical remote sensing feature information of different sizes;

[0037] An information fusion module, for fusing the radar feature information and the optical remote sensing feature information at each size to obtain fused feature information corresponding to the size;

[0038] A multi-level feature extraction module is used to perform multi-level feature extraction on the fused feature information corresponding to each of the sizes to obtain target feature information corresponding to each of the sizes;

[0039] The hidden danger prediction module is used to predict hidden dangers based on each of the target feature information to obtain hidden danger prediction results of the power transmission channel in the environment.

[0040] In a third aspect, the present application further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the above method are implemented.

[0041] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0042] In a fifth aspect, the present application further provides a computer program product, wherein the computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0043] The above-mentioned transmission channel hidden danger prediction method, device, computer equipment, computer-readable storage medium and computer program product obtain optical remote sensing information and synthetic aperture radar information collected for the environment where the transmission channel is located, and can determine the specific situation of the environment where the transmission channel is located. After that, multiple convolution feature extractions are performed on the synthetic aperture radar information and the optical remote sensing information respectively to obtain multiple radar feature information of different sizes and multiple optical remote sensing feature information of different sizes. For each size, the radar feature information and the optical remote sensing feature information under the size are fused to obtain the fused feature information corresponding to the size. Multi-level feature extraction is performed on the fused feature information corresponding to each size to obtain the target feature information corresponding to each size. Hidden danger prediction is performed based on the feature information of each target to obtain the hidden danger prediction result of the transmission channel in the environment. The effective fusion of optical remote sensing information and synthetic aperture radar information can be realized, and the advantages of both are combined to realize more comprehensive transmission channel hidden danger identification, thereby improving the accuracy of transmission channel hidden danger prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0045] Figure 1 A diagram showing an application environment of a method for predicting hidden dangers in a power transmission channel in an embodiment;

[0046] Figure 2 A schematic diagram of a flow chart of a method for predicting hidden dangers in a power transmission channel in one embodiment;

[0047] Figure 3 A schematic diagram of a flow chart of activation operation steps in an embodiment;

[0048] Figure 4 is a schematic diagram of a process of a multi-scale analysis step in an embodiment;

[0049] Figure 5 A schematic diagram of a flow chart of an enhanced fusion step in an embodiment;

[0050] Figure 6 A schematic diagram of a flow chart of a power transmission channel hidden danger prediction step in one embodiment;

[0051] Figure 7 A schematic diagram of a flow chart of a method for predicting hidden dangers in a power transmission channel in another embodiment;

[0052] Figure 8 It is a structural block diagram of a device for predicting hidden dangers in a power transmission channel in one embodiment;

[0053] Fig. 9 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0055] The transmission channel hidden danger prediction method provided in the embodiment of the present application can be applied to Figure 1In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. Among them, the terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablets, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car devices, projection devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The head-mounted device can be a virtual reality (VR) device, an augmented reality (AR) device, smart glasses, etc. The server 104 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services. Specifically, during the process of predicting hidden dangers in the power transmission channel, the server 104 obtains optical remote sensing information and synthetic aperture radar information collected for the environment in which the power transmission channel is located from the terminal 102; multiple convolution feature extractions are performed on the synthetic aperture radar information and the optical remote sensing information, respectively, to obtain multiple radar feature information of different sizes, and multiple optical remote sensing feature information of different sizes; for each size, the radar feature information and the optical remote sensing feature information under the size are fused to obtain fused feature information corresponding to the size; multi-level feature extraction is performed on the fused feature information corresponding to each size to obtain target feature information corresponding to each size; hidden danger prediction is performed based on each target feature information to obtain hidden danger prediction results of the power transmission channel in the environment.

[0056] In an exemplary embodiment, Figure 2 As shown in the figure, a method for predicting hidden dangers in power transmission channels is provided. Figure 1 The server 104 in the example is used as an example to illustrate the method, which includes the following steps S202 to S210. Among them:

[0057] Step S202: Acquire optical remote sensing information and synthetic aperture radar information collected for the environment where the power transmission channel is located.

[0058] Among them, the transmission channel refers to the collection of all nodes and edges in the network through which the current flows when the current is injected from the power node and flows out from the load node. These nodes and edges together constitute the path for the transmission of electric energy. With the continuous growth of society's demand for electricity, the safe operation of transmission lines has become crucial. However, transmission channels often pass through complex terrains, such as forests, mountains, farmlands, etc. Foreign objects in the external environment (such as fallen trees, construction, overcrowded vegetation, etc.) may pose hidden dangers to power facilities, leading to power outages or even major accidents. Therefore, the monitoring and intelligent identification of hidden dangers in transmission channels has become a key research direction for the power sector.

[0059] Optical remote sensing information is information obtained by detecting the ground or celestial bodies using optical remote sensing equipment on space carriers such as optical remote sensing satellites, space stations or space shuttles, such as space cameras, scanners and imaging spectrometers. These equipment mainly use visible light, ultraviolet light and infrared light for detection, and have high-resolution imaging and "what you see is what you get" capabilities. Synthetic aperture radar (SAR) information is information obtained using synthetic aperture radar technology. SAR is a high-resolution imaging radar that can obtain high-resolution radar images similar to optical photography under extremely low visibility meteorological conditions. It uses the relative motion between the radar and the target to synthesize a larger equivalent antenna aperture through data processing methods to obtain high-resolution radar images.

[0060] Specifically, optical remote sensing information provides the possibility of monitoring a large area. As two important information sources, optical remote sensing information and synthetic aperture radar information each have unique advantages: optical remote sensing information has high resolution and intuitive information, and can clearly present surface features and the status of transmission lines, but it is easily affected by clouds and weather conditions. Synthetic aperture radar information can penetrate clouds and vegetation through radar images obtained by different polarization methods, and has all-weather working capabilities, especially in detecting structural features and target material composition. Therefore, it is possible to obtain optical remote sensing information and synthetic aperture radar information collected for the environment where the transmission channel is located, and combine the above optical remote sensing information and synthetic aperture radar information to analyze the hidden dangers of the transmission channel.

[0061] Step S204, performing multiple convolution feature extractions on the synthetic aperture radar information and the optical remote sensing information respectively, to obtain multiple radar feature information of different sizes and multiple optical remote sensing feature information of different sizes.

[0062] The radar feature information refers to the information obtained by extracting the features of the synthetic aperture radar information, and the optical remote sensing feature information refers to the information obtained by extracting the features of the optical remote sensing information.

[0063] Specifically, convolutional feature extraction can effectively extract key features from the input information. These features include local features such as edges and textures, as well as more complex patterns obtained by combining these features at a deeper level. In addition, convolutional feature extraction reduces the complexity of the model through parameter sharing. In the convolution layer, convolution kernels (filters) of different sizes move on the input information to obtain feature information of different sizes. Among them, small convolution kernels can capture local details (such as road edges), and large convolution kernels are used to model large-scale structures (such as forest coverage and farmland block features). Therefore, multiple convolutional feature extractions are performed on synthetic aperture radar information and optical remote sensing information respectively to obtain multiple radar feature information of different sizes and multiple optical remote sensing feature information of different sizes. On the one hand, the accuracy of key feature extraction can be improved, and on the other hand, it can also facilitate the subsequent feature extraction and fusion process of feature information of different sizes.

[0064] In some specific embodiments, the server may perform a convolution operation on synthetic aperture radar information to obtain convolution radar information, perform an activation operation on the convolution radar information to obtain initial radar feature information corresponding to the convolution radar information, perform multiple rounds of downsampling processing on the initial radar feature information, and determine the processing results of each round as multiple radar feature information of different sizes; wherein each round of downsampling processing includes sequentially performed convolution operations and activation operations; the convolution object of the first round is the initial radar feature information; the convolution object of any round except the first round is the radar feature information obtained in the previous round.

[0065] In other specific embodiments, the server may also perform convolution operations on the optical remote sensing information to obtain convolution remote sensing information, perform multi-scale analysis on the convolution remote sensing information to obtain initial remote sensing feature information corresponding to the convolution remote sensing information, perform multiple rounds of downsampling processing on the initial remote sensing feature information, and determine the processing results of each round as multiple optical remote sensing feature information of different sizes; wherein each round of downsampling processing includes sequential convolution operations and multi-scale analysis; the convolution object of the first round is the initial remote sensing feature information; the convolution object of any round except the first round is the optical remote sensing feature information obtained in the previous round.

[0066] Step S206: for each size, the radar feature information and the optical remote sensing feature information under the size are fused to obtain fused feature information corresponding to the size.

[0067] The fused feature information is the information obtained by fusing the radar feature information and the optical remote sensing feature information.

[0068] Specifically, for each size, the radar feature information and the optical remote sensing feature information at that size have the same dimension. Therefore, the radar feature information and the optical remote sensing feature information can be weighted fused, that is, half of the value of each is added together to obtain the fused feature information corresponding to the size.

[0069] Step S208, performing multi-level feature extraction on the fused feature information corresponding to each size to obtain target feature information corresponding to each size.

[0070] Among them, target feature information refers to the feature information ultimately used for hidden danger prediction.

[0071] Specifically, after obtaining the fused feature information corresponding to each size, multi-level feature extraction can be performed on each fused feature information, and then the target feature information corresponding to each size can be obtained, which is convenient for subsequent hidden danger prediction of the transmission channel. In this process, through multi-layer sub-feature extraction, obstacles near transmission towers and wires (such as fallen trees, engineering equipment, etc.) can be accurately identified, even if the target size is small and complicated by the background. When it is necessary to monitor changes in ground objects (such as landslides and changes in vegetation growth), subtle change characteristics can also be captured, which is suitable for post-disaster transmission line inspections. By detecting minor anomalies such as fallen trees or equipment damage, efficient response can be achieved. Optical remote sensing information can be interfered by external factors such as weather and seasons. Through multi-scale and non-local feature modeling, the model maintains high robustness and strong generalization ability under different data distributions.

[0072] In some specific embodiments, the server may determine the maximum fused feature information corresponding to the maximum size from the fused feature information corresponding to each size, perform feature enhancement on the maximum fused feature information to obtain enhanced fused feature information, perform multi-level feature extraction on the enhanced fused feature information and the remaining fused feature information except the enhanced fused feature information to obtain target feature information corresponding to each size. In other specific embodiments, a multi-level feature extraction model may be preset, and based on the multi-level feature extraction model, multi-level feature extraction is performed on the fused feature information corresponding to each size to obtain target feature information corresponding to each size.

[0073] Step S210, performing hidden danger prediction based on each target feature information to obtain hidden danger prediction results of the power transmission channel in the environment.

[0074] The hidden danger prediction result refers to the prediction result of the hidden dangers that may exist in the power transmission channel in the environment. For example, the hidden danger prediction result may include the specific hidden danger type and location.

[0075] Specifically, after determining each target feature information, the server can perform hidden danger prediction based on each target feature information to obtain hidden danger prediction results of the power transmission channel in the environment. , , , the server can , , They are input into the target prediction head respectively, and the output is a tensor containing the prediction information. Each row corresponds to a prediction, including the bounding box coordinates, category label and confidence score.

[0076] The head is responsible for directly predicting the object's category, location (including the coordinates of the bounding box), and confidence of existence from the target feature information. The output of the detection head is a three-dimensional tensor, where each row contains the category probability, bounding box coordinates, and confidence of the object's existence for each predicted box at each spatial position (relative to the target feature information).

[0077] Bounding box prediction process: Each cell is responsible for predicting the coordinates and confidence of several bounding boxes. The coordinates of the bounding box include: the center point (relative to the location of the target feature information cell), width and height (scaled relative to the width and height of the entire image), these coordinates are passed through a sigmoid function to ensure that the output values ​​are between 0 and 1, which allows for more stable training.

[0078] (2) Category and confidence prediction process: In addition to coordinate predictions, each bounding box also has a confidence prediction, which is used to indicate the probability that an object of a specific category exists within the box.

[0079] The above-mentioned method for predicting hidden dangers in transmission channels obtains optical remote sensing information and synthetic aperture radar information collected for the environment where the transmission channel is located, and can determine the specific situation of the environment where the transmission channel is located. After that, multiple convolution feature extractions are performed on the synthetic aperture radar information and the optical remote sensing information respectively to obtain multiple radar feature information of different sizes and multiple optical remote sensing feature information of different sizes. For each size, the radar feature information and the optical remote sensing feature information under the size are fused to obtain the fused feature information corresponding to the size. Multi-level feature extraction is performed on the fused feature information corresponding to each size to obtain the target feature information corresponding to each size. Hidden danger prediction is performed based on the feature information of each target to obtain the hidden danger prediction result of the transmission channel in the environment. The effective fusion of optical remote sensing information and synthetic aperture radar information can be realized, and the advantages of both are combined to realize more comprehensive transmission channel hidden danger identification, thereby improving the accuracy of transmission channel hidden danger prediction.

[0080] In an exemplary embodiment, multiple convolution feature extractions are performed on synthetic aperture radar information to obtain multiple radar feature information of different sizes, including: performing a convolution operation on the synthetic aperture radar information to obtain convolution radar information; performing an activation operation on the convolution radar information to obtain initial radar feature information corresponding to the convolution radar information; performing multiple rounds of downsampling processing on the initial radar feature information, and determining the processing results of each round as multiple radar feature information of different sizes; wherein each round of downsampling processing includes sequentially performed convolution operations and activation operations; the convolution object of the first round is the initial radar feature information; and the convolution object of any round other than the first round is the radar feature information obtained in the previous round.

[0081] The convolution radar information is the information obtained by performing a convolution operation on the synthetic aperture radar information. The initial radar feature information is the information obtained by performing an activation operation on the convolution radar information and needs to perform other operations.

[0082] Specifically, the server can transfer the synthetic aperture radar image Input to the complex convolution (ComplexConv), The real and imaginary data of are taken as input, and the real and imaginary parts are processed separately through complex convolution to obtain synthetic aperture radar information. ;right Input to the Conv2d_BN_SiLU module with a convolution kernel size of 3×3 and a stride of 4, that is, the Conv2d layer, BN layer, and SiLU layer with a convolution kernel size of 3×3 and a stride of 4 are connected in series to obtain the intermediate convolution radar information ; Then, through the SARFB module To process, first input Perform a Conv2d (convolution) operation with a convolution kernel size of 3×3 and a step size of 1 to obtain convolution radar information ,right Perform activation operation to obtain the initial radar feature information corresponding to the convolution radar information For example, Figure 3 As shown, the above activation operation includes: Input Activate through the Cselu activation function to get the first convolution radar information ;right Input into the non-local module (NonLocal) to obtain the second convolution radar information ;right Input into the SE attention mechanism to obtain (Output) initial radar feature information .

[0083] Afterwards, the initial radar feature information Perform multiple rounds of downsampling processing, that is, perform multiple Conv2d module and SARFB module processing to obtain candidate radar feature information respectively , , , , , .in, , That is, multiple radar feature information of different sizes. Among them, the initial radar feature information is processed by the Conv2d module Perform convolution operation to obtain , through the SARFB module Perform activation operation to obtain radar feature information The subsequent operations follow the above process in sequence and will not be repeated here.

[0084] In this embodiment, features are extracted layer by layer by stacking convolution modules multiple times. Especially for the processing of synthetic aperture radar information, this structured stacking can significantly improve the accuracy of information feature extraction.

[0085] In an exemplary embodiment, an activation operation is performed on the convolution radar information to obtain initial radar feature information corresponding to the convolution radar information, including: obtaining an activation function composed of hyperparameters and adaptive parameters; and activating the convolution radar information based on the activation function to obtain initial radar feature information corresponding to the convolution radar information.

[0086] Among them, hyperparameters are parameters that are manually set before the machine learning algorithm runs to control the behavior and performance of the model. The selection of these parameters is not automatically learned through training data, but requires users to set them based on experience and domain knowledge. Adaptive parameter technology is a method for optimizing and adjusting system parameters. It ensures the best performance of the system by automatically adjusting system parameters under different operating conditions and environments. The activation function, in this application, refers to the Cselu activation function, and the calculation formula is as follows:

[0087]

[0088] in, , is a hyperparameter, usually a positive value, , is an adaptive parameter, where , adjusted according to the training effect , The size of For input, is the output result.

[0089] In this embodiment, the Cselu activation function can capture details and important information at different scales. The Cselu activation function can not only avoid gradient vanishing, but also dynamically adjust the adaptive parameters m and n to ensure that the model is more robust. The Cselu activation function can extract multi-level responses to help the model identify hidden danger features.

[0090] In an exemplary embodiment, multiple convolution feature extractions are performed on optical remote sensing information to obtain multiple optical remote sensing feature information of different sizes, including: performing a convolution operation on the optical remote sensing information to obtain convolution remote sensing information; performing multi-scale analysis on the convolution remote sensing information to obtain initial remote sensing feature information corresponding to the convolution remote sensing information; performing multiple rounds of downsampling processing on the initial remote sensing feature information, and determining the processing results of each round as multiple optical remote sensing feature information of different sizes; wherein each round of downsampling processing includes sequential convolution operations and multi-scale analysis; the convolution object of the first round is the initial remote sensing feature information; and the convolution object of any round other than the first round is the optical remote sensing feature information obtained in the previous round.

[0091] The convolution remote sensing information is the information obtained by performing a convolution operation on the optical remote sensing information. The initial remote sensing feature information is the information obtained by performing an activation operation on the convolution remote sensing information and needs to perform other operations.

[0092] Specifically, the server can transfer the optical remote sensing information Input into the Conv2d module for convolution operation to obtain intermediate convolution remote sensing information , and then Input to the Conv2d_BN_SiLU module with a convolution kernel size of 3×3 and a stride of 2, that is, the Conv2d layer, BN layer, and SiLU layer with a convolution kernel size of 3×3 and a stride of 2 are connected in series to obtain the convolution remote sensing information ; Then, the OPTICSFB module is used to convolve the remote sensing information Perform multi-scale analysis to obtain initial remote sensing feature information .

[0093] Afterwards, the initial remote sensing feature information Perform multiple rounds of downsampling processing, that is, perform multiple Conv2d module and OPTICSFB module processing to obtain candidate remote sensing feature information respectively , , , , , , .in, , , That is, multiple remote sensing feature information of different sizes. Among them, the initial radar feature information is processed by the Conv2d module Perform convolution operation to obtain , through the SARFB module Perform activation operation to obtain radar feature information The subsequent operations follow the above process in sequence and will not be repeated here.

[0094] In this embodiment, in optical remote sensing, the texture and geometric features of different objects have different scales. The OPTICSFB module extracts multi-scale features in parallel through 3×3, 5×5 and 7×7 convolution kernels. Small convolution kernels capture local details (such as road edges), and large convolution kernels are used to model large-scale structures (such as forest coverage and block features of farmland). Spectral information and spatial structure are often complementary. The OPTICSFB module introduces a fusion strategy to jointly model spectral dimensions and spatial features. Using 1×1 convolution to reduce the spectral dimension reduces computational overhead and avoids information redundancy. Detailed edges in optical remote sensing images (such as building outlines or electric tower structures) are crucial. OPTICSFB enhances the model's sensitivity to edge information by adding an edge detection convolution layer.

[0095] In optical remote sensing information, OPTICSFB can accurately identify obstacles (such as fallen trees, engineering equipment, etc.) near transmission towers and power lines, even if the target size is small and complicated by the background. When it is necessary to monitor changes in ground objects (such as landslides and changes in vegetation growth), the OPTICSFB module can capture subtle change characteristics. It is suitable for post-disaster transmission line inspections, and can achieve efficient response by detecting minor anomalies such as fallen trees or damaged equipment. Optical remote sensing information can be interfered by external factors such as weather and seasons. The OPTICSFB module uses multi-scale and non-local feature modeling to enable the model to maintain high robustness and strong generalization capabilities under different data distributions.

[0096] In an exemplary embodiment, a multi-scale analysis is performed on the convolution remote sensing information to obtain initial remote sensing feature information corresponding to the convolution remote sensing information, including: performing a convolution operation on the convolution remote sensing information based on convolution kernels of different sizes to obtain multiple convolution remote sensing feature information; merging each convolution remote sensing feature information with the optical remote sensing information to obtain merged feature information; performing an activation operation, a convolution operation, and a channel weighting operation on the merged feature information to obtain initial remote sensing feature information.

[0097] The convolution remote sensing feature information is feature information obtained by performing convolution operations on the convolution remote sensing information based on convolution kernels of different sizes. The merged feature information is information obtained by merging each convolution remote sensing feature information with the optical remote sensing information.

[0098] Specifically, Figure 4 As shown, in the OPTICSFB module, first The input is respectively fed into three convolution kernels of different sizes, namely, the depthwise separable convolution (DWConv) layer with convolution kernel size of 5×5 and 3×3, and the Conv2d layer with convolution kernel size of 1×1. The use of convolution kernels of different sizes can capture features of different scales. The 5×5 convolution kernel can extract a wider range of spatial information, while the 3×3 convolution kernel focuses more on capturing details in the medium range, thus obtaining multiple convolution remote sensing feature information. , , , then , , , Perform Add operation (merge operation) to obtain merge feature information , the original features are integrated with the features extracted from depth, thus enhancing the expressiveness of the optical feature map of hidden dangers in the transmission channel. Input to the Cselu activation layer for activation operation to obtain the first remote sensing feature information , and then The convolution operation is performed on the Conv2d layer with a convolution kernel size of 1×1 to obtain richer and more refined second remote sensing feature information. , and then Input into the SE attention mechanism for channel weighting operation to obtain (Output) initial remote sensing feature information .

[0099] In this embodiment, an OPTICSFB module is designed, which includes a multi-scale convolution submodule, a spectral-spatial feature fusion submodule, an attention mechanism submodule and an edge perception submodule to solve the diversity, complexity and interference problems in optical remote sensing data.

[0100] In an exemplary embodiment, multi-level feature extraction is performed on the fused feature information corresponding to each size to obtain target feature information corresponding to each size, including: determining the maximum fused feature information corresponding to the maximum size from the fused feature information corresponding to each size; performing feature enhancement on the maximum fused feature information to obtain enhanced fused feature information; and performing multi-level feature extraction on the enhanced fused feature information and the remaining fused feature information except the enhanced fused feature information to obtain target feature information corresponding to each size.

[0101] The enhanced fused feature information is the feature information obtained by performing feature enhancement on the maximum fused feature information.

[0102] Specifically, from the fusion feature information corresponding to each size, the maximum fusion feature information corresponding to the maximum size is determined. ,Will Input into the LYAttention module for feature enhancement to obtain enhanced fusion feature information ;Will Input into the SPPF module to obtain the first fusion feature information ;right Upsample to obtain the second fusion feature information ;Will and Perform Concat operation to obtain the third fusion feature information ;Will Input into the C3K2 module to obtain the fourth fusion feature information ;right Perform Upsample operation to obtain the fifth fusion feature information ;Will and Perform Concat operation to obtain the sixth fusion feature information ;Will Input into the C3K2 module to obtain the seventh fusion feature information ;Will Input into the Conv2d module (convolution kernel size is 3×3, step size is 2) to obtain the eighth fusion feature information ;Will and Perform Concat operation to obtain the ninth fusion feature information ;Will Input into the C3K2 module to obtain the tenth fusion feature information ;Will Input into the Conv2d module (convolution kernel size is 3×3, step size is 2) to obtain the eleventh fusion feature information ;Will and Perform Concat operation to obtain the twelfth fusion feature information ;Will Input into the C3K2 module to obtain the thirteenth fusion feature information .

[0103] In this embodiment, a LYAttention module is designed, such as Figure 5 As shown, first the maximum fusion feature information of the input Perform global average pooling, then process it through a Conv2d layer, a BN layer, a ReLU activation function, and another Conv2d layer, and then through a Cselu activation function to get the attention weights. Multiply element by element to output new enhanced fusion feature information .

[0104] For the maximum fusion feature information of the input ,in is the number of fused feature information channels, and Respectively represent the height and width of the feature map corresponding to the fused feature information. First, Perform global average pooling, each channel and height are retained in the pooling process, and the matrix is ​​obtained ,

[0105]

[0106] in, Representation Matrix The element value of the cth row and hth column;

[0107] It is then processed by a Conv2d layer, a BN layer, a ReLU activation function (Rectified Linear Unit), and another Conv2d layer to capture cross-channel interactions, i.e.;

[0108]

[0109] in is the Cselu function, and BN stands for batch normalization. Represents the attention weights calculated on different channels and height positions. The Conv2d layer calculates the attention weights by aggregating the heights of adjacent channels and z, and stacking two Conv2d layers to enhance the learning ability of this module. Application By multiplying element by element, we get new enhanced fusion feature information .

[0110] In one embodiment, Figure 6 As shown, a method for predicting hidden dangers of power transmission channels in a practical application scenario is also provided, including:

[0111] Step 1: Construct the original remote sensing image dataset D1 of hidden dangers in power transmission channels, or use the existing public original remote sensing image dataset of hidden dangers in power transmission channels, organize the acquired information, ensure that the optical remote sensing information and the synthetic aperture radar information can "correspond one to one" (that is, the acquisition time is the same and the scanned area is the same, which is called a set of data. Among them, both optical remote sensing information and synthetic aperture radar information include image data), manually annotate the acquired image data, use the annotation tool to select the hidden danger location (that is, determine the horizontal and vertical coordinates of the upper left corner and lower right corner of the box) and label it.

[0112] Embodiment: Polarimetric SAR data can be acquired through the Sentinel-1 satellite; optical remote sensing data can be acquired through the Sentinel-2 satellite.

[0113] Step 2: Perform preprocessing on D1 including but not limited to cropping, data enhancement and other operations to obtain the processed transmission channel hidden danger remote sensing image dataset D2, and divide the transmission channel hidden danger remote sensing image dataset to facilitate the subsequent training of the transmission channel hidden danger intelligent identification model that integrates polarimetric SAR and optical remote sensing data.

[0114] D2 contains multiple sets of transmission channel hidden danger data, which include a set of polarization SAR data and optical remote sensing data.

[0115] Note: In the present invention, the original remote sensing image dataset of transmission channel hidden dangers is first cropped to the same size. Since the original dataset of transmission channel hidden dangers is small, in order to ensure that the dataset has sufficient samples for training, verification and testing, and to enhance the robustness of the intelligent identification model of transmission channel hidden dangers and reduce the sensitivity of the model to images, the number of samples is increased through data enhancement, so as to better train the deep learning model, and the remote sensing image dataset of transmission channel hidden dangers after data enhancement is obtained, which is divided into training set, verification set and test set in a ratio of 8:1:1.

[0116] Step 3: Innovation: A transmission channel hidden danger intelligent identification model integrating polarimetric SAR and optical remote sensing data is constructed. The transmission channel hidden danger intelligent identification model is used to identify hidden dangers on the transmission channel. The transmission channel hidden danger data is input into the transmission channel hidden danger intelligent identification model, and the category and location of the hidden danger are output.

[0117] Intelligent identification model of hidden dangers in power transmission channels Figure 6 shown.

[0118] Step 3.1: Synthetic aperture radar image Input to the complex convolution (ComplexConv), The real and imaginary data of are taken as input, and the real and imaginary parts are processed separately through complex convolution to obtain synthetic aperture radar information. ;right Input to the Conv2d_BN_SiLU module with a convolution kernel size of 3×3 and a stride of 4, that is, the Conv2d layer, BN layer, and SiLU layer with a convolution kernel size of 3×3 and a stride of 4 are connected in series to obtain the intermediate convolution radar information ; Then, through the SARFB module To process, first input Perform a Conv2d (convolution) operation with a convolution kernel size of 3×3 and a step size of 1 to obtain convolution radar information. ,right Perform activation operation to obtain the initial radar feature information corresponding to the convolution radar information For example, Figure 3 As shown, the above activation operation includes: Activate through the Cselu activation function to get the first convolution radar information ;right Input into the non-local module (NonLocal) to obtain the second convolution radar information ;right Input into the SE attention mechanism to obtain the initial radar feature information .

[0119] Afterwards, the initial radar feature information Perform multiple rounds of downsampling processing, that is, perform multiple Conv2d module and SARFB module processing to obtain candidate radar feature information respectively , , , , , , .in, , That is, multiple radar feature information of different sizes. Among them, the initial radar feature information is processed by the Conv2d module Perform convolution operation to obtain , through the SARFB module Perform activation operation to obtain radar feature information The subsequent operations follow the above process in sequence and will not be repeated here.

[0120] Note: SAR data (Synthetic Aperture Radar images) are usually input into ComplexConv in complex form. This is because SAR data carries amplitude and phase information, which are suitable for complex expression.

[0121] Innovation: This paper designs a SARFB (SAR data processing) module. The high noise and complex scattering characteristics of SAR data increase the difficulty of feature extraction. This module is specifically optimized for feature extraction of polarimetric SAR images. This module can deeply mine the multi-scale information of SAR data and strengthen the representation of key areas through non-local feature capture and SE attention mechanism.

[0122] Note: The high noise (speckle noise) and complex scattering characteristics of SAR data (including the reflection of signals by the geometric structure, azimuth and electromagnetic properties of the ground objects) increase the difficulty of feature extraction.

[0123] Transmission channels often pass through various terrains, such as mountains, forests, and farmlands. Polarimetric SAR data may be interfered by terrain effects and multiple scattering in these complex scenes. The SARFB module can effectively filter noise information and highlight key features through non-local feature capture and SE attention mechanism. In the identification of hidden dangers in transmission channels, the model needs to accurately identify obstacles (such as trees, foreign objects, etc.) under the transmission lines and abnormalities of the tower structure. The SARFB module enhances the model's recognition ability for small targets and avoids misjudgment caused by changes in image scale through multi-scale convolution and Cselu activation function. The non-local feature capture and adaptive parameters of the SARFB module enable the model to maintain high generalization performance on diverse SAR data and reduce recognition failures caused by differences in data distribution. This design makes the model more accurate, robust, and stable in practical applications.

[0124] In the SARFB module, first enter Perform a Conv2d (convolution) operation with a convolution kernel size of 3×3 and a step size of 1 to obtain convolution radar information ,right Perform activation operation to obtain the initial radar feature information corresponding to the convolution radar information For example, Figure 3 As shown, the above activation operation includes: Activate through the Cselu activation function to get the first convolution radar information ;right Input into the non-local module (NonLocal) to obtain the second convolution radar information ;right Input into the SE attention mechanism to obtain the initial radar feature information .

[0125] Innovation: The calculation formula of the Cselu activation function is as follows:

[0126]

[0127] in, , is a hyperparameter, usually a positive value, , is an adaptive parameter, where , adjusted according to the training effect , The size of For input, is the output result.

[0128] In this embodiment, the Cselu activation function can capture details and important information at different scales. The Cselu activation function can not only avoid gradient vanishing, but also dynamically adjust the adaptive parameters m and n to ensure that the model is more robust. The Cselu activation function can extract multi-level responses to help the model identify hidden danger features.

[0129] Step 3.2: Optical remote sensing information Input into the Conv2d module for convolution operation to obtain intermediate convolution remote sensing information , and then Input to the Conv2d_BN_SiLU module with a convolution kernel size of 3×3 and a stride of 2, that is, the Conv2d layer, BN layer, and SiLU layer with a convolution kernel size of 3×3 and a stride of 2 are connected in series to obtain the convolution remote sensing information ; Then, the OPTICSFB module is used to convolve the remote sensing information Perform multi-scale analysis to obtain initial remote sensing feature information .

[0130] Afterwards, the initial remote sensing feature information Perform multiple rounds of downsampling processing, that is, perform multiple Conv2d module and OPTICSFB module processing to obtain candidate remote sensing feature information respectively , , , , , , .in, , , That is, multiple remote sensing feature information of different sizes. Among them, the initial radar feature information is processed by the Conv2d module Perform convolution operation to obtain , through the SARFB module Perform activation operation to obtain radar feature information The subsequent operations follow the above process in sequence and will not be repeated here.

[0131] In this embodiment, in optical remote sensing, the texture and geometric features of different objects have different scales. The OPTICSFB (multi-scale data processing) module extracts multi-scale features in parallel through 3×3, 5×5 and 7×7 convolution kernels. Small convolution kernels capture local details (such as road edges), and large convolution kernels are used to model large-scale structures (such as forest coverage and block features of farmland). Spectral information and spatial structure are often complementary. The OPTICSFB module introduces a fusion strategy to jointly model spectral dimensions and spatial features. Using 1×1 convolution to reduce the spectral dimension reduces computational overhead and avoids information redundancy. Detailed edges in optical remote sensing images (such as building outlines or electric tower structures) are crucial. OPTICSFB enhances the model's sensitivity to edge information by adding an edge detection convolution layer.

[0132] In optical remote sensing information, OPTICSFB can accurately identify obstacles (such as fallen trees, engineering equipment, etc.) near transmission towers and power lines, even if the target size is small and complicated by the background. When it is necessary to monitor changes in ground objects (such as landslides and changes in vegetation growth), the OPTICSFB module can capture subtle change characteristics. It is suitable for post-disaster transmission line inspections, and can achieve efficient response by detecting minor anomalies such as fallen trees or damaged equipment. Optical remote sensing information can be interfered by external factors such as weather and seasons. The OPTICSFB module uses multi-scale and non-local feature modeling to enable the model to maintain high robustness and strong generalization capabilities under different data distributions.

[0133] In the OPTICSFB module, first add The convolution kernels are input into three different sizes of convolution kernels, namely, the depthwise separable convolution (DWConv) layer with convolution kernel size of 5×5 and 3×3, and the Conv2d layer with convolution kernel size of 1×1. The convolution kernels of different sizes can capture features of different scales. The 5×5 convolution kernel can extract a wider range of spatial information, while the 3×3 convolution kernel focuses more on capturing details in the medium range, thus obtaining multiple convolution remote sensing feature information. , , , then , , , Perform Add operation (merge operation) to obtain merge feature information , the original features are integrated with the deeply extracted features, thus enhancing the expressiveness of the optical feature map of the hidden dangers in the transmission channel. Input to the Cselu activation layer for activation operation to obtain the first remote sensing feature information , and then The convolution operation is performed on the Conv2d layer with a convolution kernel size of 1×1 to obtain richer and more refined second remote sensing feature information. , and then Input into the SE attention mechanism for channel weighting operation to obtain the initial remote sensing feature information .

[0134] Step 3.3: and Perform weighted fusion. The dimensions of the two feature maps are the same. Take half of the value of each map and add them together to obtain the fused feature information. ;right and Perform weighted fusion to obtain fusion feature information ;right and Perform weighted fusion to obtain fusion feature information ;

[0135] From the fusion feature information corresponding to each size, determine the maximum fusion feature information corresponding to the maximum size ,Will Input into the LYAttention (feature enhancement module) module for feature enhancement to obtain enhanced fusion feature information ;Will Input into the SPPF module to obtain the first fusion feature information ;right Upsample to obtain the second fusion feature information ;Will and Perform the Concat operation to obtain the third fusion feature information ;Will Input into the C3K2 module to obtain the fourth fusion feature information ;right Perform Upsample operation to obtain the fifth fusion feature information ;Will and Perform Concat operation to obtain the sixth fusion feature information ;Will Input into the C3K2 module to obtain the seventh fusion feature information ;Will Input into the Conv2d module (convolution kernel size is 3×3, step size is 2) to obtain the eighth fusion feature information ;Will and Perform Concat operation to obtain the ninth fusion feature information ;Will Input into the C3K2 module to obtain the tenth fusion feature information ;Will Input into the Conv2d module (convolution kernel size is 3×3, step size is 2) to obtain the eleventh fusion feature information ;Will and Perform Concat operation to obtain the twelfth fusion feature information ;Will Input into C3K2 (feature extraction module) module to obtain the thirteenth fusion feature information .

[0136] Innovation: A LYAttention module is designed, such as Figure 5 As shown, first the maximum fusion feature information of the input Perform global average pooling, then process it through a Conv2d layer, a BN layer, a ReLU activation function, and another Conv2d layer, and then through a Cselu activation function to get the attention weights. Multiply element by element to output new enhanced fusion feature information .

[0137] For the maximum fusion feature information of the input ,in is the number of fused feature information channels, and Respectively represent the height and width of the feature map corresponding to the fused feature information. First, Perform global average pooling, each channel and height are retained in the pooling process, and the matrix is ​​obtained ,

[0138]

[0139] in, Representation Matrix The element value of the cth row and hth column;

[0140] It is then processed by a Conv2d layer, a BN layer, a ReLU activation function, and another Conv2d layer to capture cross-channel interactions, i.e.;

[0141]

[0142] in is the Cselu function, and BN stands for batch normalization. Represents the attention weights calculated on different channels and height positions. The Conv2d layer calculates the attention weights by aggregating the heights of adjacent channels and z, and stacking two Conv2d layers to enhance the learning ability of this module. Application By multiplying element by element, we get new enhanced fusion feature information .

[0143] Step 3.4: The feature information of each target is , , , the server can , , They are input into the target prediction head respectively, and the output is a tensor containing the prediction information. Each row corresponds to a prediction, including the bounding box coordinates, category label and confidence score.

[0144] The head is responsible for directly predicting the object's category, location (including the coordinates of the bounding box), and confidence of existence from the target feature information. The output of the detection head is a three-dimensional tensor, where each row contains the category probability, bounding box coordinates, and confidence of the object's existence for each predicted box at each spatial position (relative to the target feature information).

[0145] Bounding box prediction process: Each cell is responsible for predicting the coordinates and confidence of several bounding boxes. The coordinates of the bounding box include: the center point (relative to the location of the target feature information cell), width and height (scaled relative to the width and height of the entire image), these coordinates are passed through a sigmoid function to ensure that the output values ​​are between 0 and 1, which allows for more stable training.

[0146] (2) Category and confidence prediction process: In addition to coordinate predictions, each bounding box also has a confidence prediction, which is used to indicate the probability that an object of a specific category exists within the box.

[0147] Example:

[0148] Taking the case where optical remote sensing information and synthetic aperture radar information include images as an example, assume that the size of the feature maps corresponding to the optical remote sensing information and the synthetic aperture radar information are H×W×C, where H is the height of the feature map, W is the width of the feature map, and C is the number of channels of the feature map.

[0149] The image size of synthetic aperture radar information is 512×512×2 Input into the intelligent identification model of transmission channel hidden dangers, first After the ComplexConv layer, the synthetic aperture radar information with 64 channels is obtained. , the size is 512×512×64; Input into the Conv2d_BN_SiLU module to obtain the intermediate convolution radar information with 128 channels , the size is 128×128×128; Input into the SARFB module to obtain the initial radar feature information with 256 channels , the size is 128×128×256; then stack the Conv2d module and the SARFB module three times in sequence, and the candidate radar feature information of size 64×64×512 can be obtained respectively , candidate radar feature information of size 64×64×512 , candidate radar feature information of size 32×32×512 , candidate radar feature information of size 32×32×512 , candidate radar feature information of size 16×16×512 , candidate radar feature information of size 16×16×512 ;

[0150] The image size of the optical remote sensing information is 512×512×2 The input is then fed into the intelligent identification model of transmission channel hidden dangers. It first passes through the Conv2d layer with a convolution kernel size of 3×3 and a step size of 2 to obtain the intermediate convolution remote sensing information with a channel number of 64. , the size is 256×256×64; Input to the Conv2d_BN_SiLU module with a convolution kernel size of 3×3 and a step size of 2 to obtain convolution remote sensing information with a channel number of 128 , the size is 128×128×128; Input into the OPTICSFB module to obtain the initial remote sensing feature information with 256 channels , the size is 128×128×256; then stack the Conv2d module and the OPTICSFB module three times in sequence, and the candidate remote sensing feature information of size 64×64×512 can be obtained respectively , candidate remote sensing feature information of size 64×64×512 , candidate remote sensing feature information of size 32×32×512 , candidate remote sensing feature information of size 32×32×512 , candidate remote sensing feature information of size 16×16×512 , candidate remote sensing feature information of size 16×16×512 ;

[0151] right and Perform weighted fusion. The dimensions of the two feature maps are the same. Take half of the value of each map and add them together to obtain the fused feature information. , the size is 64×64×512; and Perform weighted fusion to obtain fusion feature information , the size is 32×32×512; and Perform weighted fusion to obtain fusion feature information , size is 16×16×512;

[0152] Will After input into the LYAttention module, the enhanced fusion feature information with a size of 16×16×512 is obtained ; then Input into the SPPF module to obtain the first fusion feature information of size 16×16×512 ;right Perform Upsample operation to obtain the second fusion feature information of size 32×32×512 ;Will and Perform the Concat operation to obtain the third fusion feature information with a size of 32×32×1024 ;Will Input into the C3K2 module to obtain the fourth fusion feature information of size 32×32×512 ;right Perform Upsample operation to obtain the fifth fusion feature information with a size of 64×64×512 ;Will and Perform the Concat operation to obtain the sixth fusion feature information with a size of 64×64×1024 ;Will Input into the C3K2 module to obtain the seventh fusion feature information of size 64×64×256 ;Will Input into the Conv2d module to obtain the eighth fusion feature information of size 32×32×256 ;Will and Perform the Concat operation to obtain the ninth fusion feature information with a size of 32×32×768 ;Will Input into the C3K2 module to obtain the tenth fusion feature information of size 32×32×512 ;Will Input into the Conv2d module to obtain the eleventh fusion feature information of size 16×16×512 ;Will and Perform the Concat operation to obtain the twelfth fusion feature information with a size of 16×16×1024 ;Will Input into the C3K2 module to obtain the thirteenth fusion feature information of size 16×16×512 ;Will , , They are input into the target prediction head respectively, and the output is a tensor containing the prediction information. Each row corresponds to a prediction, including the bounding box coordinates, category label and confidence score.

[0153] Step 4: Use the transmission channel hidden danger dataset to train the above model to obtain the trained transmission channel hidden danger intelligent identification model.

[0154] Train and update the parameters of each layer on the transmission channel hidden danger intelligent identification model. First initialize all neural network parameters and set the hyperparameters related to the transmission channel hidden danger intelligent identification model, including but not limited to the training rounds, batch size, optimizer selection, learning rate, etc.

[0155] After initializing the parameters, the training set and validation set data are divided into multiple batches. Each time, a batch of training set data is input into the transmission channel hidden danger intelligent identification model for training to obtain the training loss value loss of the batch. After a round of training for all batches of data in the entire training set is completed, the validation set is input into the transmission channel hidden danger intelligent identification model in batches to obtain the corresponding batch loss value batch_loss. The loss value of the validation set is mainly used to monitor whether the transmission channel hidden danger intelligent identification model is overfitting and to adjust the training strategy, such as terminating the training early or adjusting the learning rate. During training and verification, the transmission channel hidden danger intelligent identification model will automatically learn and adjust parameters according to each loss and batch_loss situation. When the training process has been carried out for one or more rounds until the batch_loss value converges, the training of the transmission channel hidden danger intelligent identification model is completed.

[0156] Step 5: After the model training is completed, the trained transmission channel hidden danger intelligent identification model is used to identify and analyze the remote sensing data of the current transmission channel. The final output includes the specific hidden danger type and location, which is used to provide accurate early warning information and subsequent processing suggestions for inspection personnel.

[0157] In a specific embodiment, Figure 7 As shown, a method for predicting hidden dangers in a power transmission channel is also provided, including:

[0158] Step S701, acquiring optical remote sensing information and synthetic aperture radar information collected for the environment where the power transmission channel is located, and performing a convolution operation on the synthetic aperture radar information to obtain convolution radar information;

[0159] Step S702, obtaining an activation function composed of a hyperparameter and an adaptive parameter, and performing an activation operation on the convolution radar information based on the activation function to obtain initial radar feature information corresponding to the convolution radar information;

[0160] Step S703, performing multiple rounds of downsampling processing on the initial radar characteristic information, and determining the processing result of each round as multiple radar characteristic information of different sizes;

[0161] Each round of downsampling processing includes convolution operations and activation operations performed in sequence; the convolution object of the first round is the initial radar feature information; the convolution object of any round other than the first round is the radar feature information obtained in the previous round;

[0162] Step S704, performing a convolution operation on the optical remote sensing information to obtain convolution remote sensing information, and performing a convolution operation on the convolution remote sensing information based on convolution kernels of different sizes to obtain a plurality of convolution remote sensing feature information;

[0163] Step S705, combining each convolution remote sensing feature information with the optical remote sensing information to obtain combined feature information;

[0164] Step S706, performing activation operation, convolution operation, and channel weighting operation on the combined feature information to obtain initial remote sensing feature information;

[0165] Step S707, performing multiple rounds of downsampling processing on the initial remote sensing feature information, and determining the processing result of each round as multiple optical remote sensing feature information of different sizes;

[0166] Each round of downsampling processing includes sequential convolution operations and multi-scale analysis; the convolution object of the first round is the initial remote sensing feature information; the convolution object of any round other than the first round is the optical remote sensing feature information obtained in the previous round.

[0167] Step S708, for each size, fusing the radar feature information and the optical remote sensing feature information under the size to obtain fused feature information corresponding to the size;

[0168] Step S709, determining the maximum fused feature information corresponding to the maximum size from the fused feature information corresponding to each size, and performing feature enhancement on the maximum fused feature information to obtain enhanced fused feature information;

[0169] Step S710, performing multi-level feature extraction on the enhanced fusion feature information and the remaining fusion feature information except the enhanced fusion feature information to obtain target feature information corresponding to each size;

[0170] Step S711, performing hidden danger prediction based on each target feature information to obtain hidden danger prediction results of the power transmission channel in the environment.

[0171] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0172] Based on the same inventive concept, the embodiment of the present application also provides a transmission channel hidden danger prediction device for implementing the above-mentioned transmission channel hidden danger prediction method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above-mentioned method, so the specific limitations in one or more transmission channel hidden danger prediction device embodiments provided below can refer to the above-mentioned limitations on the transmission channel hidden danger prediction method, and will not be repeated here.

[0173] In an exemplary embodiment, Figure 8 As shown, a transmission channel hidden danger prediction device 800 is provided, comprising: an information acquisition module 802, a convolution feature extraction module 804, an information fusion module 806, a multi-level feature extraction module 808 and a hidden danger prediction module 810, wherein:

[0174] The information acquisition module 802 is used to acquire optical remote sensing information and synthetic aperture radar information collected from the environment where the power transmission channel is located;

[0175] The convolution feature extraction module 804 is used to perform multiple convolution feature extractions on the synthetic aperture radar information and the optical remote sensing information respectively, so as to obtain multiple radar feature information of different sizes and multiple optical remote sensing feature information of different sizes;

[0176] The information fusion module 806 is used to fuse the radar feature information and the optical remote sensing feature information under each size to obtain fused feature information corresponding to the size;

[0177] A multi-level feature extraction module 808 is used to perform multi-level feature extraction on the fused feature information corresponding to each size to obtain target feature information corresponding to each size;

[0178] The hidden danger prediction module 810 is used to predict hidden dangers based on the characteristic information of each target, and obtain hidden danger prediction results of the power transmission channel in the environment.

[0179] In an exemplary embodiment, the convolution feature extraction module 804 includes:

[0180] A first convolution operation unit is used to perform a convolution operation on the synthetic aperture radar information to obtain convolution radar information;

[0181] An activation operation unit, used to perform an activation operation on the convolution radar information to obtain initial radar feature information corresponding to the convolution radar information;

[0182] The first downsampling processing unit is used to perform multiple rounds of downsampling processing on the initial radar characteristic information, and determine the processing results of each round as multiple radar characteristic information of different sizes; wherein each round of downsampling processing includes a convolution operation and an activation operation performed in sequence; the convolution object of the first round is the initial radar characteristic information; the convolution object of any round other than the first round is the radar characteristic information obtained in the previous round.

[0183] In an exemplary embodiment, the activation operation unit is specifically used to:

[0184] Get the activation function consisting of hyperparameters and adaptive parameters;

[0185] The convolution radar information is activated based on the activation function to obtain the initial radar feature information corresponding to the convolution radar information.

[0186] In an exemplary embodiment, the convolution feature extraction module 804 includes:

[0187] A second convolution operation unit is used to perform a convolution operation on the optical remote sensing information to obtain convolution remote sensing information;

[0188] A multi-scale analysis unit, used for performing multi-scale analysis on the convolution remote sensing information to obtain initial remote sensing feature information corresponding to the convolution remote sensing information;

[0189] The second downsampling processing unit is used to perform multiple rounds of downsampling processing on the initial remote sensing feature information, and determine the processing results of each round as multiple optical remote sensing feature information of different sizes; wherein each round of downsampling processing includes convolution operations and multi-scale analysis performed in sequence; the convolution object of the first round is the initial remote sensing feature information; the convolution object of any round except the first round is the optical remote sensing feature information obtained in the previous round.

[0190] In an exemplary embodiment, the multi-scale analysis unit is specifically used for:

[0191] The convolution operation is performed on the convolution remote sensing information based on convolution kernels of different sizes to obtain multiple convolution remote sensing feature information;

[0192] Merging each convolution remote sensing feature information with the optical remote sensing information to obtain merged feature information;

[0193] The combined feature information is activated, convolved, and weighted to obtain initial remote sensing feature information.

[0194] In an exemplary embodiment, the multi-level feature extraction module 808 is specifically used for:

[0195] Determine the maximum fusion feature information corresponding to the maximum size from the fusion feature information corresponding to each size;

[0196] Performing feature enhancement on the maximum fused feature information to obtain enhanced fused feature information;

[0197] Multi-level feature extraction is performed on the enhanced fusion feature information and the remaining fusion feature information except the enhanced fusion feature information to obtain target feature information corresponding to each size.

[0198] Each module in the above-mentioned power transmission channel hidden danger prediction device can be implemented in whole or in part by software, hardware and their combination. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the corresponding operations of each of the above modules.

[0199] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in FIG. Fig. 9As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, near field communication (NFC) or other technologies. When the computer program is executed by the processor, a method for predicting hidden dangers in a power transmission channel is implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0200] Those skilled in the art will understand that Fig. 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0201] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps of the above method when executing the computer program.

[0202] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0203] In one embodiment, a computer program product is provided, comprising a computer program, which implements the steps of the above method when executed by a processor.

[0204] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0205] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0206] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0207] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for predicting hidden dangers in power transmission channels, characterized in that: The method comprises: Obtain optical remote sensing information and synthetic aperture radar information collected for the environment where the transmission channel is located; Performing multiple convolution feature extractions on the synthetic aperture radar information and the optical remote sensing information respectively to obtain multiple radar feature information of different sizes and multiple optical remote sensing feature information of different sizes; For each of the sizes, the radar feature information and the optical remote sensing feature information under the size are fused to obtain fused feature information corresponding to the size; Performing multi-level feature extraction on the fused feature information corresponding to each of the sizes to obtain target feature information corresponding to each of the sizes; Hidden danger prediction is performed based on each of the target characteristic information to obtain a hidden danger prediction result of the power transmission channel in the environment.

2. The method according to claim 1, characterized in that: Performing multiple convolution feature extractions on the synthetic aperture radar information to obtain multiple radar feature information of different sizes, including: Performing a convolution operation on the synthetic aperture radar information to obtain convolved radar information; Performing an activation operation on the convolution radar information to obtain initial radar feature information corresponding to the convolution radar information; The initial radar characteristic information is subjected to multiple rounds of downsampling processing, and the processing result of each round is determined as multiple radar characteristic information of different sizes; wherein each round of downsampling processing includes a convolution operation and an activation operation performed in sequence; the convolution object of the first round is the initial radar characteristic information; and the convolution object of any round other than the first round is the radar characteristic information obtained in the previous round.

3. The method according to claim 2, characterized in that The activating operation on the convolution radar information to obtain initial radar feature information corresponding to the convolution radar information includes: Get the activation function consisting of hyperparameters and adaptive parameters; An activation operation is performed on the convolution radar information based on the activation function to obtain initial radar feature information corresponding to the convolution radar information.

4. The method according to claim 1, characterized in that: Perform multiple convolution feature extractions on the optical remote sensing information to obtain multiple optical remote sensing feature information of different sizes, including: Performing a convolution operation on the optical remote sensing information to obtain convolution remote sensing information; Performing multi-scale analysis on the convolution remote sensing information to obtain initial remote sensing feature information corresponding to the convolution remote sensing information; The initial remote sensing feature information is subjected to multiple rounds of downsampling processing, and the processing results of each round are determined as multiple optical remote sensing feature information of different sizes; wherein each round of downsampling processing includes sequential convolution operations and multi-scale analysis; the convolution object of the first round is the initial remote sensing feature information; and the convolution object of any round other than the first round is the optical remote sensing feature information obtained in the previous round.

5. The method according to claim 4, characterized in that The performing multi-scale analysis on the convolution remote sensing information to obtain initial remote sensing feature information corresponding to the convolution remote sensing information includes: Performing a convolution operation on the convolution remote sensing information based on convolution kernels of different sizes to obtain a plurality of convolution remote sensing feature information; Merging each of the convolution remote sensing feature information with the optical remote sensing information to obtain merged feature information; An activation operation, a convolution operation, and a channel weighting operation are performed on the combined feature information to obtain initial remote sensing feature information.

6. The method according to claim 1, characterized in that The multi-level feature extraction is performed on the fused feature information corresponding to each of the sizes to obtain the target feature information corresponding to each of the sizes, including: Determine the maximum fusion feature information corresponding to the maximum size from the fusion feature information corresponding to each of the sizes; Performing feature enhancement on the maximum fused feature information to obtain enhanced fused feature information; Multi-level feature extraction is performed on the enhanced fused feature information and the remaining fused feature information except the enhanced fused feature information to obtain target feature information corresponding to each of the sizes.

7. A device for predicting hidden dangers in power transmission channels, characterized in that: The device comprises: An information acquisition module, used to acquire optical remote sensing information and synthetic aperture radar information collected from the environment where the power transmission channel is located; A convolution feature extraction module, used to perform multiple convolution feature extractions on the synthetic aperture radar information and the optical remote sensing information, respectively, to obtain multiple radar feature information of different sizes and multiple optical remote sensing feature information of different sizes; An information fusion module, for fusing the radar feature information and the optical remote sensing feature information at each size to obtain fused feature information corresponding to the size; A multi-level feature extraction module is used to perform multi-level feature extraction on the fused feature information corresponding to each of the sizes to obtain target feature information corresponding to each of the sizes; The hidden danger prediction module is used to predict hidden dangers based on each of the target feature information to obtain hidden danger prediction results of the power transmission channel in the environment.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images