Methods for detecting hidden dangers in power transmission channels, computer equipment, readable storage media, and software products

Abstract

This application relates to a method, computer equipment, readable storage medium, and program product for detecting hidden dangers in power transmission channels. It is applied in the field of big data technology. The method includes: acquiring optical images and synthetic aperture radar (SAR) images; performing multi-scale feature extraction on the optical images to obtain a first feature image; passing the first feature images through multiple sequentially connected depth multi-scale feature extraction modules to obtain second feature images; fusing the second feature images to obtain a third feature image; performing channel hidden danger detection on the third feature image to obtain multiple target detection boxes; slicing the SAR image according to the multiple target detection boxes to obtain radar slice images corresponding to each target detection box; classifying each radar slice image to obtain classification results for each slice; and determining the power transmission channel hidden danger detection result based on the classification results for each slice. This method can improve the accuracy of power transmission channel hidden danger detection.

Description

Technical Field

[0001] This application relates to the field of big data technology, and in particular to a method for detecting hidden dangers in power transmission channels, computer equipment, computer-readable storage media, and computer program products. Background Technology

[0002] Power transmission channels are the objective carriers of power transmission networks, and their safe and stable operation is crucial for ensuring power supply and maintaining the normal operation of social production and life. Therefore, there is an urgent need for a method to detect potential hazards in power transmission channels.

[0003] Currently, power transmission lines are inspected manually to detect potential hazards. However, this method is labor-intensive and inefficient. Furthermore, power transmission lines are widely distributed in complex environments, making it difficult for inspection personnel to reach many areas. This results in limitations and blind spots in manual inspections. Moreover, for minor and hidden hazards, the accuracy of manual inspections depends heavily on the experience and skill level of the inspectors. Ultimately, this leads to relatively low accuracy in detecting potential hazards in power transmission lines. Summary of the Invention

[0004] Therefore, it is necessary to provide a method, apparatus, computer equipment, computer-readable storage medium, and computer program product for detecting potential hazards in power transmission channels, which can improve the accuracy of such detection, in response to the aforementioned technical problems.

[0005] Firstly, this application provides a method for detecting potential hazards in power transmission channels, including:

[0006] Acquire optical and synthetic aperture radar images of the power transmission channel to be inspected;

[0007] The optical image is subjected to multi-scale feature extraction using a multi-scale feature extraction unit to obtain a first feature image;

[0008] The first feature image is passed through at least two depth multi-scale feature extraction modules connected in sequence to obtain the second feature image output by each depth multi-scale feature extraction module, wherein the depth multi-scale feature extraction module includes a downsampling unit and a multi-scale feature extraction unit;

[0009] The second feature images are fused to obtain a third feature image, and channel hazard detection is performed on the third feature image to obtain the position information of multiple target detection boxes;

[0010] Based on the position information of the multiple target detection boxes, the synthetic aperture radar image is sliced ​​to obtain a radar slice image corresponding to each target detection box.

[0011] Each radar slice image is classified to obtain the slice classification result corresponding to each radar slice image;

[0012] Based on the classification results of each slice, the detection results of potential hazards in the power transmission channel corresponding to the power transmission channel to be detected are determined.

[0013] Secondly, this application also provides a power transmission channel hazard detection device, comprising:

[0014] The acquisition module is used to acquire optical images and synthetic aperture radar images of the power transmission channel to be detected;

[0015] An extraction module is used to perform multi-scale feature extraction on the optical image using a multi-scale feature extraction unit to obtain a first feature image; the first feature image is passed through at least two depth multi-scale feature extraction modules connected in sequence to obtain a second feature image output by each depth multi-scale feature extraction module, wherein the depth multi-scale feature extraction module includes a downsampling unit and a multi-scale feature extraction unit; and the second feature images are fused to obtain a third feature image.

[0016] The detection module is used to perform channel hazard detection on the third feature image to obtain the position information of multiple target detection boxes;

[0017] The slicing module is used to slice the synthetic aperture radar image according to the position information of the multiple target detection boxes to obtain a radar slice image corresponding to each target detection box.

[0018] The classification module is used to classify each radar slice image separately to obtain the slice classification result corresponding to each radar slice image;

[0019] The detection module is also used to determine the detection result of potential hazards in the power transmission channel corresponding to the power transmission channel to be detected based on the classification results of each slice.

[0020] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0021] Acquire optical and synthetic aperture radar images of the power transmission channel to be inspected;

[0022] The optical image is subjected to multi-scale feature extraction using a multi-scale feature extraction unit to obtain a first feature image;

[0023] The first feature image is passed through at least two depth multi-scale feature extraction modules connected in sequence to obtain the second feature image output by each depth multi-scale feature extraction module, wherein the depth multi-scale feature extraction module includes a downsampling unit and a multi-scale feature extraction unit;

[0024] The second feature images are fused to obtain a third feature image, and channel hazard detection is performed on the third feature image to obtain the position information of multiple target detection boxes;

[0025] Based on the position information of the multiple target detection boxes, the synthetic aperture radar image is sliced ​​to obtain a radar slice image corresponding to each target detection box.

[0026] Each radar slice image is classified to obtain the slice classification result corresponding to each radar slice image;

[0027] Based on the classification results of each slice, the detection results of potential hazards in the power transmission channel corresponding to the power transmission channel to be detected are determined.

[0028] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0029] Acquire optical and synthetic aperture radar images of the power transmission channel to be inspected;

[0030] The optical image is subjected to multi-scale feature extraction using a multi-scale feature extraction unit to obtain a first feature image;

[0031] The first feature image is passed through at least two depth multi-scale feature extraction modules connected in sequence to obtain the second feature image output by each depth multi-scale feature extraction module, wherein the depth multi-scale feature extraction module includes a downsampling unit and a multi-scale feature extraction unit;

[0032] The second feature images are fused to obtain a third feature image, and channel hazard detection is performed on the third feature image to obtain the position information of multiple target detection boxes;

[0033] Based on the position information of the multiple target detection boxes, the synthetic aperture radar image is sliced ​​to obtain a radar slice image corresponding to each target detection box.

[0034] Each radar slice image is classified to obtain the slice classification result corresponding to each radar slice image;

[0035] Based on the classification results of each slice, the detection results of potential hazards in the power transmission channel corresponding to the power transmission channel to be detected are determined.

[0036] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0037] Acquire optical and synthetic aperture radar images of the power transmission channel to be inspected;

[0038] The optical image is subjected to multi-scale feature extraction using a multi-scale feature extraction unit to obtain a first feature image;

[0039] The first feature image is passed through at least two depth multi-scale feature extraction modules connected in sequence to obtain the second feature image output by each depth multi-scale feature extraction module, wherein the depth multi-scale feature extraction module includes a downsampling unit and a multi-scale feature extraction unit;

[0040] The second feature images are fused to obtain a third feature image, and channel hazard detection is performed on the third feature image to obtain the position information of multiple target detection boxes;

[0041] Based on the position information of the multiple target detection boxes, the synthetic aperture radar image is sliced ​​to obtain a radar slice image corresponding to each target detection box.

[0042] Each radar slice image is classified to obtain the slice classification result corresponding to each radar slice image;

[0043] Based on the classification results of each slice, the detection results of potential hazards in the power transmission channel corresponding to the power transmission channel to be detected are determined.

[0044] The aforementioned method, apparatus, computer equipment, computer-readable storage medium, and computer program product for detecting potential hazards in power transmission channels acquire optical images and synthetic aperture radar (SAR) images of the power transmission channel to be detected; utilize a multi-scale feature extraction unit to extract multi-scale features from the optical images to obtain a first feature image; pass the first feature image through at least two sequentially connected depth multi-scale feature extraction modules to obtain second feature images output by each depth multi-scale feature extraction module, wherein each depth multi-scale feature extraction module includes a downsampling unit and a multi-scale feature extraction unit; fuse the second feature images to obtain a third feature image, and perform channel hazard detection on the third feature image to obtain the position information of multiple target detection boxes; slice the SAR image according to the position information of the multiple target detection boxes to obtain radar slice images corresponding to each target detection box; classify each radar slice image to obtain slice classification results corresponding to each radar slice image; and determine the detection result of the potential hazard of the power transmission channel to be detected based on the slice classification results.

[0045] Thus, by using both synthetic aperture radar (SAR) images and optical images as image processing sources, the extracted features are not limited by the environment of the power transmission channel being inspected. Subsequently, a multi-scale feature extraction unit performs multi-scale feature extraction on the optical image to obtain a first feature image. This ensures that the input features can represent multi-scale feature information in the input data dimension. Furthermore, the first feature image is passed through at least two sequentially connected deep multi-scale feature extraction modules to obtain second feature images output by each module. This ensures that the feature extraction process is multi-scale in the feature extraction dimension. The second feature images are then fused to obtain a third feature image, and channel hazard detection is performed on this third feature image to obtain the position information of multiple target detection boxes. In other words, multiple second feature images obtained from multi-scale feature extraction are fused to obtain further multi-scale feature extraction. The third feature image is used as the basis for detecting potential hazards in the transmission channel, thus improving the accuracy of target detection. Furthermore, based on the position information of the target detection box, the synthetic aperture radar image is sliced ​​to obtain multiple radar slice images. These radar slice images can, to a certain extent, represent the preliminary results obtained from the optical image processing. Each radar slice image is then classified to obtain the corresponding slice classification results. Finally, based on the slice classification results, the detection result of potential hazards in the transmission channel to be detected is determined. As the above analysis shows, the radar slice image is obtained after layers of feature extraction and processing. Therefore, the slice classification results obtained are more accurate, and the determined detection result of potential hazards in the transmission channel is more consistent with the actual situation of the transmission channel to be detected, thus improving the accuracy of potential hazard detection in the transmission channel. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 This is an application environment diagram of a power transmission channel hidden danger detection method in one embodiment;

[0048] Figure 2 This is a flowchart illustrating a method for detecting hidden dangers in power transmission channels in one embodiment;

[0049] Figure 3 This is a schematic diagram of a process for detecting channel defects in an optical image in one embodiment;

[0050] Figure 4 This is a flowchart illustrating the steps of processing an optical image in one embodiment;

[0051] Figure 5 This is a flowchart illustrating the process of processing a third input feature image to obtain a target downsampled feature image in one embodiment.

[0052] Figure 6 This is a flowchart illustrating the process of processing a first input feature image to obtain a target downsampled feature image in one embodiment.

[0053] Figure 7 This is a flowchart illustrating the steps of classifying radar slice images in one embodiment;

[0054] Figure 8 This is a schematic diagram illustrating the process of processing radar slice images to obtain target enhancement feature images in one embodiment;

[0055] Figure 9 This is a structural block diagram of a power transmission channel hazard detection device in one embodiment;

[0056] Figure 10 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0058] It should be noted that all information and data involved in this application (including but not limited to data used for analysis, stored data, and displayed data) are information and data authorized by the user or fully authorized by all parties, and the acquisition, transmission, storage, use, and processing of related data comply with the relevant provisions of national laws and regulations. Users can refuse or easily refuse the content pushed to them (e.g., radar slice images, optical images, synthetic aperture radar images, slice classification results, and power transmission channel hidden danger detection results). In the embodiments of this application, certain software, components, models, and other existing industry solutions may be mentioned. These should be considered exemplary, and their purpose is merely to illustrate the feasibility of implementing the technical solution of this application, but does not mean that the applicant has already used or necessarily used such solutions.

[0059] Firstly, it's understandable that researchers utilize deep learning algorithms to continuously improve target detection models and increase the efficiency of power transmission channel hazard detection in order to enhance the accuracy of hazard detection. Specifically, deep learning algorithms are used to perform hazard detection on optical images of the power transmission channel to be detected, obtaining hazard detection information. However, the environment in which power transmission channels are located is not uniform or fixed; that is, the environment of power transmission channels may differ at different locations and / or at different times. For environments with significant atmospheric influences such as clouds, rain, and snow, optical images may not accurately represent the characteristics of the power transmission channel, resulting in relatively low accuracy in hazard detection. Therefore, there is an urgent need for a method to improve the accuracy of power transmission channel hazard detection.

[0060] The power transmission channel hazard detection method provided in this application embodiment can be applied to, for example... Figure 1In the application environment shown, the image acquisition component 102 communicates with the server 104 via a network. The image acquisition component 102 includes an optical acquisition component and a synthetic aperture radar (SAR) acquisition component. The optical acquisition component acquires optical images of the power transmission channel to be inspected, and the SAR acquisition component acquires SAR images of the power transmission channel to be inspected. The terminal 106 communicates with the server 104 via a network. A data storage system can store the data that the server 104 needs to process. The data storage system can be integrated onto the server 104 or placed on a cloud or other network server. The server 104 acquires optical and synthetic aperture radar (SAR) images of the power transmission channel to be inspected. Using a multi-scale feature extraction unit, multi-scale features are extracted from the optical image to obtain a first feature image. The first feature image is then passed through at least two sequentially connected depth multi-scale feature extraction modules to obtain second feature images output by each module. Each depth multi-scale feature extraction module includes a downsampling unit and a multi-scale feature extraction unit. The second feature images are fused to obtain a third feature image, and channel hazard detection is performed on the third feature image to obtain the position information of multiple target detection boxes. Based on the position information of the multiple target detection boxes, the SAR image is sliced ​​to obtain radar slice images corresponding to each target detection box. Each radar slice image is classified to obtain slice classification results. Based on the slice classification results, the power transmission channel hazard detection result corresponding to the power transmission channel to be inspected is determined. The server 104 can push at least one of the following to the terminal 106: the first feature image, the second feature image, the third feature image, the radar slice image, the optical image, the SAR image, the slice classification result, and the power transmission channel hazard detection result. The terminal 106 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle systems, and projection devices. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted displays. Head-mounted displays can be virtual reality (VR) devices, augmented reality (AR) devices, and smart glasses. The server 104 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0061] In one exemplary embodiment, such as Figure 2 As shown, a method for detecting hidden dangers in power transmission channels is provided, and this method is applied to... Figure 1 Taking server 104 as an example, and describing it with the main body omitted, the steps 202 to 214 are included. Wherein:

[0062] Step 202: Obtain optical images and synthetic aperture radar images of the power transmission channel to be detected.

[0063] As one embodiment, acquiring an optical image of a power transmission channel to be tested includes: acquiring an optical image of the power transmission channel to be tested acquired by an optical acquisition unit deployed in the area where the power transmission channel to be tested is located. As one embodiment, acquiring a synthetic aperture radar (SAR) image of a power transmission channel to be tested includes: acquiring a SAR image of the power transmission channel to be tested acquired by a SAR acquisition unit deployed in the area where the power transmission channel to be tested is located.

[0064] Thus, for areas with complex environments that are difficult for ordinary personnel to reach, images can also be acquired by deploying synthetic aperture radar acquisition components or optical acquisition components.

[0065] As another embodiment, acquiring an optical image of the power transmission channel to be detected includes: acquiring an optical image of the power transmission channel to be detected by an optical acquisition unit deployed on a moving part. As another embodiment, acquiring a synthetic aperture radar image of the power transmission channel to be detected includes: acquiring a synthetic aperture radar image of the power transmission channel to be detected by a synthetic aperture radar acquisition unit deployed on a moving part.

[0066] In this way, the procurement costs of synthetic aperture radar acquisition components and optical acquisition components can be saved. Multiple power transmission channels to be tested can be acquired by deploying one or more aperture radar acquisition components and optical acquisition components on the moving parts, or images of a single power transmission channel to be tested can be acquired from multiple angles.

[0067] As is understandable, SAR (Synthetic Aperture Radar) is an active Earth observation system. SAR images are formed from data collected by coherent radar. The radar emits radio frequency energy pulses towards the ground and measures the intensity of the reflected signal. The distance between the radar and the target is obtained by the round-trip time of the pulse. Therefore, SAR imaging is not limited by light conditions, and its radio frequency radiation is not significantly affected by clouds, rain, snow, or other atmospheric conditions. It can continuously acquire image data under any weather conditions, unaffected by weather, cloud cover, or day / night illumination, possessing all-weather imaging characteristics. Furthermore, SAR has strong penetration capabilities through vegetation and soil, making it possible to infer surface vegetation and soil characteristics. Its altitude independence allows it to be installed on aircraft, satellites, and other flying platforms, with a wider range of satellite orbit options. It can collect information over a large area at lower resolution or collect detailed, high-resolution images within a smaller area.

[0068] Therefore, combining optical images and synthetic aperture radar images can free the subsequent power transmission channel inspection process from the influence of environmental factors and improve the accuracy of power transmission channel hazard detection.

[0069] Step 204: Using a multi-scale feature extraction unit, perform multi-scale feature extraction on the optical image to obtain the first feature image.

[0070] For example, step 204 includes: using a first preliminary feature extraction module to perform preliminary feature extraction on the optical image to obtain an optical feature image, and using a multi-scale feature extraction unit to perform multi-scale feature extraction on the optical feature image to obtain a first feature image.

[0071] As one embodiment, the first preliminary feature extraction module includes at least two key feature enhancement sub-units connected in sequence. Using the first preliminary feature extraction module, preliminary feature extraction is performed on the optical image to obtain an optical feature image, including: passing the optical image through at least two key feature enhancement sub-units connected in sequence to obtain the optical feature image. Each key feature enhancement sub-unit includes a key feature enhancement convolutional layer, a normalization layer, and an activation function layer connected in sequence. The kernel size of the key feature enhancement convolutional layer in each key feature enhancement sub-unit is the same, but the stride of the key feature enhancement convolutional layer in each key feature enhancement sub-unit is different from the stride of the key feature enhancement convolutional layers in the previous and next key feature enhancement sub-units, respectively.

[0072] As one embodiment, the number of key feature enhancement sub-units included in the first preliminary feature extraction module is a multiple of 2. When the number of key feature enhancement sub-units included in the first preliminary feature extraction module is 4, the first and third key feature enhancement sub-units in the first preliminary feature extraction module include key feature enhancement convolutional layers with a kernel size of 3×3 and a stride of 1, and the second and fourth key feature enhancement sub-units in the first preliminary feature extraction module include key feature enhancement convolutional layers with a kernel size of 3×3 and a stride of 2.

[0073] Optionally, the specific implementation steps for using a multi-scale feature extraction unit to extract multi-scale features from the optical feature image to obtain the first feature image can be referred to in step 308 below, which describes the specific implementation of the multi-scale feature extraction step using a multi-scale feature extraction unit. These details will not be elaborated here.

[0074] Step 206: The first feature image is passed through at least two depth multi-scale feature extraction modules connected in sequence to obtain the second feature image output by each depth multi-scale feature extraction module. The depth multi-scale feature extraction module includes a downsampling unit and a multi-scale feature extraction unit.

[0075] For example, step 206 includes: for the first deep multi-scale feature extraction module, performing deep multi-scale feature extraction on the first feature image and outputting the second feature image output by the first deep multi-scale feature extraction module; for each non-first deep multi-scale feature extraction module, performing deep multi-scale feature extraction on the second feature image output by the previous deep multi-scale feature extraction module and outputting the second feature image output by the non-first deep multi-scale feature extraction module.

[0076] Step 208: Fuse the second feature images to obtain a third feature image, and perform channel hazard detection on the third feature image to obtain the position information of multiple target detection boxes.

[0077] For example, fusing the second feature images to obtain a third feature image includes: fusing the second feature images to obtain a third feature image through a preset fusion method, wherein the preset fusion method includes, but is not limited to, splicing method, addition method, multiplication method, attention mechanism method, deconvolution method and pyramid pooling method, etc.

[0078] Furthermore, by using a preset fusion method, the second feature images are fused to obtain a third feature image, including: fusing the second feature images through the neck network according to the preset fusion method to obtain a third feature image.

[0079] The aforementioned multi-scale feature extraction unit and at least two deep multi-scale feature extraction modules can form a backbone network.

[0080] For example, a power transmission channel hazard detection is performed on the third feature image to obtain the location information of multiple target detection boxes. This includes detecting power transmission channel hazards in the third feature image to obtain the location information of multiple target detection boxes. These power transmission channel hazards include, but are not limited to, floating object hazards, building hazards, tree hazards, and construction machinery hazards.

[0081] Furthermore, the potential hazards of the power transmission channel in the third feature image are detected to obtain the location information of multiple target detection boxes, including: detecting potential hazards of the power transmission channel in the third feature image through the detection head to obtain the location information of multiple target detection boxes.

[0082] As an example, refer to Figure 3The optical image is subjected to multi-scale feature extraction by a multi-scale feature extraction unit to obtain a first feature image. The first feature image is then passed through at least two depth multi-scale feature extraction modules connected in sequence to obtain a second feature image output by each depth multi-scale feature extraction module. The second feature images are then fused by a neck network to obtain a third feature image. The third feature image is then subjected to channel hazard detection by a detection head to obtain the position information of multiple target detection boxes.

[0083] Step 210: Based on the position information of multiple target detection boxes, the synthetic aperture radar image is sliced ​​to obtain a radar slice image corresponding to each target detection box.

[0084] For example, step 210 includes: for the position information of any target detection box, performing coordinate transformation on the position information of the target detection box to obtain transformed position information in the coordinate system corresponding to the synthetic aperture radar image; according to the transformed position information of each target detection box, mapping each target detection box to the synthetic aperture radar image to obtain a mapped image; according to the mapped image, slicing the mapped region in the synthetic aperture radar image corresponding to each target detection box to obtain a radar slice image corresponding to each target detection box.

[0085] Furthermore, the position information of the target detection box is transformed into coordinates to obtain the transformed position information in the coordinate system corresponding to the synthetic aperture radar image. This includes: converting the position information of the target detection box from optical image coordinates to projection coordinates to obtain the target projection coordinates; and converting the target projection coordinates to synthetic aperture radar coordinates to obtain the transformed position information in the coordinate system corresponding to the synthetic aperture radar image.

[0086] As one embodiment, the position information of the target detection box is converted from optical image coordinates to projected coordinates to obtain the target projected coordinates. This includes: acquiring the image parameters of the optical image, the projected coordinates of a preset position in the optical image, the row coordinates and column coordinates of each pixel of the target detection box in the optical image, and converting each pixel of the target detection box from optical image coordinates to projected coordinates based on the image parameters of the optical image, the projected coordinates of the preset position in the optical image, and the row coordinates and column coordinates of each pixel of the target detection box in the optical image. The pixel projected coordinates of each pixel of the target detection box are then fused to obtain the target projected coordinates in the projected coordinate system. In this paper, the preset position in the image (optical image and SAR image) can be any position in the image, including but not limited to the top corner, edge, and center positions.

[0087] Furthermore, the image parameters of the optical image include optical image row rotation parameters, optical image column rotation parameters, optical image horizontal pixel width, and optical image vertical pixel width. The projection coordinates of the preset position in the optical image include optical projection abscissa and optical projection ordinate. The pixel projection coordinates include the first projection coordinate of the pixel in the horizontal direction and the second projection coordinate of the pixel in the vertical direction. Based on the image parameters of the optical image, the projection coordinates of the preset position in the optical image, and the pixel row coordinates and pixel column coordinates of each pixel in the target detection box in the optical image, each pixel of the target detection box is converted from optical image coordinates to... The projection coordinates of a pixel are obtained by: determining the first projection coordinates of the pixel in the horizontal direction based on the optical projection abscissa, the column coordinates of each pixel in the target detection box in the optical image, the row rotation parameters of the optical image, the row coordinates of each pixel in the target detection box in the optical image, and the horizontal pixel width of the optical image; and determining the second projection coordinates of the pixel in the vertical direction based on the optical projection ordinate, the column coordinates of each pixel in the target detection box in the optical image, the column rotation parameters of the optical image, the row coordinates of each pixel in the target detection box in the optical image, and the vertical pixel width of the optical image.

[0088] Optionally, the first projection coordinates of a pixel in the horizontal direction can be determined by the following formula: based on the optical projection abscissa, the column coordinates of each pixel in the target detection box in the optical image, the row rotation parameters of the optical image, the row coordinates of each pixel in the target detection box in the optical image, and the horizontal pixel width of the optical image.

[0089]

[0090] in, Let be the first projected coordinates of the pixel in the horizontal direction. The horizontal axis of the optical projection. Let each pixel of the target detection bounding box be represented by its column coordinates in the optical image. For optical image row rotation parameters, Let be the row coordinates of each pixel in the target detection bounding box within the optical image. The horizontal pixel width of the optical image.

[0091] Optionally, the second projection coordinates of a pixel in the vertical direction can be determined based on the optical projection ordinate, the column coordinates of each pixel in the target detection box in the optical image, the column rotation parameters of the optical image, the row coordinates of each pixel in the target detection box in the optical image, and the vertical pixel width of the optical image. This can be expressed by the following formula:

[0092]

[0093] in, This represents the second projected coordinates of the pixel in the vertical direction. The optical projection ordinate, For optical image column rotation parameters, The vertical pixel width of the optical image.

[0094] As one embodiment, the target projection coordinates are converted into synthetic aperture radar (SAR) coordinates to obtain the transformed position information in the coordinate system corresponding to the SAR image. This includes: obtaining the linear transformation equation, the horizontal pixel width of the SAR image, the vertical pixel width of the SAR image, the row rotation parameters of the SAR image, the column rotation parameters of the SAR image, the column coordinates of pixels in the SAR image, the row coordinates of pixels in the SAR image, and the projected abscissa and ordinate of the preset position in the SAR SAR SAR image. The projected abscissa and ordinate of the preset position in the SAR SAR SAR image can be determined using the same method as the first and second projected coordinates described above. Without going into details; based on the horizontal pixel width, vertical pixel width, row rotation parameters, column rotation parameters, column coordinates, row coordinates, and projected abscissa and ordinate of the preset position in the SAR synthetic aperture radar image, the target's projected coordinates are converted into synthetic aperture radar coordinates to obtain the SAR coordinates of the pixel in the horizontal direction and the SAR coordinates of the pixel in the vertical direction; the SAR coordinates of each pixel in the horizontal direction and the SAR coordinates of each pixel in the vertical direction are fused to obtain the labeled converted image in the coordinate system corresponding to the synthetic aperture radar image.

[0095] Optionally, the linear transformation equation may specifically include:

[0096]

[0097] in, For SAR image column rotation parameters, The horizontal pixel width of the SAR image. For SAR image column rotation parameters, The vertical pixel width of the SAR image. These are the column coordinates of pixels in the SAR image. These are the row coordinates of pixels in the SAR image. Here are the SAR coordinates of the pixel in the horizontal direction. These are the SAR coordinates of the pixel in the vertical direction. The x-coordinate of the projected position in the SAR synthetic aperture radar image. This represents the projected ordinate of a preset position in a SAR synthetic aperture radar image.

[0098] The mapped image is used to characterize the location of the labeled region in the synthetic aperture radar image.

[0099] As one embodiment, based on the mapped image, the mapped regions corresponding to each target detection box in the synthetic aperture radar image are sliced ​​to obtain a radar slice image corresponding to each target detection box. This includes: identifying the mapped regions corresponding to each target detection box in the synthetic aperture radar image based on the mapped image; determining the segmentation regions corresponding to each target detection box based on the mapped regions corresponding to each target detection box in the synthetic aperture radar image, wherein the size of the segmentation region corresponding to each target detection box is greater than or equal to the size of the mapped region corresponding to the target detection box, and the segmentation region corresponding to each target detection box includes the mapped region corresponding to the target detection box. The segmentation region corresponding to each target detection box may also include the peripheral region of the mapped region corresponding to the target detection box.

[0100] Step 212: Classify each radar slice image to obtain the slice classification result corresponding to each radar slice image.

[0101] The slice classification results are used to characterize at least one of the following: background image type, floating object hazard image type, building hazard image type, tree hazard image type, and construction machinery hazard image type.

[0102] For example, step 212 includes: performing feature enhancement on any radar slice image to obtain a target enhanced feature image; classifying the radar slice image based on the target enhanced feature image to obtain the slice classification result corresponding to the radar slice image.

[0103] The process of classifying radar slice images based on target enhancement feature images to obtain slice classification results can be performed by a slice classification model. The training process of the slice classification model includes: acquiring multiple training samples and a slice classification model to be trained. Each training sample includes a training target enhancement feature image corresponding to the training transmission channel and a real slice type label corresponding to the training transmission channel; a mapping step: mapping the training target enhancement feature image corresponding to the training transmission channel to training slice type information through the slice classification model to be trained; calculating the loss between the training slice type information and the real slice type label corresponding to the training transmission channel. When the loss tends to converge, the current slice classification model to be trained is determined as the slice classification model. When the loss does not tend to converge, the model of the slice classification model to be trained is updated. The model update method includes, but is not limited to, gradient ascent and gradient descent, and the process returns to the mapping step until the loss tends to converge.

[0104] Step 214: Based on the classification results of each slice, determine the detection results of potential hazards in the power transmission channel to be inspected.

[0105] In step 214, the power transmission channel hazard detection results are used to characterize at least one of the power transmission channel status and hazard type. The power transmission channel status is used to characterize whether there is a hazard or not. If the power transmission channel status indicates that there is a hazard, the power transmission channel hazard detection results can also characterize the hazard type. The hazard type includes at least one of the following: floating object hazard type, building hazard type, tree hazard type, and construction machinery hazard type.

[0106] For example, step 214 includes: determining the detection result of potential hazards in the power transmission channel corresponding to the power transmission channel to be detected based on the slice type represented by each slice classification result.

[0107] Furthermore, based on the slice type represented by each slice classification result, the detection result of the hidden danger of the transmission channel to be inspected is determined, including: if the slice type represented by each slice classification result contains a floating object image type, then the floating object image type is determined as the hidden danger detection result of the transmission channel to be inspected; if the slice type represented by each slice classification result contains a building image type, then the building image type is determined as the hidden danger detection result of the transmission channel to be inspected; if the slice type represented by each slice classification result contains a tree image type, then the tree image type is determined as the hidden danger detection result of the transmission channel to be inspected; if the slice type represented by each slice classification result contains a construction machinery image type, then the construction machinery image type is determined as the hidden danger detection result of the transmission channel to be inspected; if the slice type represented by each slice classification result only contains a background image type, then the hidden danger detection result without any hidden danger is determined as the hidden danger detection result of the transmission channel to be inspected.

[0108] In the aforementioned method for detecting hidden dangers in power transmission channels, synthetic aperture radar (SAR) images and optical images are used together as image processing sources. This allows the extracted features to be unrestricted by the environment of the power transmission channel to be detected. Subsequently, a multi-scale feature extraction unit extracts multi-scale features from the optical image to obtain a first feature image. This ensures that the input features can represent multi-scale feature information in terms of input data dimension. Furthermore, the first feature image is passed through at least two sequentially connected depth multi-scale feature extraction modules to obtain second feature images output by each depth multi-scale feature extraction module. This ensures that the feature extraction process is multi-scale in terms of feature extraction dimension. The second feature images are then fused to obtain a third feature image. The third feature image is then used for channel hidden danger detection to obtain the position information of multiple target detection boxes. That is, the multiple second feature images obtained from multi-scale feature extraction are fused to obtain further multi-scale features. The third feature image, extracted from the first feature, is used as the basis for detecting potential hazards in the transmission channel. This improves the accuracy of target detection. Furthermore, based on the position information of the target detection box, the synthetic aperture radar image is sliced ​​to obtain multiple radar slice images. These radar slice images can, to a certain extent, represent the preliminary results obtained from the optical image processing. Each radar slice image is then classified to obtain the corresponding slice classification results. Finally, based on the slice classification results, the detection result for potential hazards in the transmission channel to be detected is determined. As the above analysis shows, the radar slice image is obtained after layers of feature extraction and processing. Therefore, the classified slice results are more accurate, and the determined detection result for potential hazards in the transmission channel is more consistent with the actual situation of the transmission channel to be detected, thus improving the accuracy of potential hazard detection in the transmission channel.

[0109] In one exemplary embodiment, such as Figure 4 As shown, a method for accurately processing optical images is provided, the method further including steps 302 to 308. Wherein:

[0110] Step 302: Obtain the first input feature image of the downsampling unit, wherein the first input feature image is either the first feature image or the second feature image output by the previous depth multi-scale feature extraction module.

[0111] Step 304: The first input feature image is downsampled by the downsampling unit to obtain a first downsampled feature image; and the first input feature image is downsampled by the downsampling unit to obtain a second downsampled feature image. The first downsampling is used to enhance the key features of the first input feature image; the second downsampling is used to enhance the multi-channel sampling features of the first input feature image.

[0112] For example, the downsampling unit includes a first type of downsampling unit and a second type of downsampling unit; step 304 includes: downsampling the most critical features of the first input feature image through the first type of downsampling unit to obtain a downsampled feature extraction image; enhancing the key features of the downsampled feature extraction image through the first type of downsampling unit to obtain a first downsampled feature image; enhancing the key features of the first input feature image through the second type of downsampling unit to obtain a downsampled feature enhanced image; and performing multi-channel downsampling on the downsampled feature enhanced image through the second type of downsampling unit to obtain a second downsampled feature image.

[0113] The process of downsampling the most critical features of the first input feature image using the first type of downsampling unit to obtain the downsampled feature extraction image can be performed by the max pooling layer in the first type of downsampling unit. The process of enhancing the critical features of the downsampled feature extraction image using the first type of downsampling unit to obtain the first downsampled feature image can be performed by the critical feature enhancement subunit in the first type of downsampling unit. The convolutional kernel size of the critical feature enhancement convolutional layer in the critical feature enhancement subunit can be 1×1, and the stride can be 1. The process of enhancing the critical features of the first input feature image using the second type of downsampling unit to obtain the downsampled feature enhancement image can be performed by the critical feature enhancement subunit in the second type of downsampling unit. The convolutional kernel size of the critical feature enhancement convolutional layer in the critical feature enhancement subunit can be 1×1, and the stride can be 1. The process of multi-channel downsampling of the downsampled feature enhancement image using the second type of downsampling unit to obtain the second downsampled feature image can be performed by the channel-wise convolutional layer in the second type of downsampling unit. The convolutional kernel size of the channel-wise convolutional layer can be 3×3, and the stride can be 2.

[0114] Step 306: Fuse the first downsampled feature image and the second downsampled feature image to obtain the target downsampled feature image.

[0115] For example, step 306 includes: performing weighted fusion on the first downsampled feature image and the second downsampled feature image to obtain the target downsampled feature image.

[0116] Step 308: Use the target downsampled feature image as the second input feature map of the multi-scale feature extraction unit, and use the multi-scale feature extraction unit to perform multi-scale feature extraction to obtain the second feature image output by the depth multi-scale feature extraction module.

[0117] For example, the multi-scale feature extraction unit includes a channel number expansion subunit, a channel dimension slicing subunit, a splicing subunit, at least two lightweight feature enhancement subunits, and a multi-scale attention extraction subunit. Multi-scale feature extraction using the multi-scale feature extraction unit includes: acquiring a third input feature image of the multi-scale feature extraction unit, and expanding the channel number of the third input feature image using the channel number expansion subunit to obtain a channel number expanded image, wherein the third input feature image is an optical image or a target downsampled feature image; and slicing the channel number expanded image along the channel dimension using the channel dimension slicing subunit to obtain a first sliced ​​image and a second sliced ​​image with the same number of channels. The image is segmented; the first segmented image is enhanced by passing it through at least two sequentially connected lightweight feature enhancement subunits to obtain lightweight feature enhancement images output by each lightweight feature enhancement subunit; the lightweight feature enhancement images are then stitched together by a stitching subunit to obtain a first stitched image; the first stitched image is enhanced by multi-scale attention feature enhancement through a multi-scale attention extraction subunit to obtain a first multi-scale feature enhancement image; the second segmented image is enhanced by multi-scale attention feature enhancement through a multi-scale attention extraction subunit to obtain a second multi-scale feature enhancement image; and the first multi-scale feature enhancement image and the second multi-scale feature enhancement image are stitched together by a stitching subunit.

[0118] The channel expansion subunit comprises a convolutional layer with a kernel size of 1×1 and a stride of 1, a normalization layer, and an activation function layer. The lightweight feature enhancement subunit comprises at least two sequentially connected key feature enhancement subunits. The key feature enhancement convolutional layers within the lightweight feature enhancement subunits have the same kernel size and stride; the kernel size can be 3×3 and the stride can be 1. The lightweight feature enhancement image output by the lightweight feature enhancement subunit is a fused image between the input image of the lightweight feature enhancement subunit and the output image of the last key feature enhancement subunit within the lightweight feature enhancement subunit.

[0119] As one embodiment, multi-scale attention feature enhancement is performed on the first stitched image to obtain a first multi-scale feature-enhanced image, including: obtaining pre-set hyperparameters, wherein the hyperparameters are the divisors of the number of channels of the input image corresponding to the multi-scale attention extraction sub-unit (here, the first stitched image); segmenting the first stitched image according to the hyperparameters to obtain multiple hyperparameter-segmented images, wherein the number of channels among the multiple hyperparameter-segmented images is the same, and the sum of the number of channels among the multiple hyperparameter-segmented images is equal to the number of channels of the first stitched image; performing multi-scale feature extraction on the multiple hyperparameter-segmented images respectively to obtain multiple scale feature images; and fusing the multiple scale feature images according to the attention weight information corresponding to the multiple scale feature images to obtain the first multi-scale feature-enhanced image.

[0120] Furthermore, multi-scale feature extraction is performed on multiple hyperparameter-segmented images to obtain multiple scale feature images. This includes: for any hyperparameter-segmented image, determining the kernel size of the convolutional layer used for feature extraction based on the segmentation number of the hyperparameter-segmented image; determining the number of convolutional channels of the convolutional layer used for feature extraction based on the kernel size of the convolutional layer used for feature extraction; and performing scale feature extraction on the high-dimensional segmented feature image based on the kernel size and the number of convolutional channels of the convolutional layer used for feature extraction, to obtain a scale feature image.

[0121] Optionally, the kernel size of the convolutional layer used for feature extraction of the hyperparameter-segmented image is determined based on the segmentation number corresponding to the hyperparameter-segmented image. This can be expressed by the following formula:

[0122]

[0123] in, This refers to the kernel size of the convolutional layer used for feature extraction in hyperparameter-segmented images. This refers to the segmentation number corresponding to the hyperparameter-segmented image.

[0124] Optionally, the number of convolutional channels of the convolutional layer used for feature extraction of the hyperparameter-segmented image is determined based on the kernel size of the convolutional layer used for feature extraction. This can be expressed by the formula:

[0125]

[0126] in, This refers to the number of convolutional channels in the convolutional layer used for feature extraction from hyperparameter-segmented images.

[0127] Optionally, based on the kernel size and number of convolution channels of the convolutional layer used for feature extraction of the hyperparameter-segmented image, scale feature extraction is performed on the high-dimensional segmented feature image to obtain a scale feature image, which can be expressed by the formula:

[0128]

[0129] in, For scale feature images, For the first Image segmentation by hyperparameters, This is a hyperparameter.

[0130] As one embodiment, the first multi-scale feature enhancement image is obtained by fusing multiple scale feature images according to the attention weight information corresponding to multiple scale feature images, including: performing attention stitching on multiple scale feature images according to the attention weight information corresponding to multiple scale feature images to obtain an attention fusion feature image, and performing channel-by-channel fusion on the attention fusion feature image to obtain the first multi-scale feature enhancement image.

[0131] As an example, refer to Figure 5 When the size of the third input feature image is 128×128×128, corresponding to the length, width, and number of channels of the feature map, a channel-expanded image is obtained after passing through a channel-expanding subunit with a convolution kernel size of 1×1 and a stride of 1. The size of the channel-expanded image is 128×128×256. The channel-expanded image is then segmented to obtain a first segmented image and a second segmented image, both with a size of 128×128×128. The first segmented image is then processed by a first lightweight feature enhancement subunit to obtain a first lightweight feature-enhanced image. The first lightweight feature-enhanced image is then processed by a second lightweight feature enhancement subunit to obtain a second lightweight feature-enhanced image. Finally, the second lightweight feature-enhanced image is processed by a third lightweight feature enhancement subunit to obtain a third lightweight feature-enhanced image. The first, second, and third lightweight feature enhancement images are all 128×128×128 in size. These three images are then stitched together to obtain a first stitched image, which is 128×128×384 in size. Multi-scale attention feature enhancement is then applied to the first stitched image to obtain a first multi-scale feature enhancement image, also 128×128×384 in size. Multi-scale attention feature enhancement is then applied to the second segmented image to obtain a second multi-scale feature enhancement image, which is also 128×128×128 in size. Finally, the first and second multi-scale feature enhancement images are stitched together to obtain a second feature image.

[0132] As an example, refer to Figure 6When the image size of the first input feature image is 128×128×256, the most critical features of the first input feature image are downsampled to obtain a downsampled feature extraction image with an image size of 64×64×256. The downsampled feature extraction image is then enhanced with critical features using a first type of downsampling unit to obtain a first downsampled feature image with an image size of 64×64×256. The first input feature image is then enhanced with critical features using a second type of downsampling unit to obtain a downsampled enhanced feature image with an image size of 128×128×256. The enhanced downsampled feature image is then subjected to multi-channel downsampling using a second type of downsampling unit to obtain a second downsampled feature image with an image size of 64×64×256. Finally, the first downsampled feature image and the second downsampled feature image are fused to obtain a target downsampled feature image with a size of 64×64×256.

[0133] In this embodiment, by applying two feature enhancement downsampling methods to the first input feature image respectively, fusing the sampled feature images obtained by the two feature enhancement downsampling methods, and performing multi-scale feature extraction on the fused target downsampled feature image, the extracted second feature map can represent multi-scale feature information, thereby improving the accuracy of feature extraction from optical images.

[0134] In one exemplary embodiment, such as Figure 7 As shown, a method for accurately classifying radar slice images is provided, which involves classifying each radar slice image separately to obtain the slice classification result corresponding to each radar slice image, including steps 402 to 408. Wherein:

[0135] Step 402: For any radar segmented image, perform multi-channel downsampling on the radar segmented image to obtain a radar segmented downsampled feature image.

[0136] For example, step 402 includes: for any radar segmented image, using the second preliminary feature extraction module to perform preliminary feature extraction on the radar segmented image to obtain a radar segmented feature image; performing channel expansion on the radar segmented feature image to obtain a radar segmented expanded image; performing spatial feature enhancement on the radar segmented expanded image to obtain a radar segmented expanded enhanced image; and performing multi-channel downsampling on the radar segmented expanded enhanced image to obtain a radar segmented downsampled feature image.

[0137] Further, the second preliminary feature extraction module includes a key feature convolutional layer and at least two inverted residual bottleneck convolutional layers connected in sequence. The kernel size of the key feature convolutional layer in the second preliminary feature extraction module can be 3×3, and the stride can be 2. The kernel size of the first inverted residual bottleneck convolutional layer can be 3×3, and the stride can be 1. The kernel size of subsequent inverted residual bottleneck convolutional layers can be 3×3, and the stride can be 2. Further, spatial feature enhancement is performed on the radar segmented and expanded image to obtain a radar segmented and expanded enhanced image. This includes: decomposing the channel attention of the radar segmented and expanded image to obtain spatial aggregation features in two spatial directions; and performing complementary fusion enhancement on the spatial aggregation features in the two spatial directions to obtain the radar segmented and expanded enhanced image. The process of multi-channel downsampling of the radar segmented and expanded enhanced image to obtain the radar segmented downsampled feature image is executed by a channel-wise convolutional layer. The kernel size of the channel-wise convolutional layer can be 3×3, and the stride can be 1 or 2.

[0138] Step 404: Perform spatial feature enhancement on the radar segmented downsampled feature image to obtain the radar segmented enhanced feature image.

[0139] Optionally, the specific implementation steps for performing spatial feature enhancement on the radar segmented downsampled feature image to obtain the radar segmented enhanced feature image can refer to the specific implementation content of the steps for performing spatial feature enhancement on the radar segmented expanded image to obtain the radar segmented expanded enhanced image, which will not be repeated here.

[0140] Step 406: Determine the target enhancement feature image corresponding to the radar segmentation image based on the radar segmentation enhancement feature image.

[0141] For example, step 406 includes: determining the image size relationship between the radar segmentation enhancement feature image and the radar segmentation image; if the image size relationship indicates that the radar segmentation enhancement feature image and the radar segmentation image have the same image size, then the radar segmentation enhancement feature image and the radar segmentation image are added together to obtain the target enhancement feature image corresponding to the radar segmentation image; if the image size relationship indicates that the radar segmentation enhancement feature image and the radar segmentation image have different image sizes, then the radar segmentation enhancement feature image is determined as the target enhancement feature image corresponding to the radar segmentation image.

[0142] Further, determining the image size relationship between the radar segmentation enhancement feature image and the radar segmentation image includes: if the stride of the channel-wise convolutional layer corresponding to the synthetic aperture radar segmentation image is a first value, then the image size relationship between the radar segmentation enhancement feature image and the radar segmentation image indicates that the image sizes of the radar segmentation enhancement feature image and the radar segmentation image are the same; if the stride of the channel-wise convolutional layer corresponding to the synthetic aperture radar segmentation image is a second value, then the image size relationship between the radar segmentation enhancement feature image and the radar segmentation image indicates that the image sizes of the radar segmentation enhancement feature image and the radar segmentation image are different; the first value is less than the second value. As an embodiment, the first value is 1, and the second value is 2.

[0143] As an example, refer to Figure 8 When the stride of the channel-wise convolutional layer corresponding to the synthetic aperture radar segmented image is 1, the radar segmented enhancement feature image and the radar segmented image are added together to obtain the target enhancement feature image corresponding to the radar segmented image. When the stride of the channel-wise convolutional layer corresponding to the synthetic aperture radar segmented image is 2, the radar segmented enhancement feature image is determined as the target enhancement feature image corresponding to the radar segmented image.

[0144] Step 408: Based on the target enhancement feature image corresponding to each radar slice image, classify each radar slice image to obtain the slice classification result corresponding to each radar slice image.

[0145] For example, step 408 includes: classifying each radar slice image according to the target enhancement feature image corresponding to each radar slice image using a slice classification model, and obtaining the slice classification result corresponding to each radar slice image.

[0146] In this embodiment, by performing multi-channel downsampling and spatial feature enhancement on each radar segmented image, it is ensured that long-range dependencies can be captured along one spatial direction while retaining accurate positional information along another spatial direction. Subsequent channel-wise convolution and attention encoding enhance the attention representation in the radar segmented image processing, improving the accuracy of feature extraction from the target enhanced feature image. Since the target enhanced feature image is the basis for slice classification, the slicing accuracy of the radar slice image can be improved.

[0147] As a detailed embodiment, an optical image and a synthetic aperture radar (SAR) image of the power transmission channel to be detected are acquired; a multi-scale feature extraction unit is used to extract multi-scale features from the optical image to obtain a first feature image; the first feature image is passed through at least two depth multi-scale feature extraction modules connected in sequence to obtain a second feature image output by each depth multi-scale feature extraction module, wherein the depth multi-scale feature extraction module includes a downsampling unit and a multi-scale feature extraction unit; the second feature images are fused to obtain a third feature image, and channel hazard detection is performed on the third feature image to obtain the position information of multiple target detection boxes; for the position information of any target detection box, the position information of the target detection box is transformed to obtain transformed position information in the coordinate system corresponding to the SAR image; according to the transformed position information of each target detection box, each target detection box is mapped to the SAR image to obtain a mapped image; according to the mapped image, the mapped region in the SAR image corresponding to each target detection box is sliced ​​to obtain a radar slice image corresponding to each target detection box.

[0148] Furthermore, for any radar segmented image, multi-channel downsampling is performed on the radar segmented image to obtain a radar segmented downsampled feature image; spatial feature enhancement is performed on the radar segmented downsampled feature image to obtain a radar segmented enhanced feature image; the image size relationship between the radar segmented enhanced feature image and the radar segmented image is determined; if the image size relationship indicates that the radar segmented enhanced feature image and the radar segmented image have the same image size, then the radar segmented enhanced feature image and the radar segmented image are added together to obtain the target enhanced feature image corresponding to the radar segmented image; if the image size relationship indicates that the radar segmented enhanced feature image and the radar segmented image have different image sizes, then the radar segmented enhanced feature image is determined as the target enhanced feature image corresponding to the radar segmented image; based on the target enhanced feature image corresponding to each radar slice image, each radar slice image is classified to obtain the slice classification result corresponding to each radar slice image.

[0149] Specifically, for any depth-multiscale feature extraction module, the first input feature image of the downsampling unit is obtained, where the first input feature image is either the first feature image or the second feature image output by the previous depth-multiscale feature extraction module. The most critical features of the first input feature image are downsampled using a first-type downsampling unit to obtain a downsampled feature extraction image. Key features of the downsampled feature extraction image are enhanced using a first-type downsampling unit to obtain a first downsampled feature image. Key features of the first input feature image are enhanced using a second-type downsampling unit to obtain a downsampled enhanced feature image. The downsampled enhanced feature image is then downsampled using a second-type downsampling unit through multi-channel downsampling to obtain a second downsampled feature image. The first and second downsampled feature images are fused to obtain a target downsampled feature image. The target downsampled feature image is used as the second input feature image of the multiscale feature extraction unit to obtain a third input feature image. The third input feature image is then processed using a channel expansion subunit. The feature image is expanded by channel number to obtain an expanded channel image, wherein the third input feature image is an optical image or a target downsampled feature image. The expanded channel image is then segmented along the channel dimension by a channel-dimensional segmentation unit to obtain a first segmented image and a second segmented image with the same number of channels. The first segmented image is then enhanced by passing it through at least two sequentially connected lightweight feature enhancement subunits to obtain lightweight feature enhancement images output by each lightweight feature enhancement subunit. These lightweight feature enhancement images are then stitched together by a stitching subunit to obtain a first stitched image. The first stitched image is then enhanced by multi-scale attention features through a multi-scale attention extraction subunit to obtain a first multi-scale feature enhancement image. The second segmented image is then enhanced by multi-scale attention features through a multi-scale attention extraction subunit to obtain a second multi-scale feature enhancement image. Finally, the first multi-scale feature enhancement image and the second multi-scale feature enhancement image are stitched together by a stitching subunit to obtain the second feature image output by the depth multi-scale feature extraction module.

[0150] Thus, by using both synthetic aperture radar (SAR) images and optical images as image processing sources, the extracted features are not limited by the environment of the power transmission channel being inspected. Subsequently, a multi-scale feature extraction unit performs multi-scale feature extraction on the optical image to obtain a first feature image. This ensures that the input features can represent multi-scale feature information in the input data dimension. Furthermore, the first feature image is passed through at least two sequentially connected deep multi-scale feature extraction modules to obtain second feature images output by each module. This ensures that the feature extraction process is multi-scale in the feature extraction dimension. The second feature images are then fused to obtain a third feature image, and channel hazard detection is performed on this third feature image to obtain the position information of multiple target detection boxes. In other words, multiple second feature images obtained from multi-scale feature extraction are fused to obtain further multi-scale feature extraction. The third feature image is used as the basis for detecting potential hazards in the transmission channel, thus improving the accuracy of target detection. Furthermore, based on the position information of the target detection box, the synthetic aperture radar image is sliced ​​to obtain multiple radar slice images. These radar slice images can, to a certain extent, represent the preliminary results obtained from the optical image processing. Each radar slice image is then classified to obtain the corresponding slice classification results. Finally, based on the slice classification results, the detection result of potential hazards in the transmission channel to be detected is determined. As the above analysis shows, the radar slice image is obtained after layers of feature extraction and processing. Therefore, the slice classification results obtained are more accurate, and the determined detection result of potential hazards in the transmission channel is more consistent with the actual situation of the transmission channel to be detected, thus improving the accuracy of potential hazard detection in the transmission channel.

[0151] Furthermore, by applying two different feature enhancement downsampling methods to the first input feature image and fusing the sampled feature images obtained from the two downsampling methods, and then performing multi-scale feature extraction on the fused target downsampled feature image, the extracted second feature map can represent multi-scale feature information, thus improving the accuracy of feature extraction from optical images. Additionally, multi-channel downsampling and spatial feature enhancement are performed on each radar segmented image to ensure that long-range dependencies can be captured along one spatial direction while retaining precise positional information along another spatial direction. Subsequent channel-wise convolution and attention encoding enhance the attention representation in the radar segmented image processing, improving the accuracy of feature extraction from the target enhanced feature image. Since the target enhanced feature image is the basis for slice classification, the slicing accuracy of radar slice images can be improved.

[0152] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. Based on the same inventive concept, this application also provides a power transmission channel hidden danger detection device for implementing the above-mentioned power transmission channel hidden danger detection method. The solution provided by this device is similar to the solution described in the above-described method. Therefore, the specific limitations of one or more embodiments of the power transmission channel hidden danger detection device provided below can be found in the limitations of the power transmission channel hidden danger detection method above, and will not be repeated here.

[0153] In one exemplary embodiment, such as Figure 9 As shown, a power transmission channel hazard detection device 900 is provided, including: an acquisition module 902, an extraction module 904, a detection module 906, a slicing module 908, and a classification module 910, wherein:

[0154] The acquisition module 902 acquires optical images and synthetic aperture radar images of the power transmission channel to be detected;

[0155] Extraction module 904 is used to perform multi-scale feature extraction on optical images using a multi-scale feature extraction unit to obtain a first feature image; the first feature image is passed through at least two depth multi-scale feature extraction modules connected in sequence to obtain a second feature image output by each depth multi-scale feature extraction module, wherein the depth multi-scale feature extraction module includes a downsampling unit and a multi-scale feature extraction unit; the second feature images are fused to obtain a third feature image;

[0156] Detection module 906 is used to perform channel hazard detection on the third feature image and obtain the position information of multiple target detection boxes;

[0157] The slicing module 908 is used to slice the synthetic aperture radar image based on the position information of multiple target detection boxes to obtain a radar slice image corresponding to each target detection box.

[0158] The classification module 910 is used to classify each radar slice image separately and obtain the slice classification result corresponding to each radar slice image;

[0159] The detection module 906 is also used to determine the detection results of potential hazards in the power transmission channel corresponding to the power transmission channel to be detected based on the classification results of each slice.

[0160] In one embodiment, the extraction module 904 is further configured to acquire a first input feature image of the downsampling unit, wherein the first input feature image is a first feature image, or a second feature image output by the previous depth multi-scale feature extraction module; perform a first type of downsampling on the first input feature image through the downsampling unit to obtain a first downsampled feature image, and perform a second type of downsampling on the first input feature image through the downsampling unit to obtain a second downsampled feature image, wherein the first type of downsampling is used to enhance the key features of the first input feature image; the second type of downsampling is used to enhance the multi-channel sampling features of the first input feature image; fuse the first downsampled feature image and the second downsampled feature image to obtain a target downsampled feature image; use the target downsampled feature image as the second input feature image of the multi-scale feature extraction unit, and perform multi-scale feature extraction using the multi-scale feature extraction unit to obtain the second feature image output by the depth multi-scale feature extraction module.

[0161] In one embodiment, the downsampling unit includes a first type of downsampling unit and a second type of downsampling unit; the extraction module 904 is further configured to perform the most critical feature downsampling on the first input feature image through the first type of downsampling unit to obtain a downsampled feature extraction image; perform key feature enhancement on the downsampled feature extraction image through the first type of downsampling unit to obtain a first downsampled feature image; perform key feature enhancement on the first input feature image through the second type of downsampling unit to obtain a downsampled feature enhanced image; and perform multi-channel downsampling on the downsampled feature enhanced image through the second type of downsampling unit to obtain a second downsampled feature image.

[0162] In one embodiment, the multi-scale feature extraction unit includes a channel number expansion subunit, a channel dimension slicing subunit, a splicing subunit, at least two lightweight feature enhancement subunits, and a multi-scale attention extraction subunit. The extraction module 904 is further configured to acquire a third input feature image of the multi-scale feature extraction unit, and through the channel number expansion subunit, expand the third input feature image by channel number to obtain a channel number expanded image, wherein the third input feature image is an optical image or a target downsampled feature image; and through the channel dimension slicing subunit, segment the channel number expanded image along the channel dimension to obtain a first segmented image and a second segmented image with the same number of channels. The first segmented image is enhanced by passing it through at least two sequentially connected lightweight feature enhancement subunits to obtain lightweight feature enhancement images output by each lightweight feature enhancement subunit. These lightweight feature enhancement images are then stitched together by a stitching subunit to obtain a first stitched image. A multi-scale attention feature enhancement is performed on the first stitched image by a multi-scale attention extraction subunit to obtain a first multi-scale feature enhancement image. Similarly, a multi-scale attention enhancement is performed on the second segmented image by a multi-scale attention extraction subunit to obtain a second multi-scale feature enhancement image. Finally, the first multi-scale feature enhancement image and the second multi-scale feature enhancement image are stitched together by a stitching subunit.

[0163] In one embodiment, the classification module 910 is further configured to perform multi-channel downsampling on any radar segmented image to obtain a radar segmented downsampled feature image; perform spatial feature enhancement on the radar segmented downsampled feature image to obtain a radar segmented enhanced feature image; determine the target enhanced feature image corresponding to the radar segmented image based on the radar segmented enhanced feature image; and classify each radar segmented image according to the target enhanced feature image corresponding to each radar segmented image to obtain the segment classification result corresponding to each radar segmented image.

[0164] In one embodiment, the classification module 910 is further configured to determine the image size relationship between the radar segmentation enhancement feature image and the radar segmentation image; if the image size relationship indicates that the radar segmentation enhancement feature image and the radar segmentation image have the same image size, then the radar segmentation enhancement feature image and the radar segmentation image are added together to obtain the target enhancement feature image corresponding to the radar segmentation image; if the image size relationship indicates that the radar segmentation enhancement feature image and the radar segmentation image have different image sizes, then the radar segmentation enhancement feature image is determined as the target enhancement feature image corresponding to the radar segmentation image.

[0165] In one embodiment, the slicing module 908 is further configured to perform coordinate transformation on the position information of any target detection box to obtain transformed position information in the coordinate system corresponding to the synthetic aperture radar image; based on the transformed position information of each target detection box, map each target detection box to the synthetic aperture radar image to obtain a mapped image; and based on the mapped image, slice the mapped region in the synthetic aperture radar image corresponding to each target detection box to obtain a radar slice image corresponding to each target detection box.

[0166] Each module in the aforementioned power transmission channel hazard detection device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0167] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 10 As shown, the computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements a method for detecting potential hazards in power transmission channels. The display unit is used to form a visually visible image and can be a display screen, projection device, or virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0168] Those skilled in the art will understand that Figure 10The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0169] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0170] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0171] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0172] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0173] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0174] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for detecting hidden dangers in power transmission channels, characterized in that, The method includes: Acquire optical and synthetic aperture radar images of the power transmission channel to be inspected; The optical image is subjected to multi-scale feature extraction using a multi-scale feature extraction unit to obtain a first feature image; The first feature image is passed through at least two depth multi-scale feature extraction modules connected in sequence to obtain the second feature image output by each depth multi-scale feature extraction module, wherein the depth multi-scale feature extraction module includes a downsampling unit and a multi-scale feature extraction unit; The second feature images are fused to obtain a third feature image, and channel hazard detection is performed on the third feature image to obtain the position information of multiple target detection boxes; Based on the position information of the multiple target detection boxes, the synthetic aperture radar image is sliced ​​to obtain a radar slice image corresponding to each target detection box. Each radar slice image is classified to obtain the slice classification result corresponding to each radar slice image; Based on the classification results of each slice, the detection results of potential hazards in the power transmission channel corresponding to the power transmission channel to be detected are determined.

2. The method according to claim 1, characterized in that, For any of the aforementioned deep multi-scale feature extraction modules, the method further includes: Obtain the first input feature image of the downsampling unit, wherein the first input feature image is the first feature image, or the second feature image output by the previous depth multi-scale feature extraction module; The downsampling unit performs a first type of downsampling on the first input feature image to obtain a first downsampled feature image, and the downsampling unit performs a second type of downsampling on the first input feature image to obtain a second downsampled feature image, wherein the first type of downsampling is used to enhance the key features of the first input feature image; and the second type of downsampling is used to enhance the multi-channel sampling features of the first input feature image. The first downsampled feature image and the second downsampled feature image are fused to obtain the target downsampled feature image; The target downsampled feature image is used as the second input feature map of the multi-scale feature extraction unit. Multi-scale feature extraction is performed using the multi-scale feature extraction unit to obtain the second feature image output by the depth multi-scale feature extraction module.

3. The method according to claim 2, characterized in that, The downsampling unit includes a first type of downsampling unit and a second type of downsampling unit; the step of performing first-type downsampling on the first input feature image through the downsampling unit to obtain a first downsampled feature image, and performing second-type downsampling on the first input feature image through the downsampling unit to obtain a second downsampled feature image, includes: The most critical features of the first input feature image are downsampled using the first type of downsampling unit to obtain the downsampled feature extraction image. The first type of downsampling unit is used to enhance key features in the downsampled feature extraction image to obtain a first downsampled feature image. The first input feature image is enhanced by the second type of downsampling unit to obtain a downsampled feature-enhanced image. The second type of downsampling unit performs multi-channel downsampling on the downsampling feature enhancement image to obtain a second downsampling feature image.

4. The method according to claim 1 or 2, characterized in that, The multi-scale feature extraction unit includes a channel number expansion subunit, a channel dimension slicing subunit, a splicing subunit, at least two lightweight feature enhancement subunits, and a multi-scale attention extraction subunit; multi-scale feature extraction is performed using the multi-scale feature extraction unit, including: The third input feature image of the multi-scale feature extraction unit is obtained, and the channel number expansion subunit is used to expand the channel number of the third input feature image to obtain a channel number expanded image, wherein the third input feature image is an optical image or a target downsampled feature image; By using the channel-dimensional segmentation unit, the channel-number expanded image is segmented along the channel dimension to obtain a first segmented image and a second segmented image with the same number of channels; The first segmented image is enhanced by passing at least two lightweight feature enhancement subunits connected in sequence to obtain lightweight feature enhancement images output by each lightweight feature enhancement subunit. The lightweight feature enhancement images are then stitched together by the stitching subunit to obtain a first stitched image. The first stitched image is enhanced with multi-scale attention features by the multi-scale attention extraction subunit to obtain a first multi-scale feature enhanced image, and the second segmented image is enhanced with multi-scale attention by the multi-scale attention extraction subunit to obtain a second multi-scale feature enhanced image. The first multi-scale feature enhancement image and the second multi-scale feature enhancement image are stitched together using the stitching sub-unit.

5. The method according to claim 1, characterized in that, The step of classifying each radar slice image to obtain the slice classification result corresponding to each radar slice image includes: For any radar segmented image, multi-channel downsampling is performed on the radar segmented image to obtain a radar segmented downsampled feature image; Spatial feature enhancement is performed on the radar segmented downsampled feature image to obtain a radar segmented enhanced feature image; Based on the radar segmentation enhanced feature image, determine the target enhanced feature image corresponding to the radar segmentation image; Based on the target enhancement feature image corresponding to each radar slice image, each radar slice image is classified to obtain the slice classification result corresponding to each radar slice image.

6. The method according to claim 5, characterized in that, The step of determining the target enhancement feature image corresponding to the radar segmentation image based on the radar segmentation enhancement feature image includes: Determine the image size relationship between the radar segmentation enhancement feature image and the radar segmentation image; If the image size relationship indicates that the radar segmentation enhancement feature image and the radar segmentation image have the same image size, then the radar segmentation enhancement feature image and the radar segmentation image are added together to obtain the target enhancement feature image corresponding to the radar segmentation image; If the image size relationship indicates that the radar segmentation enhancement feature image and the radar segmentation image have different image sizes, then the radar segmentation enhancement feature image is determined as the target enhancement feature image corresponding to the radar segmentation image.

7. The method according to claim 1, characterized in that, The step of slicing the synthetic aperture radar image based on the position information of the plurality of target detection boxes to obtain a radar slice image corresponding to each target detection box includes: For the position information of any of the target detection boxes, the position information of the target detection boxes is transformed by coordinates to obtain the transformed position information in the coordinate system corresponding to the synthetic aperture radar image; Based on the transformation position information of each target detection box, each target detection box is mapped to the synthetic aperture radar image to obtain a mapped image; Based on the mapped image, the mapped region in the synthetic aperture radar image corresponding to each target detection box is sliced ​​to obtain a radar slice image corresponding to each target detection box.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images