Power transmission channel hidden danger detection method, computer equipment, readable storage medium and program product

Abstract

The invention relates to a power transmission channel hidden danger detection method, computer equipment, a readable storage medium and a program product, and is applied to the technical field of big data, and the method comprises the steps: obtaining an optical image and a synthetic aperture radar image; performing multi-scale feature extraction on the optical image to obtain a first feature image; enabling the first feature image to pass through a plurality of depth multi-scale feature extraction modules which are connected in sequence to obtain second feature images; fusing the second feature images to obtain a third feature image, and performing channel hidden danger detection on the third feature image to obtain a plurality of target detection frames; according to the plurality of target detection frames, slicing the synthetic aperture radar image to obtain a radar slice image corresponding to each target detection frame; classifying each radar slice image to obtain a classification result of each slice; and determining a power transmission channel hidden danger detection result according to each slice classification result. The method can improve the accuracy of power transmission channel hidden danger detection.

Description

Technical Field

[0001] The present application relates to the field of big data technology, and in particular to a method for detecting hidden dangers in power transmission channels, a computer device, a computer-readable storage medium, and a computer program product. Background Art

[0002] The transmission channel is the objective carrier of the power transmission network, and its safe and stable operation is crucial to ensuring power supply and maintaining the normal operation of social production and life. Therefore, a method for detecting hidden dangers in the transmission channel is urgently needed.

[0003] At present, the hidden danger detection of power transmission channels relies on manual inspection. However, this method is labor-intensive and inefficient. In addition, the power transmission lines are widely distributed and the environment is complex. It is difficult for inspectors to reach many areas, resulting in certain limitations and blind spots in the manual inspection method. In addition, for some minor and hidden hidden dangers, the accuracy of manual inspection depends on the experience and skill level of the inspectors. As a result, the accuracy of hidden danger detection in power transmission channels is low. Summary of the invention

[0004] Based on this, it is necessary to provide a method, device, computer equipment, computer-readable storage medium and computer program product for detecting hidden dangers in power transmission channels, which can improve the accuracy of hidden danger detection in power transmission channels, in order to address the above technical problems.

[0005] In a first aspect, the present application provides a method for detecting hidden dangers in a power transmission channel, comprising:

[0006] Acquire optical images and synthetic aperture radar images of the power transmission channel to be inspected;

[0007] Using a multi-scale feature extraction unit, performing multi-scale feature extraction on the optical image to obtain a first feature image;

[0008] Passing the first feature image through at least two sequentially connected deep multi-scale feature extraction modules to obtain a second feature image output by each of the deep multi-scale feature extraction modules, wherein the deep multi-scale feature extraction module includes a downsampling unit and a multi-scale feature extraction unit;

[0009] The second feature images are fused to obtain a third feature image, and channel hidden danger detection is performed on the third feature image to obtain position information of multiple target detection frames;

[0010] Slice the synthetic aperture radar image according to the position information of the multiple target detection frames to obtain a radar slice image corresponding to each target detection frame;

[0011] Classifying each of the radar slice images respectively to obtain a slice classification result corresponding to each of the radar slice images;

[0012] According to the classification results of each of the slices, a power transmission channel hidden danger detection result corresponding to the power transmission channel to be detected is determined.

[0013] In a second aspect, the present application also provides a power transmission channel hidden danger detection device, comprising:

[0014] An acquisition module, used for acquiring an optical image and a synthetic aperture radar image of the power transmission channel to be detected;

[0015] An extraction module, configured to perform multi-scale feature extraction on the optical image using a multi-scale feature extraction unit to obtain a first feature image; pass the first feature image through at least two sequentially connected deep multi-scale feature extraction modules to obtain a second feature image output by each of the deep multi-scale feature extraction modules, wherein the deep multi-scale feature extraction module includes a downsampling unit and a multi-scale feature extraction unit; and fuse each of the second feature images to obtain a third feature image;

[0016] A detection module, used to perform channel hidden danger detection on the third feature image to obtain position information of multiple target detection frames;

[0017] A slicing module, configured to perform slicing processing on the synthetic aperture radar image according to the position information of the multiple target detection frames, so as to obtain a radar slice image corresponding to each target detection frame;

[0018] A classification module, used to classify each of the radar slice images respectively to obtain a slice classification result corresponding to each of the radar slice images;

[0019] The detection module is also used to determine the power transmission channel hidden danger detection result corresponding to the power transmission channel to be detected according to the classification results of each slice.

[0020] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0021] Acquire optical images and synthetic aperture radar images of the power transmission channel to be inspected;

[0022] Using a multi-scale feature extraction unit, performing multi-scale feature extraction on the optical image to obtain a first feature image;

[0023] Passing the first feature image through at least two sequentially connected deep multi-scale feature extraction modules to obtain a second feature image output by each of the deep multi-scale feature extraction modules, wherein the deep multi-scale feature extraction module includes a downsampling unit and a multi-scale feature extraction unit;

[0024] The second feature images are fused to obtain a third feature image, and channel hidden danger detection is performed on the third feature image to obtain position information of multiple target detection frames;

[0025] Slice the synthetic aperture radar image according to the position information of the multiple target detection frames to obtain a radar slice image corresponding to each target detection frame;

[0026] Classifying each of the radar slice images respectively to obtain a slice classification result corresponding to each of the radar slice images;

[0027] According to the classification results of each of the slices, a power transmission channel hidden danger detection result corresponding to the power transmission channel to be detected is determined.

[0028] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0029] Acquire optical images and synthetic aperture radar images of the power transmission channel to be inspected;

[0030] Using a multi-scale feature extraction unit, performing multi-scale feature extraction on the optical image to obtain a first feature image;

[0031] Passing the first feature image through at least two sequentially connected deep multi-scale feature extraction modules to obtain a second feature image output by each of the deep multi-scale feature extraction modules, wherein the deep multi-scale feature extraction module includes a downsampling unit and a multi-scale feature extraction unit;

[0032] The second feature images are fused to obtain a third feature image, and channel hidden danger detection is performed on the third feature image to obtain position information of multiple target detection frames;

[0033] Slice the synthetic aperture radar image according to the position information of the multiple target detection frames to obtain a radar slice image corresponding to each target detection frame;

[0034] Classifying each of the radar slice images respectively to obtain a slice classification result corresponding to each of the radar slice images;

[0035] According to the classification results of each of the slices, a power transmission channel hidden danger detection result corresponding to the power transmission channel to be detected is determined.

[0036] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0037] Acquire optical images and synthetic aperture radar images of the power transmission channel to be inspected;

[0038] Using a multi-scale feature extraction unit, performing multi-scale feature extraction on the optical image to obtain a first feature image;

[0039] Passing the first feature image through at least two sequentially connected deep multi-scale feature extraction modules to obtain a second feature image output by each of the deep multi-scale feature extraction modules, wherein the deep multi-scale feature extraction module includes a downsampling unit and a multi-scale feature extraction unit;

[0040] The second feature images are fused to obtain a third feature image, and channel hidden danger detection is performed on the third feature image to obtain position information of multiple target detection frames;

[0041] Slice the synthetic aperture radar image according to the position information of the multiple target detection frames to obtain a radar slice image corresponding to each target detection frame;

[0042] Classifying each of the radar slice images respectively to obtain a slice classification result corresponding to each of the radar slice images;

[0043] According to the classification results of each of the slices, a power transmission channel hidden danger detection result corresponding to the power transmission channel to be detected is determined.

[0044] The above-mentioned power transmission channel hidden danger detection method, device, computer equipment, computer-readable storage medium and computer program product obtain an optical image and a synthetic aperture radar image of the power transmission channel to be detected; use a multi-scale feature extraction unit to perform multi-scale feature extraction on the optical image to obtain a first feature image; pass the first feature image through at least two deep multi-scale feature extraction modules connected in sequence to obtain a second feature image output by each of the deep multi-scale feature extraction modules, wherein the deep multi-scale feature extraction module includes a downsampling unit and a multi-scale feature extraction unit; fuse each of the second feature images to obtain a third feature image, and perform channel hidden danger detection on the third feature image to obtain position information of multiple target detection frames; slice the synthetic aperture radar image according to the position information of the multiple target detection frames to obtain a radar slice image corresponding to each of the target detection frames; classify each of the radar slice images respectively to obtain a slice classification result corresponding to each of the radar slice images; and determine the transmission channel hidden danger detection result corresponding to the power transmission channel to be detected according to each of the slice classification results.

[0045] In this way, by taking the synthetic aperture radar image and the optical image as the source of image processing, the extracted features are not restricted by the environment of the power transmission channel to be detected. Then, the optical image is subjected to multi-scale feature extraction by a multi-scale feature extraction unit to obtain a first feature image, thereby ensuring that the input features can represent multi-scale feature information in the input data dimension, and the first feature image is passed through at least two deep multi-scale feature extraction modules connected in sequence to obtain a second feature image output by each deep multi-scale feature extraction module, thereby ensuring that the feature extraction process is multi-scale extraction in the feature extraction dimension, and the second feature images are fused to obtain a third feature image, and the third feature image is subjected to channel hidden danger detection to obtain the position information of multiple target detection frames, that is, the multiple second feature images obtained by the multi-scale feature extraction are fused to obtain a further multi-scale feature extraction. The third characteristic image under the condition is obtained, and the third characteristic image is used as the basis for channel hidden danger detection. Therefore, the target detection accuracy is improved. Then, based on the position information of the target detection frame, the synthetic aperture radar image is sliced ​​to obtain multiple radar slice images, so that the radar slice image can, to a certain extent, characterize the preliminary results obtained under the optical image processing process; thereby, each radar slice image is classified respectively to obtain the slice classification results corresponding to each radar slice image, and finally, according to each slice classification result, the transmission channel hidden danger detection result corresponding to the transmission channel to be detected is determined. It is known from the above analysis that the radar slice image is an image obtained after layer-by-layer feature extraction and processing. Therefore, the slice classification result obtained by classification can be made more accurate, and then the transmission channel hidden danger detection result determined is more in line with the actual situation of the transmission channel to be detected, so the accuracy of transmission channel hidden danger detection is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0047] Figure 1 A diagram showing an application environment of a method for detecting hidden dangers in a power transmission channel in an embodiment;

[0048] Figure 2 It is a schematic diagram of a flow chart of a method for detecting hidden dangers in a power transmission channel in one embodiment;

[0049] Figure 3 A schematic diagram of a process of performing channel hidden danger detection on an optical image in one embodiment;

[0050] Figure 4 is a schematic flow chart of steps for processing an optical image in one embodiment;

[0051] Figure 5 It is a schematic diagram of a process of processing a third input feature image to obtain a target down-sampled feature image in one embodiment;

[0052] Figure 6 It is a schematic diagram of a process of processing a first input feature image to obtain a target down-sampled feature image in one embodiment;

[0053] Figure 7 A schematic diagram of a flow chart of the steps of classifying radar slice images in one embodiment;

[0054] Figure 8 A schematic diagram of a process of processing a radar slice image to obtain a target enhanced feature image in one embodiment;

[0055] Fig. 9 is a structural block diagram of a power transmission channel hidden danger detection device in one embodiment;

[0056] Fig.10 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0058] It should be noted that the information and data involved in this application (including but not limited to data for analysis, stored data, displayed data, etc.) are all information and data authorized by the user or fully authorized by all parties, and the acquisition, transmission, storage, use and processing of relevant data are in compliance with the relevant provisions of national laws and regulations. The content pushed to users (for example, radar slice images, optical images, synthetic aperture radar images, slice classification results, and transmission channel hidden danger detection results, etc.) can be rejected by the user or can be easily rejected by the user. In the embodiments of the present application, certain software, components, models and other existing solutions in the industry may be mentioned, and they should be regarded as exemplary. Their purpose is only to illustrate the feasibility of the implementation of the technical solution of this application, but it does not mean that the applicant has or will necessarily use the solution.

[0059] First of all, it is understandable that in order to improve the accuracy of hidden danger detection in power transmission channels, researchers use deep learning algorithms to continuously improve target detection models to improve the efficiency of hidden danger detection in power transmission channels. Specifically, through deep learning algorithms, hidden danger detection is performed on the optical image of the power transmission channel to be detected to obtain hidden danger detection information. However, the environment in which the transmission channel is located is not uniform and fixed, that is, the environment in which the transmission channel is located may be different in different locations and / or at different time points. For environments with other obvious atmospheric influences such as clouds, rain, and snow, there may be situations where the optical image cannot accurately characterize the characteristics of the transmission channel, resulting in the accuracy of hidden danger detection in the transmission channel is still low. Therefore, there is an urgent need for a method for detecting hidden dangers in a transmission channel that improves the accuracy of hidden danger detection in the transmission channel.

[0060] The power transmission channel hidden danger detection method provided in the embodiment of the present application can be applied to Figure 1In the application environment shown. Among them, the image acquisition component 102 communicates with the server 104 through the network, and the image acquisition component 102 includes an optical acquisition component and a synthetic aperture radar acquisition component. The optical acquisition component is used to acquire an optical image of the power transmission channel to be detected, and the synthetic aperture radar acquisition component is used to acquire a synthetic aperture radar image of the power transmission channel to be detected. The terminal 106 communicates with the server 104 through the network. The data storage system can store data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. The optical image and synthetic aperture radar image of the power transmission channel to be detected are obtained through the server 104; the optical image is subjected to multi-scale feature extraction by a multi-scale feature extraction unit to obtain a first feature image; the first feature image is passed through at least two deep multi-scale feature extraction modules connected in sequence to obtain a second feature image output by each deep multi-scale feature extraction module, wherein the deep multi-scale feature extraction module includes a downsampling unit and a multi-scale feature extraction unit; the second feature images are fused to obtain a third feature image, and the third feature image is subjected to channel hidden danger detection to obtain the position information of multiple target detection frames; according to the position information of multiple target detection frames, the synthetic aperture radar image is sliced ​​to obtain a radar slice image corresponding to each target detection frame; each radar slice image is classified to obtain a slice classification result corresponding to each radar slice image; according to each slice classification result, the transmission channel hidden danger detection result corresponding to the power transmission channel to be detected is determined. The server 104 can push at least one of the first feature image, the second feature image, the third feature image, the radar slice image, the optical image, the synthetic aperture radar image, the slice classification result and the power transmission channel hidden danger detection result to the terminal 106. Among them, the terminal 106 can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car devices, projection devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. Head-mounted devices can be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc. The server 104 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services.

[0061] In an exemplary embodiment, Figure 2 As shown, a method for detecting hidden dangers in a power transmission channel is provided. Figure 1 The server 104 in the example is used as an example, and the description is made in the form of omitting the main body, including the following steps 202 to 214. Among them:

[0062] Step 202: Acquire an optical image and a synthetic aperture radar image of the power transmission channel to be detected.

[0063] As one embodiment, obtaining an optical image of the power transmission channel to be detected includes: obtaining an optical image of the power transmission channel to be detected collected by an optical collection component deployed in the area where the power transmission channel to be detected is located. As one embodiment, obtaining a synthetic aperture radar image of the power transmission channel to be detected includes: obtaining a synthetic aperture radar image of the power transmission channel to be detected collected by a synthetic aperture radar collection component deployed in the area where the power transmission channel to be detected is located.

[0064] In this way, for areas with complex environments that are difficult for ordinary people to reach, image collection can also be achieved by deploying synthetic aperture radar collection components or optical collection components.

[0065] As another embodiment, obtaining an optical image of the power transmission channel to be detected includes: obtaining an optical image of the power transmission channel to be detected collected by an optical collection component deployed on a mobile component. As another embodiment, obtaining a synthetic aperture radar image of the power transmission channel to be detected includes: obtaining a synthetic aperture radar image of the power transmission channel to be detected collected by a synthetic aperture radar collection component deployed on a mobile component.

[0066] In this way, the purchase cost of synthetic aperture radar acquisition components and optical acquisition components can be saved, and image acquisition of multiple power transmission channels to be detected can be achieved by deploying a single or multiple aperture radar acquisition components and optical acquisition components on the mobile component, or image acquisition at multiple angles of a single power transmission channel to be detected can be achieved.

[0067] It is understandable that SAR (Synthetic Aperture Radar) is an active earth observation system. Synthetic aperture radar images are formed by data collected by coherent radar. The radar emits radio frequency energy pulses to the ground and measures the strength of the reflected signal. The distance between the radar and the target object is obtained by the round-trip time of the pulse. Therefore, synthetic aperture radar imaging is not limited by light, and radio frequency radiation is not significantly affected by other atmospheric conditions such as clouds, rain, and snow. Image data can be continuously obtained under any weather conditions, and it is not restricted by factors such as weather, clouds, day and night light. It has the characteristics of all-day and all-weather imaging; synthetic aperture radar has a strong penetration ability for vegetation and soil, which provides the possibility for inverting surface vegetation and soil characteristics; the characteristics of not relying on flight altitude enable it to be installed on some flying platforms such as aircraft and satellites, and the optional satellite orbits are more abundant; it can collect information over a large range at a lower resolution, or collect detailed high-resolution images in a smaller area.

[0068] Therefore, combining optical images with synthetic aperture radar images can free the subsequent transmission channel detection process from the influence of environmental factors and improve the accuracy of transmission channel hidden danger detection.

[0069] Step 204: Use a multi-scale feature extraction unit to perform multi-scale feature extraction on the optical image to obtain a first feature image.

[0070] Exemplarily, step 204 includes: using a first preliminary feature extraction module to perform preliminary feature extraction on the optical image to obtain an optical feature image, and using a multi-scale feature extraction unit to perform multi-scale feature extraction on the optical feature image to obtain a first feature image.

[0071] As an embodiment, the first preliminary feature extraction module includes at least two key feature enhancement subunits connected in sequence; using the first preliminary feature extraction module, performing preliminary feature extraction on the optical image to obtain an optical feature image includes: passing the optical image through at least two key feature enhancement subunits connected in sequence to obtain an optical feature image. Each key feature enhancement subunit includes a key feature enhancement convolution layer, a normalization layer, and an activation function layer connected in sequence, the convolution kernel size of the key feature enhancement convolution layer of each key feature enhancement subunit is the same, and the step size of the key feature enhancement convolution layer of each key feature enhancement subunit is different from the step size of the key feature enhancement convolution layer of the previous key feature enhancement subunit and the key feature enhancement convolution layer of the next key feature enhancement subunit.

[0072] As an embodiment, the number of key feature enhancer units included in the first preliminary feature extraction module is a multiple of 2. When the number of key feature enhancer units included in the first preliminary feature extraction module is 4, the first key feature enhancer unit and the third key feature enhancer unit in the first preliminary feature extraction module include a key feature enhancement convolution layer with a convolution kernel size of 3×3 and a step size of 1, and the second key feature enhancer unit and the fourth key feature enhancer unit in the first preliminary feature extraction module include a key feature enhancement convolution layer with a convolution kernel size of 3×3 and a step size of 2.

[0073] Optionally, a multi-scale feature extraction unit is used to perform multi-scale feature extraction on the optical feature image. The specific implementation steps for obtaining the first feature image can refer to the specific implementation content of the multi-scale feature extraction step performed by the multi-scale feature extraction unit in the following step 308, which will not be repeated here.

[0074] Step 206: Pass the first feature image through at least two deep multi-scale feature extraction modules connected in sequence to obtain a second feature image output by each deep multi-scale feature extraction module, wherein the deep multi-scale feature extraction module includes a downsampling unit and a multi-scale feature extraction unit.

[0075] Exemplarily, step 206 includes: for the first deep multi-scale feature extraction module, performing deep multi-scale feature extraction on the first feature image, outputting the second feature image output by the first deep multi-scale feature extraction module, and for each non-first deep multi-scale feature extraction module, performing deep multi-scale feature extraction on the second feature image output by the previous deep multi-scale feature extraction module, and outputting the second feature image output by the non-first deep multi-scale feature extraction module.

[0076] Step 208: fuse the second feature images to obtain a third feature image, and perform channel hidden danger detection on the third feature image to obtain position information of multiple target detection frames.

[0077] Exemplarily, the second feature images are fused to obtain a third feature image, including: fusing the second feature images through a preset fusion method to obtain the third feature image, wherein the preset fusion method includes but is not limited to a splicing method, an addition method, a multiplication method, an attention mechanism method, a deconvolution method, and a pyramid pooling method.

[0078] Furthermore, the second feature images are fused in a preset fusion manner to obtain a third feature image, including: fusing the second feature images in a preset fusion manner through a neck network to obtain a third feature image.

[0079] Among them, the above-mentioned multi-scale feature extraction unit and at least two deep multi-scale feature extraction modules can form a backbone network.

[0080] Exemplarily, performing channel hidden danger detection on the third characteristic image to obtain position information of multiple target detection frames includes: detecting hidden dangers of the power transmission channel in the third characteristic image to obtain position information of multiple target detection frames. Among them, hidden dangers of the power transmission channel include but are not limited to hidden dangers of floating objects, hidden dangers of buildings, hidden dangers of trees, hidden dangers of construction machinery, etc.

[0081] Furthermore, the hidden dangers of the power transmission channel in the third characteristic image are detected to obtain the position information of multiple target detection frames, including: detecting the hidden dangers of the power transmission channel in the third characteristic image by a detection head to obtain the position information of multiple target detection frames.

[0082] As an example, refer to Figure 3, multi-scale feature extraction is performed on the optical image through a multi-scale feature extraction unit to obtain a first feature image; the first feature image is passed through at least two deep multi-scale feature extraction modules connected in sequence to obtain a second feature image output by each deep multi-scale feature extraction module; the second feature images are fused through a neck network to obtain a third feature image, and channel hidden danger detection is performed on the third feature image through a detection head to obtain position information of multiple target detection frames.

[0083] Step 210: Slice the synthetic aperture radar image according to the position information of the multiple target detection frames to obtain a radar slice image corresponding to each target detection frame.

[0084] Exemplarily, step 210 includes: for the position information of any target detection frame, performing coordinate transformation on the position information of the target detection frame to obtain the transformed position information in the coordinate system corresponding to the synthetic aperture radar image; mapping each target detection frame to the synthetic aperture radar image according to the transformed position information of each target detection frame to obtain a mapping image; and slicing the mapping area corresponding to each target detection frame in the synthetic aperture radar image according to the mapping image to obtain a radar slice image corresponding to each target detection frame.

[0085] Furthermore, the position information of the target detection frame is subjected to coordinate transformation to obtain the transformed position information in the coordinate system corresponding to the synthetic aperture radar image, including: transforming the position information of the target detection frame from optical image coordinates to projection coordinates to obtain target projection coordinates; transforming the target projection coordinates to synthetic aperture radar coordinates to obtain the transformed position information in the coordinate system corresponding to the synthetic aperture radar image.

[0086] As an embodiment, the position information of the target detection frame is converted from optical image coordinates to projection coordinates to obtain the target projection coordinates, including: acquiring image parameters of the optical image, and the projection coordinates of a preset position in the optical image, the pixel row coordinates and the pixel column coordinates of each pixel point of the target detection frame in the optical image, according to the image parameters of the optical image, the projection coordinates of the preset position in the optical image, the pixel row coordinates and the pixel column coordinates of each pixel point of the target detection frame in the optical image, converting each pixel point of the target detection frame from the optical image coordinates to the projection coordinates to obtain the pixel projection coordinates, and fusing the pixel projection coordinates of each pixel point of the target detection frame to obtain the target projection coordinates in the projection coordinate system, wherein the preset position in the image (optical image and SAR image) involved in this article is any position in the image, which can be but is not limited to the vertex position, edge position and center position.

[0087] Furthermore, the image parameters of the optical image include an optical image row rotation parameter, an optical image column rotation parameter, an optical image horizontal pixel width, and an optical image vertical pixel width; the projection coordinates of a preset position in the optical image include an optical projection horizontal coordinate and an optical projection vertical coordinate; the pixel point projection coordinates include a first projection coordinate of the pixel point in the horizontal direction, and a second projection coordinate of the pixel point in the vertical direction; according to the image parameters of the optical image, the projection coordinates of the preset position in the optical image, and the pixel point row coordinates and pixel point column coordinates of each pixel point of the target detection frame in the optical image, each pixel point of the target detection frame is converted from the optical image coordinates to Projection coordinates, obtaining pixel projection coordinates, including: determining the first projection coordinates of the pixel in the horizontal direction according to the optical projection abscissa, the pixel column coordinates of each pixel of the target detection frame in the optical image, the optical image row rotation parameter, the pixel row coordinates of each pixel of the target detection frame in the optical image, and the horizontal pixel width of the optical image; and determining the second projection coordinates of the pixel in the vertical direction according to the optical projection ordinate, the pixel column coordinates of each pixel of the target detection frame in the optical image, the optical image column rotation parameter, the pixel row coordinates of each pixel of the target detection frame in the optical image, and the vertical pixel width of the optical image.

[0088] Optionally, according to the optical projection abscissa, the pixel column coordinates of each pixel of the target detection frame in the optical image, the optical image row rotation parameter, the pixel row coordinates of each pixel of the target detection frame in the optical image and the horizontal pixel width of the optical image, the first projection coordinate of the pixel in the horizontal direction can be determined by the formula:

[0089]

[0090] in, is the first projection coordinate of the pixel point in the horizontal direction, is the optical projection abscissa, Column coordinates of each pixel of the target detection box in the optical image, is the optical image row rotation parameter, is the pixel row coordinate of each pixel of the target detection box in the optical image, is the horizontal pixel width of the optical image.

[0091] Optionally, according to the optical projection ordinate, the pixel column coordinates of each pixel of the target detection frame in the optical image, the optical image column rotation parameter, the pixel row coordinates of each pixel of the target detection frame in the optical image and the vertical pixel width of the optical image, the second projection coordinate of the pixel in the vertical direction can be determined by the formula:

[0092]

[0093] in, is the second projection coordinate of the pixel point in the vertical direction, is the optical projection ordinate, is the optical image column rotation parameter, is the vertical pixel width of the optical image.

[0094] As an embodiment, the target projection coordinates are converted into synthetic aperture radar coordinates to obtain conversion position information in a coordinate system corresponding to the synthetic aperture radar image, including: obtaining a linear conversion equation, a horizontal pixel width of the SAR image, a vertical pixel width of the SAR image, a row rotation parameter of the SAR image, a column rotation parameter of the SAR image, a column coordinate of a pixel point in the SAR image, a row coordinate of a pixel point in the SAR image, and a projection abscissa and a projection ordinate corresponding to a preset position in the SAR synthetic aperture radar image, wherein the projection abscissa and the projection ordinate corresponding to the preset position in the SAR synthetic aperture radar image can refer to the determination method of the first projection coordinate and the second projection coordinate. Without further elaboration, according to the horizontal pixel width of the SAR image, the vertical pixel width of the SAR image, the row rotation parameter of the SAR image, the column rotation parameter of the SAR image, the column coordinates of the pixel points in the SAR image, the row coordinates of the pixel points in the SAR image, and the projection abscissa and ordinate corresponding to the preset position in the SAR synthetic aperture radar image, the target projection coordinates are converted into synthetic aperture radar coordinates to obtain the SAR coordinates of the pixel points in the horizontal direction and the SAR coordinates of the pixel points in the vertical direction; the SAR coordinates of each pixel point in the horizontal direction and the SAR coordinates in the vertical direction are fused to obtain a labeled conversion image in the coordinate system corresponding to the synthetic aperture radar image.

[0095] Optionally, the linear transformation equation may specifically include:

[0096]

[0097] in, is the SAR image column rotation parameter, is the horizontal pixel width of the SAR image, is the SAR image column rotation parameter, is the vertical pixel width of the SAR image, is the pixel coordinates in the SAR image, is the row coordinate of the pixel point in the SAR image, is the SAR coordinate of the pixel point in the horizontal direction, is the SAR coordinate of the pixel in the vertical direction, is the projection horizontal coordinate corresponding to the preset position in the SAR synthetic aperture radar image, It is the projection ordinate corresponding to the preset position in the SAR synthetic aperture radar image.

[0098] The mapping image is used to represent the position of the marked area in the synthetic aperture radar image.

[0099] As an embodiment, according to the mapping image, the mapping areas corresponding to each target detection frame in the synthetic aperture radar image are sliced ​​respectively to obtain the radar slice image corresponding to each target detection frame, including: according to the mapping image, respectively identifying the mapping areas corresponding to each target detection frame in the synthetic aperture radar image; according to the mapping areas corresponding to each target detection frame in the synthetic aperture radar image, determining the segmented areas corresponding to each target detection frame, wherein the area size of the segmented area corresponding to each target detection frame is greater than or equal to the area size of the mapping area corresponding to the target detection frame, the segmented area corresponding to each target detection frame includes the mapping area corresponding to the target detection frame, and the segmented area corresponding to each target detection frame may also include a peripheral area of ​​the mapping area corresponding to the target detection frame.

[0100] Step 212: classify each radar slice image respectively to obtain a slice classification result corresponding to each radar slice image.

[0101] The slice classification result is used to characterize at least one of the background image type, floating object hidden danger image type, building hidden danger image type, tree hidden danger image type and construction machinery hidden danger image type.

[0102] Exemplarily, step 212 includes: for any radar slice image, performing feature enhancement on the radar slice image to obtain a target enhanced feature image, and classifying the radar slice image according to the target enhanced feature image to obtain a slice classification result corresponding to the radar slice image.

[0103] Among them, the process of classifying the radar slice image according to the target enhanced feature image and obtaining the slice classification result corresponding to the radar slice image can be executed by the slice classification model, and the training process of the slice classification model includes: obtaining multiple training samples and the slice classification model to be trained, each training sample includes the training target enhanced feature image corresponding to the training transmission channel and the real slice type label corresponding to the training transmission channel; mapping step: mapping the training target enhanced feature image corresponding to the training transmission channel to the training slice type information through the slice classification model to be trained; calculating the loss between the training slice type information and the real slice type label corresponding to the training transmission channel, and when the loss tends to converge, determining the current slice classification model to be trained as the slice classification model, and when the loss does not tend to converge, updating the model of the slice classification model to be trained, wherein the model update method includes but is not limited to the gradient ascent method and the gradient descent method, and returning to the mapping step until the loss tends to converge.

[0104] Step 214, determining the power transmission channel hidden danger detection result corresponding to the power transmission channel to be detected according to the classification results of each slice.

[0105] Among them, the transmission channel hidden danger detection result in step 214 is used to characterize at least one of the transmission channel state and the hidden danger type. The transmission channel state is used to characterize the presence or absence of hidden dangers in the transmission channel. When the transmission channel state characterizes the presence of hidden dangers in the transmission channel, the transmission channel hidden danger detection result can also characterize the hidden danger type, and the hidden danger type includes at least one of the floating object hidden danger type, the building hidden danger type, the tree hidden danger type and the construction machinery hidden danger type.

[0106] Exemplarily, step 214 includes: determining a power transmission channel hidden danger detection result corresponding to the power transmission channel to be detected according to the slice type represented by each slice classification result.

[0107] Furthermore, according to the slice types represented by the slice classification results, the transmission channel hidden danger detection results corresponding to the transmission channel to be detected are determined, including: if the slice types represented by the slice classification results have floating object hidden danger image types, the floating object hidden danger type is determined as the transmission channel hidden danger detection results corresponding to the transmission channel to be detected; if the slice types represented by the slice classification results have building hidden danger image types, the building hidden danger type is determined as the transmission channel hidden danger detection results corresponding to the transmission channel to be detected; if the slice types represented by the slice classification results have tree hidden danger image types, the tree hidden danger type is determined as the transmission channel hidden danger detection results corresponding to the transmission channel to be detected; if the slice types represented by the slice classification results have construction machinery hidden danger image types, the construction machinery hidden danger type is determined as the transmission channel hidden danger detection results corresponding to the transmission channel to be detected; if the slice types represented by the slice classification results have only background image types, the absence of hidden danger detection results is determined as the transmission channel hidden danger detection results corresponding to the transmission channel to be detected.

[0108] In the above-mentioned transmission channel hidden danger detection method, by using the synthetic aperture radar image and the optical image as the source of image processing, the extracted features are not restricted by the environment of the transmission channel to be detected, and then the optical image is subjected to multi-scale feature extraction by a multi-scale feature extraction unit to obtain a first feature image, thereby ensuring that the input features can represent multi-scale feature information in the input data dimension, and the first feature image is passed through at least two deep multi-scale feature extraction modules connected in sequence to obtain a second feature image output by each deep multi-scale feature extraction module, thereby ensuring that the feature extraction process is multi-scale extraction in the feature extraction dimension, and the second feature images are fused to obtain a third feature image, and the third feature image is subjected to channel hidden danger detection to obtain the position information of multiple target detection frames, that is, the multiple second feature images obtained by the multi-scale feature extraction are fused to obtain a further multi-scale feature image. The third feature image under the degree feature extraction is obtained, and the third feature image is used as the basis for channel hidden danger detection, so the target detection accuracy is improved, and then based on the position information of the target detection frame, the synthetic aperture radar image is sliced ​​to obtain multiple radar slice images, so that the radar slice image can, to a certain extent, characterize the preliminary results obtained under the optical image processing process; thereby, each radar slice image is classified respectively to obtain the slice classification results corresponding to each radar slice image, and finally, according to each slice classification result, the transmission channel hidden danger detection result corresponding to the transmission channel to be detected is determined. It is known from the above analysis that the radar slice image is an image obtained after layer-by-layer feature extraction and processing, so the slice classification result obtained by classification can be made more accurate, and then the transmission channel hidden danger detection result determined is more in line with the actual situation of the transmission channel to be detected, so the accuracy of transmission channel hidden danger detection is improved.

[0109] In an exemplary embodiment, Figure 4 As shown, a method for accurately processing an optical image is provided, and the method further includes steps 302 to 308. Among them:

[0110] Step 302: Obtain a first input feature image of a downsampling unit, wherein the first input feature image is a first feature image, or a second feature image output by a previous deep multi-scale feature extraction module.

[0111] Step 304: Perform a first type of downsampling on the first input feature image through a downsampling unit to obtain a first downsampled feature image, and perform a second type of downsampling on the first input feature image through a downsampling unit to obtain a second downsampled feature image, wherein the first type of downsampling is used to enhance the key features of the first input feature image; and the second type of downsampling is used to enhance the multi-channel sampling features of the first input feature image.

[0112] Exemplarily, the downsampling unit includes a first type of downsampling unit and a second type of downsampling unit; step 304 includes: performing most critical feature downsampling on the first input feature image through the first type of downsampling unit to obtain a downsampled feature extraction image; performing key feature enhancement on the downsampled feature extraction image through the first type of downsampling unit to obtain a first downsampled feature image; performing key feature enhancement on the first input feature image through the second type of downsampling unit to obtain a downsampled feature enhanced image; performing multi-channel downsampling on the downsampled feature enhanced image through the second type of downsampling unit to obtain a second downsampled feature image.

[0113] Wherein, by using the first type of downsampling unit, the first input feature image is subjected to the most critical feature downsampling, and the process of obtaining the downsampled feature extraction image can be performed by the maximum pooling layer in the first type of downsampling unit. Wherein, by using the first type of downsampling unit, the downsampled feature extraction image is subjected to the key feature enhancement, and the first downsampled feature image is obtained by the key feature enhancement subunit in the first type of downsampling unit. The convolution kernel size of the key feature enhancement convolution layer contained in the key feature enhancement subunit can be 1×1, and the step size can be 1. Wherein, by using the second type of downsampling unit, the first input feature image is subjected to the key feature enhancement, and the downsampled feature enhancement image is obtained by the key feature enhancement subunit in the second type of downsampling unit. The convolution kernel size of the key feature enhancement convolution layer contained in the key feature enhancement subunit can be 1×1, and the step size can be 1. Wherein, by using the second type of downsampling unit, the downsampled feature enhancement image is subjected to multi-channel downsampling, and the second downsampled feature image is obtained by the channel-by-channel convolution layer in the second type of downsampling unit. The convolution kernel size of the channel-by-channel convolution layer can be 3×3, and the step size can be 2.

[0114] Step 306: Fuse the first down-sampled feature image and the second down-sampled feature image to obtain a target down-sampled feature image.

[0115] Exemplarily, step 306 includes: performing weighted fusion on the first down-sampled feature image and the second down-sampled feature image to obtain a target down-sampled feature image.

[0116] Step 308: Use the target downsampled feature image as the second input feature image of the multi-scale feature extraction unit, perform multi-scale feature extraction using the multi-scale feature extraction unit, and obtain a second feature image output by the deep multi-scale feature extraction module.

[0117] Exemplarily, the multi-scale feature extraction unit includes a channel number expansion subunit, a channel dimension slicing subunit, a splicing subunit, at least two lightweight feature enhancement subunits and a multi-scale attention extraction subunit; the multi-scale feature extraction unit is used to perform multi-scale feature extraction, including: obtaining a third input feature image of the multi-scale feature extraction unit, and performing channel number expansion on the third input feature image through the channel number expansion subunit to obtain a channel number expanded image, wherein the third input feature image is an optical image, or a target down-sampled feature image; performing channel dimension slicing on the channel number expanded image through the channel dimension slicing subunit to obtain a first slicing image and a second slicing image with the same number of channels. Slicing an image; performing feature enhancement on a first sliced ​​image through at least two sequentially connected lightweight feature enhancement subunits to obtain lightweight feature enhanced images output by each lightweight feature enhancement subunit, and splicing each lightweight feature enhanced image through a splicing subunit to obtain a first spliced ​​image; performing multi-scale attention feature enhancement on the first spliced ​​image through a multi-scale attention extraction subunit to obtain a first multi-scale feature enhanced image, and performing multi-scale attention enhancement on the second sliced ​​image through a multi-scale attention extraction subunit to obtain a second multi-scale feature enhanced image; splicing the first multi-scale feature enhanced image and the second multi-scale feature enhanced image through a splicing subunit.

[0118] The channel number expansion subunit includes a convolution layer with a convolution kernel size of 1×1 and a step size of 1, a normalization layer, and an activation function layer. The lightweight feature enhancement subunit includes at least two key feature enhancement subunits connected in sequence, the convolution kernel size and step size of the key feature enhancement convolution layer of the key feature enhancement subunit in the lightweight feature enhancement subunit are the same, the convolution kernel size can be 3×3, and the step size can be 1, and the lightweight feature enhancement image output by the lightweight feature enhancement subunit is a fusion image between the input image of the lightweight feature enhancement subunit and the output image of the last key feature enhancement subunit in the lightweight feature enhancement subunit.

[0119] As an embodiment, multi-scale attention feature enhancement is performed on a first stitched image to obtain a first multi-scale feature enhanced image, including: obtaining a preset hyperparameter, wherein the hyperparameter is a divisor of the number of channels of an input image corresponding to a multi-scale attention extraction subunit (here, the first stitched image), and segmenting the first stitched image according to the hyperparameter to obtain a plurality of hyper-parameter segmented images, wherein the number of channels between the plurality of hyper-parameter segmented images is the same, and the sum of the number of channels between the plurality of hyper-parameter segmented images is equal to the number of channels of the first stitched image; performing multi-scale feature extraction on the plurality of hyper-parameter segmented images respectively to obtain a plurality of scale feature images; and fusing the plurality of scale feature images according to attention weight information corresponding to the plurality of scale feature images to obtain a first multi-scale feature enhanced image.

[0120] Furthermore, multi-scale feature extraction is performed on multiple hyperparameter segmentation images respectively to obtain multiple scale feature images, including: for any hyperparameter segmentation image, according to the segmentation number corresponding to the hyperparameter segmentation image, determining the convolution kernel size of the convolution layer used when performing feature extraction on the hyperparameter segmentation image, according to the convolution kernel size of the convolution layer used when performing feature extraction on the hyperparameter segmentation image, determining the number of convolution channels of the convolution layer used when performing feature extraction on the hyperparameter segmentation image, according to the convolution kernel size and the number of convolution channels of the convolution layer used when performing feature extraction on the hyperparameter segmentation image, performing scale feature extraction on the high-dimensional segmentation feature image, to obtain a scale feature image.

[0121] Optionally, according to the segmentation number corresponding to the hyperparameter segmentation image, the convolution kernel size of the convolution layer used when extracting features from the hyperparameter segmentation image is determined, which can be expressed by the formula:

[0122]

[0123] in, is the convolution kernel size of the convolution layer used to extract features from the hyperparameter segmented image. The segmentation number corresponding to the hyperparameter segmentation image.

[0124] Optionally, the number of convolution channels of the convolution layer used when extracting features from the hyperparameter segmented image is determined according to the convolution kernel size of the convolution layer used when extracting features from the hyperparameter segmented image, which can be expressed in the formula:

[0125]

[0126] in, The number of convolution channels of the convolution layer used to extract features from the hyperparameter segmentation image.

[0127] Optionally, according to the convolution kernel size and the number of convolution channels of the convolution layer used when extracting features from the hyperparameter segmented image, scale features are extracted from the high-dimensional segmented feature image to obtain a scale feature image, which can be expressed as follows:

[0128]

[0129] in, is the scale feature image, For the Hyperparameters segment the image, is a hyperparameter.

[0130] As an embodiment, according to the attention weight information corresponding to the multiple scale feature images, multiple scale feature images are fused to obtain a first multi-scale feature enhanced image, including: according to the attention weight information corresponding to the multiple scale feature images, the multiple scale feature images are attention stitched to obtain an attention fused feature image, and the attention fused feature images are fused channel by channel to obtain a first multi-scale feature enhanced image.

[0131] As an example, refer to Figure 5 , when the image size of the third input feature image is 128×128×128, corresponding to the length, width and number of channels of the feature map, a channel number expansion subunit with a convolution kernel size of 1×1 and a step size of 1 is used to obtain a channel number expansion image, and the image size of the channel number expansion image is 128×128×256; the channel number expansion image is segmented to obtain a first segmented image and a second segmented image, and the image sizes of the first segmented image and the second segmented image are both 128×128×128; the first segmented image is feature processed by the first lightweight feature enhancement subunit to obtain a first lightweight feature enhanced image, the first lightweight feature enhanced image is feature processed by the second lightweight feature enhancement subunit to obtain a second lightweight feature enhanced image, and the second lightweight feature enhanced image is feature processed by the third lightweight feature enhancement subunit to obtain a third lightweight feature. Enhanced image, the image sizes of the first lightweight feature enhanced image, the second lightweight feature enhanced image and the third lightweight feature enhanced image are all 128×128×128; the first lightweight feature enhanced image, the second lightweight feature enhanced image and the third lightweight feature enhanced image are spliced ​​to obtain a first spliced ​​image, and the image size of the first spliced ​​image is 128×128×384; the first spliced ​​image is enhanced with multi-scale attention features to obtain a first multi-scale feature enhanced image, and the image size of the first multi-scale feature enhanced image is 128×128×384; the second segmented image is enhanced with multi-scale attention features to obtain a second multi-scale feature enhanced image, and the size of the second multi-scale feature enhanced image is 128×128×128; finally, the first multi-scale feature enhanced image and the second multi-scale feature enhanced image are spliced ​​to obtain a second feature image.

[0132] As an example, refer to Figure 6When the image size of the first input feature image is 128×128×256, the most critical feature of the first input feature image is downsampled to obtain a downsampled feature extraction image, and the image size of the downsampled feature extraction image is 64×64×256. The downsampled feature extraction image is enhanced by the first type of downsampling unit to obtain a first downsampled feature image, and the image size of the first downsampled feature image is 64×64×256; the first input feature image is enhanced by the second type of downsampling unit to obtain a downsampled feature enhancement image, and the image size of the downsampled feature enhancement image is 128×128×256; the downsampled feature enhancement image is multi-channel downsampled by the second type of downsampling unit to obtain a second downsampled feature image, and the image size of the second downsampled feature image is 64×64×256; the first downsampled feature image and the second downsampled feature image are fused to obtain a target downsampled feature image, and the size of the target downsampled feature image is 64×64×256.

[0133] In this embodiment, by respectively adopting two feature enhancement types of downsampling methods for the first input feature image, the sampled feature images obtained by the two feature enhancement types of downsampling methods are fused, and multi-scale feature extraction is performed on the fused target down-sampled feature image, thereby ensuring that the extracted second feature map can represent multi-scale feature information, thereby improving the accuracy of feature extraction of optical images.

[0134] In an exemplary embodiment, Figure 7 As shown, a method for accurately classifying radar slice images is provided, and each radar slice image is classified separately to obtain the slice classification result corresponding to each radar slice image, including steps 402 to 408. Among them:

[0135] Step 402: for any radar segmentation image, perform multi-channel downsampling on the radar segmentation image to obtain a radar segmentation downsampling feature image.

[0136] Exemplarily, step 402 includes: for any radar segmentation image, using the second preliminary feature extraction module to perform preliminary feature extraction on the radar segmentation image to obtain a radar segmentation feature image; expanding the number of channels of the radar segmentation feature image to obtain a radar segmentation expanded image; enhancing the spatial features of the radar segmentation expanded image to obtain a radar segmentation expanded enhanced image; and performing multi-channel downsampling on the radar segmentation expanded enhanced image to obtain a radar segmentation downsampled feature image.

[0137] Further, the second preliminary feature extraction module includes a key feature convolution layer and at least two inverted residual bottleneck convolution layers connected in sequence, the convolution kernel size of the key feature convolution layer in the second preliminary feature extraction module can be 3×3, the step size can be 2, the convolution kernel size of the first inverted residual bottleneck convolution layer can be 3×3, the step size can be 1, and the convolution kernel size of the non-first inverted residual bottleneck convolution layer can be 3×3, the step size can be 2. Further, the radar segmentation dilation image is spatially enhanced to obtain a radar segmentation dilation enhanced image, including: decomposing the channel attention of the radar segmentation dilation image to obtain spatial aggregation features in two spatial directions; complementary fusion enhancement of the spatial aggregation features in the two spatial directions to obtain the radar segmentation dilation enhanced image. Among them, the radar segmentation dilation enhanced image is multi-channel down-sampled to obtain the radar segmentation down-sampled feature image by a channel-by-channel convolution layer, and the convolution kernel size of the channel-by-channel convolution layer can be 3×3, and the step size can be 1 or 2.

[0138] Step 404 , performing spatial feature enhancement on the radar segmentation down-sampled feature image to obtain a radar segmentation enhanced feature image.

[0139] Optionally, the specific implementation steps of performing spatial feature enhancement on the radar segmentation and downsampling feature image to obtain the radar segmentation and enhanced feature image can refer to the specific implementation content of the above-mentioned steps of performing spatial feature enhancement on the radar segmentation and dilated image to obtain the radar segmentation and dilated enhanced image, which will not be repeated here.

[0140] Step 406: Determine the target enhanced feature image corresponding to the radar segmentation image according to the radar segmentation enhanced feature image.

[0141] Exemplarily, step 406 includes: determining an image size relationship between the radar segmentation enhanced feature image and the radar segmentation image; if the image size relationship indicates that the image sizes of the radar segmentation enhanced feature image and the radar segmentation image are the same, then performing image addition on the radar segmentation enhanced feature image and the radar segmentation image to obtain a target enhanced feature image corresponding to the radar segmentation image; if the image size relationship indicates that the image sizes of the radar segmentation enhanced feature image and the radar segmentation image are different, then determining the radar segmentation enhanced feature image as the target enhanced feature image corresponding to the radar segmentation image.

[0142] Further, determining the image size relationship between the radar segmentation enhanced feature image and the radar segmentation image includes: if the step size of the channel-by-channel convolution layer corresponding to the synthetic aperture radar segmentation image is a first value, then determining the image size relationship between the radar segmentation enhanced feature image and the radar segmentation image indicates that the image sizes of the radar segmentation enhanced feature image and the radar segmentation image are the same; if the step size of the channel-by-channel convolution layer corresponding to the synthetic aperture radar segmentation image is a second value, then determining the image size relationship between the radar segmentation enhanced feature image and the radar segmentation image indicates that the image sizes of the radar segmentation enhanced feature image and the radar segmentation image are different; the first value is less than the second value. As an embodiment, the first value is 1 and the second value is 2.

[0143] As an example, refer to Figure 8 When the step size of the channel-by-channel convolution layer corresponding to the synthetic aperture radar segmentation image is 1, the radar segmentation enhancement feature image and the radar segmentation image are added to obtain the target enhancement feature image corresponding to the radar segmentation image. When the step size of the channel-by-channel convolution layer corresponding to the synthetic aperture radar segmentation image is 2, the radar segmentation enhancement feature image is determined as the target enhancement feature image corresponding to the radar segmentation image.

[0144] Step 408 , classifying each radar slice image according to the target enhanced feature image corresponding to each radar slice image, and obtaining a slice classification result corresponding to each radar slice image.

[0145] Exemplarily, step 408 includes: using a slice classification model to classify each radar slice image according to a target enhanced feature image corresponding to each radar slice image, to obtain a slice classification result corresponding to each radar slice image.

[0146] In this embodiment, multi-channel downsampling and spatial feature enhancement are performed on each radar slice image to ensure that long-range dependencies can be captured along one spatial direction while retaining precise position information along another spatial direction. Subsequent channel-by-channel convolution and attention encoding can enhance the attention representation of the radar slice image processing process and improve the accuracy of feature extraction of the target enhanced feature image, which is the basis for slice classification. Therefore, the slice accuracy of the radar slice image can be improved.

[0147] As a detailed embodiment, an optical image and a synthetic aperture radar image of a power transmission channel to be detected are obtained; a multi-scale feature extraction unit is used to perform multi-scale feature extraction on the optical image to obtain a first feature image; the first feature image is passed through at least two deep multi-scale feature extraction modules connected in sequence to obtain a second feature image output by each deep multi-scale feature extraction module, wherein the deep multi-scale feature extraction module includes a downsampling unit and a multi-scale feature extraction unit; each second feature image is fused to obtain a third feature image, and channel hidden danger detection is performed on the third feature image to obtain position information of multiple target detection frames; for the position information of any target detection frame, the position information of the target detection frame is coordinate-converted to obtain conversion position information in a coordinate system corresponding to the synthetic aperture radar image; according to the conversion position information of each target detection frame, each target detection frame is respectively mapped to the synthetic aperture radar image to obtain a mapping image; according to the mapping image, the mapping area corresponding to each target detection frame in the synthetic aperture radar image is sliced ​​to obtain a radar slice image corresponding to each target detection frame.

[0148] Furthermore, for any radar segmentation image, multi-channel downsampling is performed on the radar segmentation image to obtain a radar segmentation downsampled feature image; spatial feature enhancement is performed on the radar segmentation downsampled feature image to obtain a radar segmentation enhanced feature image; the image size relationship between the radar segmentation enhanced feature image and the radar segmentation image is determined; if the image size relationship indicates that the image size of the radar segmentation enhanced feature image is the same as that of the radar segmentation image, the radar segmentation enhanced feature image and the radar segmentation image are added to obtain a target enhanced feature image corresponding to the radar segmentation image; if the image size relationship indicates that the image size of the radar segmentation enhanced feature image is different from that of the radar segmentation image, the radar segmentation enhanced feature image is determined as the target enhanced feature image corresponding to the radar segmentation image; and according to the target enhanced feature image corresponding to each radar slice image, each radar slice image is classified respectively to obtain a slice classification result corresponding to each radar slice image.

[0149] Among them, for any deep multi-scale feature extraction module, a first input feature image of the downsampling unit is obtained, wherein the first input feature map is the first feature image, or the second feature image output by the previous deep multi-scale feature extraction module; the first input feature image is downsampled with the most critical features through the first type of downsampling unit to obtain a downsampled feature extraction image; the downsampled feature extraction image is enhanced with the first type of downsampling unit to obtain a first downsampled feature image; the first input feature image is enhanced with the second type of downsampling unit to obtain a downsampled feature enhanced image; the downsampled feature enhanced image is multi-channel downsampled through the second type of downsampling unit to obtain a second downsampled feature image; the first downsampled feature image and the second downsampled feature image are fused to obtain a target downsampled feature image; the target downsampled feature image is used as the second input feature map of the multiscale feature extraction unit to obtain a third input feature image of the multiscale feature extraction unit, and the third input feature is enhanced through the channel number expansion subunit. The method comprises the steps of: performing channel number expansion on a feature image to obtain a channel number expanded image, wherein the third input feature image is an optical image, or a target downsampled feature image; dividing the channel number expanded image in the channel dimension through a channel dimension dividing subunit to obtain a first divided image and a second divided image with the same number of channels; performing feature enhancement on the first divided image through at least two lightweight feature enhancement subunits connected in sequence to obtain a lightweight feature enhanced image output by each lightweight feature enhancement subunit, and splicing each lightweight feature enhanced image through a splicing subunit to obtain a first spliced ​​image; performing multi-scale attention feature enhancement on the first spliced ​​image through a multi-scale attention extraction subunit to obtain a first multi-scale feature enhanced image, and performing multi-scale attention enhancement on the second divided image through the multi-scale attention extraction subunit to obtain a second multi-scale feature enhanced image; splicing the first multi-scale feature enhanced image and the second multi-scale feature enhanced image through a splicing subunit to obtain a second feature image output by a deep multi-scale feature extraction module.

[0150] In this way, by taking the synthetic aperture radar image and the optical image as the source of image processing, the extracted features are not restricted by the environment of the power transmission channel to be detected. Then, the optical image is subjected to multi-scale feature extraction by a multi-scale feature extraction unit to obtain a first feature image, thereby ensuring that the input features can represent multi-scale feature information in the input data dimension, and the first feature image is passed through at least two deep multi-scale feature extraction modules connected in sequence to obtain a second feature image output by each deep multi-scale feature extraction module, thereby ensuring that the feature extraction process is multi-scale extraction in the feature extraction dimension, and the second feature images are fused to obtain a third feature image, and the third feature image is subjected to channel hidden danger detection to obtain the position information of multiple target detection frames, that is, the multiple second feature images obtained by the multi-scale feature extraction are fused to obtain a further multi-scale feature extraction. The third characteristic image under the condition is obtained, and the third characteristic image is used as the basis for channel hidden danger detection. Therefore, the target detection accuracy is improved. Then, based on the position information of the target detection frame, the synthetic aperture radar image is sliced ​​to obtain multiple radar slice images, so that the radar slice image can, to a certain extent, characterize the preliminary results obtained under the optical image processing process; thereby, each radar slice image is classified respectively to obtain the slice classification results corresponding to each radar slice image, and finally, according to each slice classification result, the transmission channel hidden danger detection result corresponding to the transmission channel to be detected is determined. It is known from the above analysis that the radar slice image is an image obtained after layer-by-layer feature extraction and processing. Therefore, the slice classification result obtained by classification can be made more accurate, and then the transmission channel hidden danger detection result determined is more in line with the actual situation of the transmission channel to be detected, so the accuracy of transmission channel hidden danger detection is improved.

[0151] Furthermore, by respectively adopting two types of feature enhancement downsampling methods for the first input feature image, fusing the sampled feature images obtained by the two types of feature enhancement downsampling methods, and performing multi-scale feature extraction on the fused target downsampled feature image, it is ensured that the extracted second feature map can represent multi-scale feature information, thereby improving the accuracy of feature extraction of the optical image; and, multi-channel downsampling and spatial feature enhancement are performed on each radar slice image, thereby ensuring that long-range dependencies can be captured along one spatial direction, while retaining accurate position information along another spatial direction, and subsequent channel-by-channel convolution and attention encoding can enhance the attention representation of the radar slice image processing process, and improve the accuracy of feature extraction of the target enhanced feature image, and the target enhanced feature image is the basis for slice classification, therefore, the slice accuracy of the radar slice image can be improved.

[0152] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indications of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless clearly stated in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps. Based on the same inventive concept, the embodiment of the present application also provides a power transmission channel hidden danger detection device for implementing the power transmission channel hidden danger detection method involved in the above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific definition in one or more power transmission channel hidden danger detection device embodiments provided below can refer to the definition of the power transmission channel hidden danger detection method in the above text, and will not be repeated here.

[0153] In an exemplary embodiment, Fig. 9 As shown, a power transmission channel hidden danger detection device 900 is provided, including: an acquisition module 902, an extraction module 904, a detection module 906, a slicing module 908 and a classification module 910, wherein:

[0154] An acquisition module 902 acquires an optical image and a synthetic aperture radar image of the power transmission channel to be detected;

[0155] The extraction module 904 is used to perform multi-scale feature extraction on the optical image using a multi-scale feature extraction unit to obtain a first feature image; pass the first feature image through at least two sequentially connected deep multi-scale feature extraction modules to obtain a second feature image output by each deep multi-scale feature extraction module, wherein the deep multi-scale feature extraction module includes a downsampling unit and a multi-scale feature extraction unit; and fuse the second feature images to obtain a third feature image;

[0156] A detection module 906 is used to perform channel hidden danger detection on the third feature image to obtain position information of multiple target detection frames;

[0157] The slicing module 908 is used to slice the synthetic aperture radar image according to the position information of the multiple target detection frames to obtain a radar slice image corresponding to each target detection frame;

[0158] A classification module 910 is used to classify each radar slice image respectively to obtain a slice classification result corresponding to each radar slice image;

[0159] The detection module 906 is further used to determine the power transmission channel hidden danger detection result corresponding to the power transmission channel to be detected according to the classification results of each slice.

[0160] In one embodiment, the extraction module 904 is also used to obtain a first input feature image of a downsampling unit, wherein the first input feature image is a first feature image, or a second feature image output by a previous deep multi-scale feature extraction module; the first input feature image is subjected to a first type of downsampling through the downsampling unit to obtain a first downsampled feature image, and the first input feature image is subjected to a second type of downsampling through the downsampling unit to obtain a second downsampled feature image, wherein the first type of downsampling is used to enhance key features of the first input feature image; the second type of downsampling is used to enhance multi-channel sampling features of the first input feature image; the first downsampled feature image and the second downsampled feature image are fused to obtain a target downsampled feature image; the target downsampled feature image is used as the second input feature image of the multiscale feature extraction unit, and the multiscale feature extraction unit is used to perform multiscale feature extraction to obtain a second feature image output by the deep multi-scale feature extraction module.

[0161] In one embodiment, the downsampling unit includes a first type of downsampling unit and a second type of downsampling unit; the extraction module 904 is also used to perform the most critical feature downsampling on the first input feature image through the first type of downsampling unit to obtain a downsampled feature extraction image; perform key feature enhancement on the downsampled feature extraction image through the first type of downsampling unit to obtain a first downsampled feature image; perform key feature enhancement on the first input feature image through the second type of downsampling unit to obtain a downsampled feature enhanced image; perform multi-channel downsampling on the downsampled feature enhanced image through the second type of downsampling unit to obtain a second downsampled feature image.

[0162] In one embodiment, the multi-scale feature extraction unit includes a channel number expansion subunit, a channel dimension slicing subunit, a splicing subunit, at least two lightweight feature enhancement subunits and a multi-scale attention extraction subunit; the extraction module 904 is also used to obtain a third input feature image of the multi-scale feature extraction unit, and expand the channel number of the third input feature image through the channel number expansion subunit to obtain a channel number expanded image, wherein the third input feature image is an optical image, or a target down-sampled feature image; the channel number expanded image is sliced ​​on the channel dimension through the channel dimension slicing subunit to obtain a first sliced ​​image and a second sliced ​​image with the same number of channels. ; Perform feature enhancement on the first segmented image through at least two lightweight feature enhancement subunits connected in sequence to obtain lightweight feature enhanced images output by each lightweight feature enhancement subunit, and splice each lightweight feature enhanced image through a splicing subunit to obtain a first spliced ​​image; perform multi-scale attention feature enhancement on the first spliced ​​image through a multi-scale attention extraction subunit to obtain a first multi-scale feature enhanced image, and perform multi-scale attention enhancement on the second segmented image through a multi-scale attention extraction subunit to obtain a second multi-scale feature enhanced image; splice the first multi-scale feature enhanced image and the second multi-scale feature enhanced image through a splicing subunit.

[0163] In one of the embodiments, the classification module 910 is also used to perform multi-channel downsampling on any radar segmentation image to obtain a radar segmentation downsampled feature image; perform spatial feature enhancement on the radar segmentation downsampled feature image to obtain a radar segmentation enhanced feature image; determine a target enhanced feature image corresponding to the radar segmentation image based on the radar segmentation enhanced feature image; and classify each radar slice image according to the target enhanced feature image corresponding to each radar slice image to obtain a slice classification result corresponding to each radar slice image.

[0164] In one of the embodiments, the classification module 910 is further used to determine the image size relationship between the radar segmentation enhancement feature image and the radar segmentation image; if the image size relationship indicates that the image size of the radar segmentation enhancement feature image is the same as that of the radar segmentation image, the radar segmentation enhancement feature image and the radar segmentation image are added to obtain a target enhanced feature image corresponding to the radar segmentation image; if the image size relationship indicates that the image size of the radar segmentation enhancement feature image and the radar segmentation image is different, the radar segmentation enhancement feature image is determined as the target enhanced feature image corresponding to the radar segmentation image.

[0165] In one embodiment, the slicing module 908 is further configured to perform coordinate transformation on the position information of any target detection box to obtain transformed position information in the coordinate system corresponding to the synthetic aperture radar image; map each target detection box to the synthetic aperture radar image respectively according to the transformed position information of each target detection box to obtain a mapped image; and slice the mapped regions corresponding to each target detection box in the synthetic aperture radar image respectively according to the mapped image to obtain a radar slice image corresponding to each target detection box.

[0166] Each module in the above power transmission channel hidden danger detection device can be implemented in whole or in part by software, hardware, and their combination. Each of the above modules can be embedded in the processor in the computer device in hardware form or be independent of the processor, or can be stored in the memory in the computer device in software form so that the processor can call and execute the operations corresponding to each of the above modules.

[0167] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as Fig.10 shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface, the display unit, and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used for exchanging information between the processor and external devices. The communication interface of the computer device is used for communicating with external terminals in a wired or wireless manner. The wireless manner can be implemented through WIFI, a mobile cellular network, near field communication (NFC), or other technologies. When the computer program is executed by the processor, it implements a power transmission channel hidden danger detection method. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the computer device housing, or an external keyboard, a touchpad, or a mouse, etc.

[0168] Those skilled in the art can understand that Fig.10The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0169] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0170] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0171] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0172] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0173] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0174] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for detecting hidden dangers in a power transmission channel, characterized in that: The method comprises: Acquire optical images and synthetic aperture radar images of the power transmission channel to be inspected; Using a multi-scale feature extraction unit, performing multi-scale feature extraction on the optical image to obtain a first feature image; Passing the first feature image through at least two sequentially connected deep multi-scale feature extraction modules to obtain a second feature image output by each of the deep multi-scale feature extraction modules, wherein the deep multi-scale feature extraction module includes a downsampling unit and a multi-scale feature extraction unit; The second feature images are fused to obtain a third feature image, and channel hidden danger detection is performed on the third feature image to obtain position information of multiple target detection frames; Slice the synthetic aperture radar image according to the position information of the multiple target detection frames to obtain a radar slice image corresponding to each target detection frame; Classifying each of the radar slice images respectively to obtain a slice classification result corresponding to each of the radar slice images; According to the classification results of each of the slices, a power transmission channel hidden danger detection result corresponding to the power transmission channel to be detected is determined.

2. The method according to claim 1, characterized in that For any of the deep multi-scale feature extraction modules, the method further includes: Acquire a first input feature image of the downsampling unit, wherein the first input feature image is the first feature image, or a second feature image output by a previous deep multi-scale feature extraction module; The first input feature image is subjected to a first type of downsampling by the downsampling unit to obtain a first downsampled feature image, and the first input feature image is subjected to a second type of downsampling by the downsampling unit to obtain a second downsampled feature image, wherein the first type of downsampling is used to enhance key features of the first input feature image; and the second type of downsampling is used to enhance multi-channel sampling features of the first input feature image; fusing the first down-sampled feature image and the second down-sampled feature image to obtain a target down-sampled feature image; The target downsampled feature image is used as the second input feature map of the multiscale feature extraction unit, and the multiscale feature extraction unit is used to perform multiscale feature extraction to obtain a second feature image output by the deep multiscale feature extraction module.

3. The method according to claim 2, characterized in that The downsampling unit includes a first type of downsampling unit and a second type of downsampling unit; the first input feature image is subjected to a first type of downsampling by the downsampling unit to obtain a first downsampled feature image, and the first input feature image is subjected to a second type of downsampling by the downsampling unit to obtain a second downsampled feature image, including: Downsampling the most critical features of the first input feature image by the first type of downsampling unit to obtain a downsampled feature extraction image; Performing key feature enhancement on the downsampled feature extraction image by using the first type of downsample units to obtain a first downsampled feature image; Performing key feature enhancement on the first input feature image by using the second type of downsampling unit to obtain a downsampled feature enhanced image; The downsampled feature enhanced image is subjected to multi-channel downsampling by the second-type downsampling unit to obtain a second downsampled feature image.

4. The method according to claim 1 or 2, characterized in that: The multi-scale feature extraction unit includes a channel number expansion subunit, a channel dimension slicing subunit, a splicing subunit, at least two lightweight feature enhancement subunits and a multi-scale attention extraction subunit; using the multi-scale feature extraction unit to perform multi-scale feature extraction includes: Acquire a third input feature image of the multi-scale feature extraction unit, and perform channel expansion on the third input feature image through the channel expansion subunit to obtain a channel expansion image, wherein the third input feature image is an optical image or a target down-sampled feature image; The channel-divided image is segmented in the channel dimension by the channel-dimension segmentation subunit to obtain a first segmented image and a second segmented image with the same number of channels; Performing feature enhancement on the first segmented image through at least two sequentially connected lightweight feature enhancement subunits to obtain lightweight feature enhanced images output by each of the lightweight feature enhancement subunits, and splicing the lightweight feature enhanced images through the splicing subunit to obtain a first spliced ​​image; Performing multi-scale attention feature enhancement on the first spliced ​​image through the multi-scale attention extraction subunit to obtain a first multi-scale feature enhanced image, and performing multi-scale attention enhancement on the second segmented image through the multi-scale attention extraction subunit to obtain a second multi-scale feature enhanced image; The first multi-scale feature enhanced image and the second multi-scale feature enhanced image are spliced ​​by the splicing subunit.

5. The method according to claim 1, characterized in that The step of classifying the radar slice images respectively to obtain slice classification results corresponding to the radar slice images includes: For any radar segmentation image, multi-channel downsampling is performed on the radar segmentation image to obtain a radar segmentation downsampling feature image; Performing spatial feature enhancement on the radar segmentation downsampled feature image to obtain a radar segmentation enhanced feature image; Determine, according to the radar segmentation enhanced feature image, a target enhanced feature image corresponding to the radar segmentation image; According to the target enhanced feature image corresponding to each radar slice image, each radar slice image is classified respectively to obtain a slice classification result corresponding to each radar slice image.

6. The method according to claim 5, characterized in that The step of determining the target enhanced feature image corresponding to the radar segmentation image according to the radar segmentation enhanced feature image comprises: Determining an image size relationship between the radar segmentation enhanced feature image and the radar segmentation image; If the image size relationship indicates that the radar segmentation enhanced feature image and the radar segmentation image have the same image size, then performing image addition on the radar segmentation enhanced feature image and the radar segmentation image to obtain a target enhanced feature image corresponding to the radar segmentation image; If the image size relationship indicates that the image sizes of the radar segmentation enhanced feature image and the radar segmentation image are different, the radar segmentation enhanced feature image is determined as the target enhanced feature image corresponding to the radar segmentation image.

7. The method according to claim 1, characterized in that The step of slicing the synthetic aperture radar image according to the position information of the multiple target detection frames to obtain a radar slice image corresponding to each target detection frame includes: For any of the position information of the target detection frame, coordinate transformation is performed on the position information of the target detection frame to obtain transformation position information in a coordinate system corresponding to the synthetic aperture radar image; According to the conversion position information of each target detection frame, each target detection frame is mapped to the synthetic aperture radar image to obtain a mapping image; According to the mapping image, the mapping areas corresponding to the target detection frames in the synthetic aperture radar image are sliced ​​respectively to obtain radar slice images corresponding to each target detection frame.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images