Substation complex background-oriented semantic segmentation and image enhancement method and system

Abstract

The invention provides a semantic segmentation and image enhancement method and system for a complex background of a transformer substation, and relates to the technical field of computer vision, and the method comprises the following steps: collecting original image data of the transformer substation, and combining heat map saliency analysis and a spectrum residual error method based on a reciprocal function to obtain a semantic segmentation and image enhancement result; preprocessing the collected original image data to obtain a hue saturation brightness image; performing brightness component extraction on the hue saturation brightness image, and establishing an end-to-end image enhancement model based on an IAT model; performing brightness component enhancement by using the image enhancement model to obtain an enhanced image; and performing diffusion processing on the enhanced image based on a diffusion coefficient function, and performing semantic segmentation processing on the image after diffusion processing by using a semantic segmentation model in combination with a Laplacian operator to obtain a semantic segmentation result. According to the method, image distortion and excessive problems caused by image enhancement are avoided, and the readability of the image is ensured, so that a reliable data basis is provided for subsequent analysis.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and more specifically, to a semantic segmentation and image enhancement method and system for complex backgrounds in substations. Background Technology

[0002] With the continuous expansion of the power grid and the advancement of artificial intelligence technology, the inspection of power facilities has become increasingly complex and crucial. Traditional power inspection methods mainly rely on manual operation, which is not only inefficient but also poses significant safety risks in complex terrain or severe weather conditions. Furthermore, manual inspections struggle to achieve high-frequency monitoring, and their detection and response speed to substation faults and worker misconduct is relatively slow. In recent years, drone technology has developed rapidly and is increasingly being applied to the field of power inspection. Drones can quickly reach locations inaccessible to humans, performing efficient and comprehensive inspection tasks.

[0003] However, in complex power line environments, specific components such as poles, insulators, and conductors are often difficult to identify accurately. Furthermore, due to the complexity of the inspection environment, such as poor lighting conditions and weather effects, the captured images may have quality issues, such as blurriness, uneven brightness, and inaccurate colors. This not only limits the UAV system's ability to detect and analyze faults in specific components but also affects the accuracy of inspection personnel's assessment of equipment status.

[0004] No effective solutions have yet been proposed to address the problems in the relevant technologies. Summary of the Invention

[0005] In view of this, the present invention provides a semantic segmentation and image enhancement method and system for complex backgrounds in substations, in order to solve the aforementioned problems.

[0006] To solve the above problems, the specific technical solution adopted by the present invention is as follows:

[0007] According to a first aspect of the present invention, a semantic segmentation and image enhancement method for complex backgrounds in substations is provided, the method comprising the following steps:

[0008] S1. The drone with the camera on board collects the original image data of the substation, and combines the heat map significance analysis and the spectral residual method based on the reciprocal function to preprocess the collected original image data to obtain the hue, saturation and brightness image.

[0009] S2. Extract the luminance component from the hue, saturation, and luminance image, and establish an end-to-end image enhancement model based on the IAT model; use the image enhancement model to enhance the luminance component to obtain the enhanced image;

[0010] S3. The enhanced image is diffused based on the diffusion coefficient function, and then the semantic segmentation model is used to perform semantic segmentation on the diffused image to obtain the semantic segmentation result.

[0011] Preferably, the step of using a drone equipped with a camera to collect raw image data of the substation, and combining it with heatmap saliency analysis and the spectral residual method based on the reciprocal function to preprocess the collected raw image data to obtain a hue, saturation, and brightness image includes the following steps:

[0012] S11. Perform preliminary processing on the raw image data of the substation, including image denoising, geometric correction, and resolution normalization.

[0013] S12. Based on the original image data of the substation after preliminary processing, perform salience detection based on the salience target detection model to generate a first visual salience heatmap;

[0014] S13. Based on the original image data of the substation after preliminary processing, perform multi-feature saliency detection using the spectral residual method based on the reciprocal function to generate a second visual saliency heatmap.

[0015] S14. The first visual saliency heatmap and the second visual saliency heatmap are fused to obtain the fused visual saliency heatmap.

[0016] S15. Perform threshold segmentation on the fused visual saliency heatmap to generate a binary mask for the device region;

[0017] S16. Convert the raw image data of the substation after preliminary processing to the HSV color space, and enhance the HSV channel using a binary mask to obtain an optimized hue, saturation and brightness image.

[0018] Preferably, the step of generating a first visual saliency heatmap by performing saliency detection based on a saliency target detection model using the pre-processed original image data of the substation includes the following steps:

[0019] S121. Based on the feedforward approach, the original image data of the substation after preliminary processing is processed using a salient target detection model to generate a salient heat map of the equipment.

[0020] S122. The original image data of the substation after preliminary processing is divided into salient and non-salient regions. Bilateral filtering is performed on the salient regions and non-local mean filtering is performed on the non-salient regions to obtain bilateral filtered output image and non-local mean filtered image.

[0021] S123. Using the device saliency heatmap as a weighted guide, and combining the bilateral filter output image and the non-local mean filter image, calculate the first visual saliency heatmap.

[0022] Preferably, the step of generating a second visual saliency heatmap by performing multi-feature saliency detection based on the spectral residual method using the reciprocal function according to the pre-processed original image data of the substation includes the following steps:

[0023] S131. Using the Fourier transform method, the pre-processed original substation image data is transformed from the spatial domain to the frequency domain to obtain a frequency domain image.

[0024] S132. Perform a reciprocal function transformation on the frequency domain image, and calculate the spectral residual spectrum based on the transformation result; extract the saliency response maps of color features, texture features and shape features according to the spectral residual spectrum, and fuse them to obtain the fused saliency response map;

[0025] S133. Using a normalization function, the fused saliency response map is normalized to generate a second visual saliency heatmap.

[0026] Preferably, the step of performing diffusion processing on the enhanced image based on the diffusion coefficient function, and then combining it with the Laplacian operator to perform semantic segmentation processing on the diffused image using a semantic segmentation model to obtain the semantic segmentation result includes the following steps:

[0027] S31. Based on the dual evaluation method, image gradient calculation is performed on the raw image data of the substation after preliminary processing, and diffusion coefficient function is constructed by combining the first visual saliency heatmap.

[0028] S32. Apply the diffusion coefficient function to the enhanced image to obtain the enhanced image after diffusion processing;

[0029] S33. Input the enhanced image after diffusion processing into a semantic segmentation model combined with Laplacian for pixel-level classification to obtain the final semantic segmentation result.

[0030] Preferably, the step of calculating the image gradient of the pre-processed original image data of the substation based on the dual evaluation method and constructing the diffusion coefficient function in combination with the first visual saliency heatmap includes the following steps:

[0031] S311. Divide the pre-processed original image data of the substation into several fixed blocks of a preset size;

[0032] S312. Calculate the gradient smoothness index for each fixed block in the horizontal, vertical, diagonal, and anti-diagonal directions; and calculate the dual evaluation value of the center point of each block based on the gradient smoothness index, including the current gradient evaluation value and the evaluation value after the gradient change.

[0033] S313. Based on the dual evaluation values ​​of the center point of each fixed block, select the fixed blocks that meet the evaluation threshold; and based on the current gradient evaluation values ​​of the selected fixed blocks, use a weighted fusion method to calculate the image gradient of the original image data of the substation after preliminary processing.

[0034] S314. Based on the image gradient first visual saliency heatmap of the original image data of the substation after preliminary processing, construct the diffusion coefficient function of the first visual saliency heatmap.

[0035] Preferably, the step of inputting the enhanced image after diffusion processing into a semantic segmentation model combined with Laplacian for pixel-level classification to obtain the final semantic segmentation result includes the following steps:

[0036] S331. Input the enhanced image after diffusion processing into the encoder in the semantic segmentation model combined with Laplacian to obtain multi-scale feature maps with decreasing resolution but increasing number of channels.

[0037] S332. Apply the Laplacian convolution operator to the multi-scale feature map and enhance the edge features by calculating the second derivative to obtain the multi-scale feature map with enhanced edges.

[0038] S333: Use the decoder to perform upsampling and skip connections, fuse multi-scale feature maps and gradually restore the feature map resolution to obtain the fused feature map;

[0039] S334. The classifier is used to adjust the number of channels in the fused feature map, and the soft maximum function is combined to generate the final semantic segmentation result.

[0040] According to a second aspect of the present invention, a semantic segmentation and image enhancement system for complex backgrounds in substations is provided, the system comprising:

[0041] The image acquisition and processing module is used to collect raw image data of the substation using a drone equipped with a camera, and to preprocess the collected raw image data by combining heat map saliency analysis and spectral residual method based on reciprocal function to obtain hue, saturation and brightness images.

[0042] The image enhancement module is used to extract the luminance component from the hue, saturation, and luminance image, and to establish an end-to-end image enhancement model based on the IAT model; the luminance component is enhanced using the image enhancement model to obtain the enhanced image;

[0043] The semantic segmentation module is used to perform diffusion processing on the enhanced image based on the diffusion coefficient function, and combined with the Laplacian operator, it uses the semantic segmentation model to perform semantic segmentation processing on the diffused image to obtain the semantic segmentation result.

[0044] According to a third aspect of the present invention, an electronic device is provided, the electronic device comprising: one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the programs to implement the steps of the above-described method.

[0045] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, wherein a computer program is stored therein, wherein the steps of the above-described method are implemented when the computer program controls the device in which the computer-readable storage medium is located to execute during runtime.

[0046] The beneficial effects of this invention are as follows:

[0047] 1. This invention is based on the IAT image enhancement network and designs a mapping relationship between exposures. The network iterates pixel by pixel to learn the best fit between the original image and the enhanced image. Exposure loss, color loss and semantic loss are introduced during the model training process to avoid image distortion and over-enhancement caused by image enhancement, and to ensure the readability of the image, thereby providing a reliable data foundation for subsequent analysis.

[0048] 2. In order to ensure the interpretability of image details and key targets, this invention introduces Laplacian convolution, which, combined with edge information, helps the lightweight U-Net semantic segmentation network to locate targets of interest in complex environments. Furthermore, by calculating semantic loss, the robustness of the image enhancement model is further improved.

[0049] 3. This invention designs an image enhancement model to help UAVs perform inspection tasks under various lighting conditions in complex geographical and environmental environments. It performs subsequent processing based on high-quality images, which improves the level of automation and efficiency of the inspection process. Based on accurate inspection results, it can rationally allocate resources, prioritize the most urgent issues, and ensure that limited resources are used most effectively. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly described below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings:

[0051] Figure 1 This is a flowchart of a semantic segmentation and image enhancement method for complex backgrounds in substations according to an embodiment of the present invention;

[0052] Figure 2This is a logical flowchart of a semantic segmentation and image enhancement method for complex backgrounds in substations according to an embodiment of the present invention;

[0053] Figure 3 This is an algorithm structure diagram of a semantic segmentation and image enhancement method for complex backgrounds in substations according to an embodiment of the present invention.

[0054] Figure 4 This is a schematic diagram of the enhancement module structure in a semantic segmentation and image enhancement method for complex backgrounds in substations according to an embodiment of the present invention;

[0055] Figure 5 This is a schematic diagram of a semantic segmentation network in a semantic segmentation and image enhancement method for complex backgrounds in substations according to an embodiment of the present invention;

[0056] Figure 6 This is a block diagram of a semantic segmentation and image enhancement system for complex backgrounds in substations, according to an embodiment of the present invention.

[0057] Figure 7 This is a block diagram of the hardware structure of the host device in a semantic segmentation and image enhancement method for complex backgrounds in substations, according to an embodiment of the present invention.

[0058] In the picture:

[0059] 1. Image acquisition and processing module; 2. Image enhancement module; 3. Semantic segmentation module. Detailed Implementation

[0060] To enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this application.

[0061] The methods and embodiments provided in this application can be executed on a host device or a similar computing device. Taking running on a host device as an example, Figure 7 This is a hardware structure block diagram of a host device for a semantic segmentation and image enhancement method for complex backgrounds in substations, according to an embodiment of this application. As shown in Figure 7, the host device may include one or more ( Figure 7Only one is shown in the diagram. The processor (which may include, but is not limited to, a microprocessor (MCU) or programmable logic device (FPGA), etc.) and storage for storing data are also shown. The host device may further include transmission devices for communication functions and input / output devices. Those skilled in the art will understand that... Figure 7 The structure shown is for illustrative purposes only and does not limit the structure of the host device described above. For example, the host device may also include components that are larger than... Figure 7 The more or fewer components shown, or having the same Figure 7 The different configurations shown.

[0062] The memory can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the exception handling method in this embodiment. The processor executes various functional applications and data processing by running the computer program stored in the memory, thus implementing the above-described method. The memory may include high-speed random access memory (RAM) and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the host device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks (LANs), mobile communication networks, and combinations thereof.

[0063] Transmission devices are used to receive or send data over a network. Specific examples of the network described above may include a wireless network provided by a communication provider for the host device. In one example, the transmission device includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0064] According to embodiments of the present invention, a semantic segmentation and image enhancement method and system for complex backgrounds in substations are provided.

[0065] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments, such as... Figure 1-3 As shown, according to a first embodiment of the present invention, a semantic segmentation and image enhancement method for complex backgrounds in substations is provided, the method comprising the following steps:

[0066] S1. The drone with the camera on board collects the original image data of the substation, and combines the heat map significance analysis and the spectral residual method based on the reciprocal function to preprocess the collected original image data to obtain the hue, saturation and brightness image.

[0067] In a preferred embodiment, the process of using a drone equipped with a camera to collect raw image data of a substation, and then preprocessing the collected raw image data using heatmap saliency analysis and the spectral residual method based on the reciprocal function to obtain a hue, saturation, and brightness image includes the following steps:

[0068] S11. Perform preliminary processing on the raw image data of the substation, including image denoising, geometric correction, and resolution normalization.

[0069] Specifically, the image acquisition equipment carried by drones is used to collect comprehensive images of the substation. To ensure data diversity, the images cover different seasons, weather conditions (sunny, cloudy, rainy, foggy, etc.), lighting conditions (strong, inverted, dark, etc.), and the operating status and layout of various power equipment.

[0070] The acquired images undergo preprocessing operations such as denoising, normalization, and data augmentation. Median filtering is used to remove noise from the images, and appropriate cropping and scaling are performed to meet the input requirements of the model and improve its generalization ability. The RGB color space is converted to the HSV color space to separate color and brightness, facilitating subsequent image feature extraction and analysis.

[0071] S12. Based on the original image data of the substation after preliminary processing, perform salience detection based on the salience target detection model to generate a first visual salience heatmap;

[0072] In a preferred embodiment, the step of generating a first visual saliency heatmap by performing saliency detection based on a saliency target detection model using the pre-processed original image data of the substation includes the following steps:

[0073] S121. Based on the feedforward approach, the original image data of the substation after preliminary processing is processed using a salient target detection model to generate a salient heat map of the equipment.

[0074] S122. The original image data of the substation after preliminary processing is divided into salient and non-salient regions. Bilateral filtering is performed on the salient regions and non-local mean filtering is performed on the non-salient regions to obtain bilateral filtered output image and non-local mean filtered image.

[0075] S123. Using the device saliency heatmap as a weighted guide, and combining the bilateral filter output image and the non-local mean filter image, calculate the first visual saliency heatmap.

[0076] Specifically, given the complex nature of substations, the core focus of this invention is on power equipment. Therefore, in the preprocessing stage, a weighted preprocessing mechanism based on equipment saliency is introduced, as follows:

[0077] Integrate a lightweight salient object detection model (based on U) 2 -A simplified version of the Net network with pruning and channel reduction, it quickly processes the preprocessed input image I in a feedforward manner to generate a device saliency heatmap M of the same size as the input image. The value M(x,y)∈[0,1] of each pixel represents the probability that the location belongs to a salient power device region.

[0078] M = f Saliency (I;θ);

[0079] Among them, f Saliency Simplified U 2 -Net model, where θ represents its pre-trained parameters.

[0080] Saliency refers to the fact that certain regions in an image are more attractive to attention due to their visual attributes (color, texture, shape); the testing method is based on a deep learning model (U). 2 -Net performs feedforward computation and outputs a probability heatmap; the filtering operation is to apply different filtering algorithms (bilateral filtering and nonlocal mean filtering) to different spatial regions of the image according to the heatmap weights, and perform pixel-level fusion.

[0081] Using the heatmap M as a weighted guide, more conservative filtering parameters (bilateral filtering) are applied to the salient regions (devices) for denoising to preserve more edge and texture details, resulting in a bilaterally filtered output image I. bilateral The weight of the variable is relatively large; for non-saliency background regions, a more powerful filter (non-local mean filtering) can be used to remove complex background noise, and the output non-local mean filtered image I is obtained. NLM The weight of is relatively large. Ultimately, the denoised image I denoised It is formed by weighted fusion of the two based on the significance heatmap:

[0082] I denoised =M·I bilateral +(1-M)·I NLM ;

[0083] Guided by the saliency heatmap, priority is given to enhancing salient regions, such as randomly erasing or copying and pasting only the device region, so that the model can focus more on learning the features of the device.

[0084] S13. Based on the original image data of the substation after preliminary processing, perform multi-feature saliency detection using the spectral residual method based on the reciprocal function to generate a second visual saliency heatmap.

[0085] In a preferred embodiment, the step of performing multi-feature saliency detection based on the spectral residual method using the reciprocal function to generate a second visual saliency heatmap, according to the pre-processed original image data of the substation, includes the following steps:

[0086] S131. Using the Fourier transform method, the pre-processed original substation image data is transformed from the spatial domain to the frequency domain to obtain a frequency domain image.

[0087] It's important to note that the Fourier transform is a mathematical tool that converts a signal from the time domain (or spatial domain for images) to the frequency domain. In image processing, the spatial domain represents information such as the location and grayscale value of pixels in an image, while the frequency domain reflects the distribution of different frequency components in the image. High-frequency components in an image typically correspond to rapidly changing parts such as edges, details, and noise, while low-frequency components correspond to the overall outline and slowly changing parts of the image.

[0088] A two-dimensional discrete Fourier transform is performed on the pre-processed raw image data of the substation. This transform converts the pixel information of the image in the spatial domain into a complex form in the frequency domain, where the modulus of the complex number represents the amplitude of the frequency component, and the phase represents the phase information of the frequency component. After the transform, a frequency domain image is obtained, which describes the features of the original image in a different way.

[0089] S132. Perform a reciprocal function transformation on the frequency domain image, and calculate the spectral residual spectrum based on the transformation result; extract the saliency response maps of color features, texture features and shape features according to the spectral residual spectrum, and fuse them to obtain the fused saliency response map;

[0090] It should be noted that the reciprocal function transform is a nonlinear transform, and applying the reciprocal function transform to a frequency domain image can change the weights of different frequency components in the frequency domain.

[0091] The frequency domain logarithmic amplitude spectrum of natural images (including substation backgrounds) exhibits a 1 / f pattern (low-frequency components are high intensity and smooth, representing redundant background; high-frequency components are low intensity and abrupt, representing significant equipment areas). The spectral residual spectrum is the logarithmic amplitude spectrum minus the average logarithmic amplitude spectrum, and the formula is:

[0092] R(u,v)=L(u,v)-A avg (u,v);

[0093] Among them: A avg(u,v) represents the average logarithmic amplitude spectrum, which is obtained by smoothing the logarithmic amplitude spectrum L(u,v) of the frequency domain image through a 5×5 mean filter (simulating the low-frequency redundancy distribution of the background); R(u,v) represents the spectral residual spectrum. The spectral residual spectrum R(u,v) value of the high-frequency region (device) is significantly higher than that of the low-frequency region (background), thus achieving background suppression and device highlighting.

[0094] Furthermore, by analyzing the color-related frequency components in the spectral residual spectrum, a saliency response map of color features can be extracted. In the frequency domain, texture information is typically associated with specific high-frequency components. By analyzing the high-frequency components in the spectral residual spectrum, a saliency response map of texture features can be extracted. In the frequency domain, shape information can be extracted by analyzing the frequency components corresponding to the edges and contours of the image. Using the frequency information related to edges and contours in the spectral residual spectrum, a saliency response map of shape features can be generated. A weighted fusion method is used, assigning different weights to the saliency response maps of color features, texture features, and shape features according to the importance of different features in saliency detection, and then summing them to obtain the fused saliency response map.

[0095] S133. Using a normalization function, the fused saliency response map is normalized to generate a second visual saliency heatmap.

[0096] It's important to note that normalization is the process of mapping data to a specific range (usually [0,1]). In image processing, commonly used normalization functions include linear normalization. Applying a normalization function to the fused saliency response map can adjust the pixel values ​​in the response map to the range of [0,1]. This makes the saliency values ​​between different images comparable and allows for a more intuitive display of the intensity of salient regions.

[0097] S14. The first visual saliency heatmap and the second visual saliency heatmap are fused to obtain the fused visual saliency heatmap.

[0098] Specifically, the first and second visual saliency heatmaps are generated using different methods or based on different features, each capturing different saliency information in the image. For the fusion of the first and second visual saliency heatmaps, a clustering algorithm is used to segment the heatmap into several regions with similar saliency features. Then, for each region, different fusion strategies are adopted based on its saliency performance in the two heatmaps. For example, if the saliency of a region is significantly higher in the first heatmap than in the second, the fusion process emphasizes the information from the first heatmap; otherwise, it emphasizes the information from the second heatmap. Finally, the fused regions are combined into a complete fused heatmap.

[0099] S15. Perform threshold segmentation on the fused visual saliency heatmap to generate a binary mask for the device region;

[0100] Specifically, the basic idea of ​​threshold segmentation is to divide an image into foreground and background based on the comparison between the grayscale values ​​of pixels in the image and a preset threshold. In the fused visual saliency heatmap, salient regions typically have higher pixel values, while non-salient regions (background) have lower pixel values. By selecting an appropriate threshold, salient regions (potentially power plant equipment areas) can be separated from the background. This results in a binary image where white areas (pixel values ​​of 1) represent equipment areas and black areas (pixel values ​​of 0) represent background areas. This binary image serves as the binary mask for the equipment areas.

[0101] S16. Convert the raw image data of the substation after preliminary processing to the HSV color space, and enhance the HSV channel using a binary mask to obtain an optimized hue, saturation and brightness image.

[0102] Specifically, the HSV (Hue, Saturation, Value) color space is a color representation method that better aligns with human visual perception. Compared to the RGB color space, HSV separates the three attributes of color—hue, saturation, and value—making it easier to process and analyze color information in an image independently. In substation images, the color characteristics of equipment are crucial for identification and classification. Converting to the HSV color space allows for more effective extraction and enhancement of equipment color information. After enhancing the three HSV channels, the enhanced H, S, and V channel values ​​are recombine to form an HSV image. This HSV image is then converted back to the RGB color space, resulting in an optimized hue, saturation, and value image.

[0103] S2. Extract the luminance component from the hue, saturation, and luminance image, and establish an end-to-end image enhancement model based on the IAT model; use the image enhancement model to enhance the luminance component to obtain the enhanced image;

[0104] Specifically, to adjust the dynamic range of the image, based on the obtained HSV image, the V channel (luminance) in HSV is essentially a form of grayscale image. Its luminance component (V channel) is taken as the grayscale image required in this step. Without considering hue and saturation, a pixel-wise image mapping curvature is designed to act on this luminance component, and a deep learning network is used to learn the transformation relationship of the curvature parameters. Let matrix A be the curvature map of the same size as the image:

[0105] EI n (x,y)=EI n-1 (x,y)+A n (x,y)EIn-1 (x,y)(1-EI n-1 (x,y));

[0106] In the formula, EI n (x,y) represents the enhanced output image, (x,y) represents the image coordinates, and n represents the number of network training iterations.

[0107] In order to better fit the mapping relationship between the enhanced image and the input image, an end-to-end Transformer model is established based on the IAT model (Illimination-Adaptiv-Transformer) and the image-related parameters are adjusted through a dynamic query learning mechanism to enhance the accuracy of visual adaptive inspection tasks of power equipment under poor lighting conditions.

[0108] like Figure 4 As shown, the IAT network contains two branches. The first branch predicts the per-pixel curvature map and additive map by two independent branches. The second branch combines a query learning mechanism, and as the algorithm iterates, it continuously evaluates the color matrix and gamma value of global information through a self-attention mechanism.

[0109] EI n (x,y)=G(I(x,y));

[0110] Among them, EI n (x,y) represents the enhanced output given the input; G(·) represents the key parameter adjustment function, including the color matrix, gamma, and curvature map;

[0111]

[0112] In the formula, Z represents the color matrix, Z represents the additive graph, and κ represents the gamma value.

[0113] Luminance component EI to be enhanced n After the (x,y) values ​​are calculated, they are combined with the retained hue (H) and saturation (S) components and converted back to the RGB color space to obtain the final enhanced image.

[0114] S3. The enhanced image is diffused based on the diffusion coefficient function, and then the semantic segmentation model is used to perform semantic segmentation on the diffused image to obtain the semantic segmentation result.

[0115] In a preferred embodiment, the step of performing diffusion processing on the enhanced image based on the diffusion coefficient function, and then combining it with the Laplacian operator to perform semantic segmentation processing on the diffused image using a semantic segmentation model to obtain the semantic segmentation result includes the following steps:

[0116] S31. Based on the dual evaluation method, image gradient calculation is performed on the raw image data of the substation after preliminary processing, and diffusion coefficient function is constructed by combining the first visual saliency heatmap.

[0117] In a preferred embodiment, the step of calculating the image gradient of the pre-processed original image data of the substation based on the dual evaluation method and constructing the diffusion coefficient function in combination with the first visual saliency heatmap includes the following steps:

[0118] S311. Divide the pre-processed original image data of the substation into several fixed blocks of a preset size;

[0119] It should be noted that a uniform division method can be used to segment the image according to a preset number of rows and columns.

[0120] S312. Calculate the gradient smoothness index for each fixed block in the horizontal, vertical, diagonal, and anti-diagonal directions; and calculate the dual evaluation value of the center point of each block based on the gradient smoothness index, including the current gradient evaluation value and the evaluation value after the gradient change.

[0121] Specifically, the gradient reflects the rate of change of pixel values ​​in an image. By calculating gradients in different directions, edge and texture information of the image in that direction can be captured. For each fixed block, the gradient in the horizontal direction (G) is calculated separately. x ), vertical direction (G) y ), diagonal direction (G) 45 ) and the opposite diagonal direction (G 135 The gradient of a vector is denoted as . Common gradient operators such as the Sobel operator and the Prewitt operator can be used to calculate it. Taking the Sobel operator as an example, the Sobel operator templates for the horizontal and vertical directions are as follows:

[0122]

[0123] The Sobel operator template is multiplied element-wise by the pixel matrix of a 3×3 fixed block, and the summation yields the gradient values ​​in the horizontal and vertical directions. The calculation formula is as follows:

[0124]

[0125] In the formula, i and j represent operator template indices, and G x (i+1,j+1) represents the weight of the (i+1)th row and (j+1)th column of the operator template.

[0126] Furthermore, the curved edges of substation equipment (such as GIS flanges and cable bends) cannot be fully captured by horizontal / vertical gradients and require the use of G... x With G yDerivation of the diagonal gradient using linear combinations:

[0127] Diagonal direction (from top left to bottom right, 45°): The grayscale change in this direction is the superposition of horizontal to the right and vertical downward, and the gradient formula is:

[0128]

[0129] In the opposite diagonal direction (from top right to bottom left, 135°): the grayscale change in this direction is the superposition of horizontal leftward and vertical downward changes, and the gradient formula is:

[0130]

[0131] In the formula, -Gx represents the gradient to the left, corresponding to the direction from the upper right to the lower left.

[0132] The gradient smoothness index measures the uniformity of gradient changes within a fixed block. A common calculation method is to calculate the variance of the gradient magnitude.

[0133] Based on the calculated gradient smoothness indices for the four directions, and taking into account the importance of each direction, the current gradient evaluation value is calculated.

[0134] Furthermore, to more comprehensively evaluate image gradients, the impact of gradient changes on the image should be considered. This can be achieved by transforming the pixel values ​​within a fixed block (e.g., adding noise, applying local blurring), and then recalculating the gradient smoothness index and the current gradient evaluation value of the transformed image to obtain the evaluation value after the gradient change.

[0135] S313. Based on the dual evaluation values ​​of the center point of each fixed block, select the fixed blocks that meet the evaluation threshold; and based on the current gradient evaluation values ​​of the selected fixed blocks, use a weighted fusion method to calculate the image gradient of the original image data of the substation after preliminary processing.

[0136] S314. Based on the image gradient first visual saliency heatmap of the original image data of the substation after preliminary processing, construct the diffusion coefficient function of the first visual saliency heatmap.

[0137] Specifically, in order to overcome the inherent defect of the Laplacian operator being sensitive to noise and to suppress background interference while enhancing edges, this invention introduces a saliency-guided anisotropic diffusion filter as a preprocessing step before performing Laplacian convolution.

[0138] The filtering process is based on partial differential equations, and its diffusion behavior is jointly controlled by the image gradient and the first visual saliency heatmap M. The diffusion coefficient function is defined. as follows:

[0139]

[0140] In the formula, the threshold function k(M) is dynamically adjusted by the device saliency heatmap:

[0141] k(M)=k min +(k max -k min )·(1-M);

[0142] Where u represents the diffusion coefficient, a function of the image gradient, and is key to controlling smoothing behavior; ▽I represents the image gradient; M is the first visual saliency heatmap; k min k max This represents the preset constant threshold.

[0143] The above design allows for the use of a smaller threshold k in the highly saliency device region (M→1). min This preserves all subtle gradient changes; in the low-significance background region (M→0), a larger threshold k is used. max Only the strongest edges are retained, and noise is smoothed. The feature map after this filtering process is then fed into a Laplacian convolutional layer for edge feature enhancement, thereby significantly improving the accuracy and robustness of edge extraction.

[0144] S32. Apply the diffusion coefficient function to the enhanced image to obtain the enhanced image after diffusion processing;

[0145] S33. Input the enhanced image after diffusion processing into a semantic segmentation model combined with Laplacian for pixel-level classification to obtain the final semantic segmentation result.

[0146] In a preferred embodiment, the step of inputting the enhanced image after diffusion processing into a semantic segmentation model combined with Laplacian for pixel-level classification to obtain the final semantic segmentation result includes the following steps:

[0147] S331. Input the enhanced image after diffusion processing into the encoder in the semantic segmentation model combined with Laplacian to obtain multi-scale feature maps with decreasing resolution but increasing number of channels.

[0148] S332. Apply the Laplacian convolution operator to the multi-scale feature map and enhance the edge features by calculating the second derivative to obtain the multi-scale feature map with enhanced edges.

[0149] S333: Use the decoder to perform upsampling and skip connections, fuse multi-scale feature maps and gradually restore the feature map resolution to obtain the fused feature map;

[0150] S334. The classifier is used to adjust the number of channels in the fused feature map, and the soft maximum function is combined to generate the final semantic segmentation result.

[0151] Specifically, the enhanced image is input into a lightweight U-Net network for semantic segmentation. The network automatically performs multi-level feature extraction and reconstruction through an encoder-decoder structure. The specific process is as follows:

[0152] The enhanced image first enters the encoder, which consists of multiple stages. Each stage contains two consecutive depthwise separable convolutional operations (replacing standard convolutions to reduce computation and parameter count, achieving lightweight processing) and a max-pooling operation. The convolutional operations in each stage extract features at that resolution, while the max-pooling operation halves the feature map size, achieving dimensionality reduction and expanding the receptive field of subsequent convolutions. This generates a series of feature maps with decreasing resolution but increasing channel count, enabling the network to capture more global contextual information.

[0153] To enhance the network's ability to perceive critical edges of power equipment, a Laplacian convolutional layer is added after the shallow network output of the encoder path.

[0154] Let P be the feature map obtained from the shallowest layer of the encoder, and L be the Laplacian filter function. filter Therefore, the edge-enhanced feature map after filtering is represented as:

[0155] Out = L filter (P);

[0156] That is, each pixel in the image uses a 3×3 convolution kernel to perform convolution calculation with the feature map, and the output value of each pixel is:

[0157]

[0158] In the formula, Out(x,y) represents the feature map for edge enhancement, K represents the 3×3 Laplacian convolution kernel, i and j represent indices, P represents the shallowest feature map, and (x,y) represents the coordinate position of the image pixel.

[0159] Then, in the decoder path, the low-resolution feature map obtained from the bottom of the encoder, which contains high-level semantic information (the output of the last layer of the U-Net encoder), is restored to a high-resolution layer layer by layer via a top-down path, thereby performing spatially aware semantic segmentation. The specific process is as follows:

[0160] The low-resolution feature map output from the encoder bottleneck layer, containing rich high-level semantic information, is used as the starting point of the decoder. A bilinear upsampling operation is used to double its resolution, resulting in a pre-amplified feature map. This current feature map is then fused with corresponding feature maps of the same resolution in the decoder path via skip connections. Specifically, for example... Figure 5As shown, the decoder path first performs bilinear upsampling on the low-resolution feature map (after Laplacian edge enhancement) from the encoder bottleneck layer, doubling its spatial resolution. Then, it selects a corresponding feature map from the encoder path with the same spatial resolution as the current upsampled feature map. The feature maps from different paths are then concatenated along the channel dimension. This process is repeated twice until the spatial size of the feature map is restored to the same as the original input image. This mechanism organically combines high-level semantics, low-level details, and enhanced edge information.

[0161] Here, starting with the smallest feature sampled by the encoder (the feature map here has undergone edge enhancement processing), it is upsampled to double the resolution f1, and then concatenated with the second smallest feature f2 of the encoder along the channel dimension. like Figure 5 As shown, the process is executed sequentially from top to bottom until the image is restored to the same size as the input image.

[0162] The fused feature map undergoes dimensionality reduction and feature reshaping through multiple convolutional layers to generate a new feature map. This new feature map is then subjected to the same operations twice; that is, after multiple upsampling and fusion processes, the spatial dimensions of the resulting feature map are restored to the same resolution as the input image.

[0163] A 1×1 convolutional layer is used as the final classifier. The number of channels in the feature map is adjusted to match the number of target classes. Finally, a softmax function is applied to all channel values ​​at each pixel location to calculate the probability of each pixel belonging to each class. The class with the highest probability is used as the predicted label for that pixel, thereby generating the final spatially aware semantic segmentation map.

[0164] A 1×1 convolutional layer generates a raw score for each class for each pixel, and then the scores are converted into a probability distribution using a softmax function:

[0165]

[0166] In the formula, p x,y,c p represents the probability that pixel position (x, y) belongs to class c. x,y,c This represents the original predicted score of the pixel in category c. class This represents the total number of categories. k represents the category index, and Exp(·) represents the natural exponential function.

[0167] Finally, the predicted category for each pixel is determined through the argmax operation:

[0168]

[0169] in, This represents the final predicted label for pixel (x, y).

[0170] Furthermore, for the image enhancement model and semantic segmentation model in this invention, a multi-stage alternating training strategy is adopted to optimize the performance of the image enhancement model and semantic segmentation model. The specific process is as follows:

[0171] (a) Pre-training phase: Image enhancement model and semantic segmentation model are pre-trained independently to provide good initial parameters for alternating training;

[0172] (b) The alternating optimization phase specifically includes:

[0173] (1) Fix the parameters of the image enhancement model and use its output image to train the semantic segmentation model;

[0174] When fixing the parameters of the image enhancement model and updating the parameters of the semantic segmentation model, this invention introduces a dynamic weight adjustment mechanism based on uncertainty to precisely control the contribution of the semantic segmentation loss to the training of the enhancement model:

[0175] First, based on the probability distribution P output by the semantic segmentation model, the prediction uncertainty U(x,y) at each pixel location is calculated using information entropy:

[0176]

[0177] In the formula, the loss weight λ(x,y) for each pixel is then dynamically calculated based on the uncertainty, ensuring that the weight is significantly reduced in regions of high uncertainty:

[0178] λ(x,y)=λ base ·exp(-γ·U(x,y));

[0179] In the formula, λ base γ represents an adjustable hyperparameter.

[0180] Ultimately, the total loss function of the image enhancement model is the sum of the image reconstruction loss and the weighted semantic segmentation loss:

[0181]

[0182] In the formula, y x,y,c p represents the true label value of pixel (x, y) in category c; x,y,c λ(x,y) represents the probability that pixel (x,y) belongs to category c after the enhanced image is predicted by the semantic segmentation model; C represents the total number of categories; λ(x,y) represents the dynamic weight coefficient calculated based on the prediction uncertainty.

[0183] To further improve the consistency between the segmentation results and the physical shape of the power equipment, this invention introduces a shape-constraint-based regularization term into the loss function. Since power equipment (insulators, transformers, clamps, etc.) has specific, regular geometric shapes (circular, cylindrical, rectangular), it encourages the edges of the predicted segmentation map to maintain consistency with the edges of the true labels in terms of gradient distribution. By calculating the mean square error between the binary contour maps of the predicted segmentation map and the true labels, the model is forced to learn and produce segmentation results with more reasonable geometric shapes.

[0184]

[0185] In the formula, L shape This represents the shape constraint loss, where N represents the number of samples during training. p represents the model's predicted probability. x,y The predicted contour binary image obtained after Argmax and edge extraction, where Z represents the true semantic label y. x,y The true contour binary image obtained after edge extraction.

[0186] Among them, the probability graph p output by the semantic segmentation model x,y Perform the Argmax operation along the category dimension to obtain the predicted category index for each pixel, thus obtaining the predicted label map. C represents the category index:

[0187]

[0188] In addition, the predicted label map will be Compared with the real label image y x,y In this binary mask, all non-background category pixels (pixels with values ​​greater than 0) are set to 1, and the background category pixels (pixels with values ​​equal to 0) are set to 0, resulting in a binary mask. and Using the same gradient Sobel operator The gradient magnitudes of the binary mask are calculated separately, and the contours are extracted by thresholding τ, ultimately yielding the predicted contour binary image. And the true contour binary image Z i 1(·) is used for binarization, converting pixels that are true (||·||>τ) to 1 and pixels that are false to 0.

[0189]

[0190] Then, the shape constraint loss L shape That is and Z i The mean square error between them.

[0191] In summary, the total loss function is L1:

[0192] L1=β1L ce-weighted +β2L shape ;

[0193] In the formula, β1 and β2 are hyperparameters.

[0194] The above mechanism can effectively suppress the interference of unreliable prediction signals, making the optimization process of the augmentation model more robust and efficient, and ultimately obtaining an augmentation model with significantly improved performance.

[0195] (2) Fix the semantic segmentation model parameters and feed the output semantic segmentation loss back to the image enhancement model. This loss, along with image exposure loss, constitutes the total loss, and the image enhancement model parameters are updated accordingly. When fixing the semantic segmentation model parameters and updating the image enhancement model parameters, a multi-task loss function is used for updating.

[0196] To ensure consistency in exposure, color, semantic information, and reconstruction loss in the enhanced image, the following loss functions are considered for evaluating and training the enhancement model:

[0197] The semantic loss function adopts the improved weighted cross-entropy loss function described above:

[0198]

[0199] In the formula, L ce-weighted Represents cross-entropy loss; y x,y,c p represents the true label value of pixel (x, y) in category c; x,y,c To enhance the probability that a pixel (x,y) belongs to category c after the image is predicted by the semantic segmentation model; C is the total number of categories; λ(x,y) is a dynamic weight coefficient calculated based on prediction uncertainty.

[0200] To limit underexposure and overexposure in an image, the difference between the average brightness of pixels in the target region and the preset exposure level is defined as the exposure loss L. exp :

[0201]

[0202] In the formula, D represents the total number of pixels in the target region, and Y... m E represents the measured pixel brightness, and E represents the ideal preset exposure expectation.

[0203] It should be noted that, in order to evaluate the consistency between the colors of the enhanced image and the original image, and to reduce the distortion of black objects caused by brightness enhancement, the hue (H) and saturation (S) of the original image are separated and their color gamuts are compared with those of the enhanced image in HSV format. The difference is calculated using a loss function L. HSV Methods for participating in model training:

[0204]

[0205] In the formula, H represents enhancing the hue and saturation of an image. h,w S h,w This represents the hue and saturation of the original image.

[0206] The original input image itself is used as the target for calculating the reconstruction loss, which constrains the enhancement model's ability to preserve image content and prevents excessive distortion.

[0207]

[0208] In the formula, L MSE I represents the reconstruction loss. enhanced I represents the enhanced image, and I represents the input image.

[0209] Simultaneously, by incorporating prior constraints such as exposure loss and color loss, and ultimately using weighted semantic segmentation loss as the core guide, a task-oriented image enhancement method that does not require paired data is achieved:

[0210] L2=α1L ce-weighted +α2L exp +α3L hsv +α4L MSE ;

[0211] In the formula, α1, α2, α3 and α4 are weighting coefficients, respectively.

[0212] (3) Iteratively execute steps (1) and (2) to optimize the two models together.

[0213] This strategy achieves task-oriented adaptive image enhancement by backpropagating the loss of the semantic segmentation task to the image enhancement module, which significantly improves the segmentation accuracy and robustness of the entire system.

[0214] By continuously adjusting the diversity of model input data and optimizing model parameters, the generalization ability of the model in processing image enhancement in complex environments can be further improved.

[0215] In addition, before training, both the image enhancement model and the semantic segmentation model need to collect and label sample image data. The labeling accurately identifies different categories of electrical equipment, such as transformers, cranes, and human behavior. The labeled image data is then divided into training, validation, and test sets. These sets are randomly allocated according to a preset ratio: 70% for training, 20% for validation, and 10% for testing. During training, the image enhancement model is first trained using the training set, and hyperparameters are adjusted using the validation set. After training, the image enhancement effect is evaluated using the test set. During iteration, the images processed by the enhancement model are used as input to the semantic segmentation model. Similarly, the model is trained using the training set, hyperparameters are tuned using the validation set, and segmentation accuracy is evaluated using the test set.

[0216] like Figure 6 As shown, according to a second embodiment of the present invention, a semantic segmentation and image enhancement system for complex backgrounds in substations is provided, the system comprising:

[0217] Image acquisition and processing module 1 is used to acquire raw image data of the substation using a drone equipped with a camera, and to preprocess the acquired raw image data by combining heat map saliency analysis and spectral residual method based on reciprocal function to obtain hue, saturation and brightness images.

[0218] Image enhancement module 2 is used to extract the luminance component from the hue, saturation, and luminance image, and to establish an end-to-end image enhancement model based on the IAT model; the luminance component is enhanced using the image enhancement model to obtain an enhanced image;

[0219] Semantic segmentation module 3 is used to perform diffusion processing on the enhanced image based on the diffusion coefficient function, and combined with the Laplacian operator, to perform semantic segmentation processing on the diffused image using the semantic segmentation model to obtain the semantic segmentation result.

[0220] Specifically, this semantic segmentation and image enhancement system, designed for complex substation environments, is applied in daily substation inspections to achieve rapid and accurate monitoring and analysis of substation equipment. During drone inspections, image quality is monitored in real time, and the image enhancement module is activated to optimize images damaged by insufficient lighting or overexposure. A user-friendly interface is designed to facilitate operation and viewing of processing results by inspection personnel. Data and feedback from practical applications are continuously collected to further improve and refine the system's performance and functionality.

[0221] According to a third embodiment of the present invention, an electronic device is provided, the electronic device comprising: one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to perform the steps in any of the above method embodiments.

[0222] According to a fourth embodiment of the present invention, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, wherein the computer program, when running, controls the device where the computer-readable storage medium is located to perform the steps in any of the above method embodiments.

[0223] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0224] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, etc.) containing computer-usable program code.

[0225] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A semantic segmentation and image enhancement method for complex backgrounds in substations, characterized in that, The method includes the following steps: S1. The drone with the camera on board collects the original image data of the substation, and combines the heat map significance analysis and the spectral residual method based on the reciprocal function to preprocess the collected original image data to obtain the hue, saturation and brightness image. S2. Extract the luminance component from the hue, saturation, and luminance image, and establish an end-to-end image enhancement model based on the IAT model; use the image enhancement model to enhance the luminance component to obtain the enhanced image; S3. The enhanced image is diffused based on the diffusion coefficient function, and then the semantic segmentation model is used to perform semantic segmentation on the diffused image to obtain the semantic segmentation result.

2. The semantic segmentation and image enhancement method for complex backgrounds in substations according to claim 1, characterized in that, The process of using a drone equipped with a camera to collect raw image data of a substation, and then preprocessing the collected raw image data using heatmap saliency analysis and the spectral residual method based on the reciprocal function to obtain a hue, saturation, and brightness image includes the following steps: S11. Perform preliminary processing on the raw image data of the substation, including image denoising, geometric correction, and resolution normalization. S12. Based on the original image data of the substation after preliminary processing, perform salience detection based on the salience target detection model to generate a first visual salience heatmap; S13. Based on the original image data of the substation after preliminary processing, perform multi-feature saliency detection using the spectral residual method based on the reciprocal function to generate a second visual saliency heatmap. S14. The first visual saliency heatmap and the second visual saliency heatmap are fused to obtain the fused visual saliency heatmap. S15. Perform threshold segmentation on the fused visual saliency heatmap to generate a binary mask for the device region. S16. Convert the raw image data of the substation after preliminary processing to the HSV color space, and enhance the HSV channel using a binary mask to obtain an optimized hue, saturation and brightness image.

3. The semantic segmentation and image enhancement method for complex backgrounds in substations according to claim 2, characterized in that, The step of generating a first visual saliency heatmap by performing saliency detection based on a saliency target detection model using the pre-processed original image data of the substation includes the following steps: S121. Based on the feedforward approach, the original image data of the substation after preliminary processing is processed using a salient target detection model to generate a salient heat map of the equipment. S122. The original image data of the substation after preliminary processing is divided into salient and non-salient regions. Bilateral filtering is performed on the salient regions and non-local mean filtering is performed on the non-salient regions to obtain bilateral filtered output image and non-local mean filtered image. S123. Using the device saliency heatmap as a weighted guide, and combining the bilateral filter output image and the non-local mean filter image, calculate the first visual saliency heatmap.

4. The semantic segmentation and image enhancement method for complex backgrounds in substations according to claim 2, characterized in that, The process of generating a second visual saliency heatmap by performing multi-feature saliency detection based on the spectral residual method using the reciprocal function of the original image data of the substation after preliminary processing includes the following steps: S131. Using the Fourier transform method, the pre-processed original substation image data is transformed from the spatial domain to the frequency domain to obtain a frequency domain image. S132. Perform a reciprocal function transformation on the frequency domain image, and calculate the spectral residual spectrum based on the transformation result; extract the saliency response maps of color features, texture features and shape features according to the spectral residual spectrum, and fuse them to obtain the fused saliency response map; S133. Using a normalization function, the fused saliency response map is normalized to generate a second visual saliency heatmap.

5. A semantic segmentation and image enhancement method for complex backgrounds in substations according to claim 2, characterized in that, The process of performing diffusion processing on the enhanced image based on the diffusion coefficient function, and then combining it with the Laplacian operator to perform semantic segmentation processing on the diffused image using a semantic segmentation model to obtain the semantic segmentation result includes the following steps: S31. Based on the dual evaluation method, image gradient calculation is performed on the raw image data of the substation after preliminary processing, and diffusion coefficient function is constructed by combining the first visual saliency heatmap. S32. Apply the diffusion coefficient function to the enhanced image to obtain the enhanced image after diffusion processing; S33. Input the enhanced image after diffusion processing into a semantic segmentation model combined with Laplacian for pixel-level classification to obtain the final semantic segmentation result.

6. The semantic segmentation and image enhancement method for complex backgrounds in substations according to claim 5, characterized in that, The process of calculating image gradients from the pre-processed raw image data of the substation based on the dual evaluation method, and constructing a diffusion coefficient function by combining the first visual saliency heatmap, includes the following steps: S311. Divide the pre-processed original image data of the substation into several fixed blocks of a preset size; S312. Calculate the gradient smoothness index for each fixed block in the horizontal, vertical, diagonal, and anti-diagonal directions; and calculate the dual evaluation value of the center point of each block based on the gradient smoothness index, including the current gradient evaluation value and the evaluation value after the gradient change. S313. Based on the dual evaluation values ​​of the center point of each fixed block, select the fixed blocks that meet the evaluation threshold; and based on the current gradient evaluation values ​​of the selected fixed blocks, use a weighted fusion method to calculate the image gradient of the original image data of the substation after preliminary processing. S314. Based on the image gradient first visual saliency heatmap of the original image data of the substation after preliminary processing, construct the diffusion coefficient function of the first visual saliency heatmap.

7. A semantic segmentation and image enhancement method for complex backgrounds in substations according to claim 5, characterized in that, The process of inputting the enhanced image after diffusion processing into a semantic segmentation model incorporating Laplacian for pixel-level classification to obtain the final semantic segmentation result includes the following steps: S331. Input the enhanced image after diffusion processing into the encoder in the semantic segmentation model combined with Laplacian to obtain multi-scale feature maps with decreasing resolution but increasing number of channels. S332. Apply the Laplacian convolution operator to the multi-scale feature map and enhance the edge features by calculating the second derivative to obtain the multi-scale feature map with enhanced edges. S333: Use the decoder to perform upsampling and skip connections, fuse multi-scale feature maps and gradually restore the feature map resolution to obtain the fused feature map; S334. The classifier is used to adjust the number of channels in the fused feature map, and the soft maximum function is combined to generate the final semantic segmentation result.

8. A semantic segmentation and image enhancement system for complex backgrounds in substations, used to implement the semantic segmentation and image enhancement method for complex backgrounds in substations as described in any one of claims 1-7, characterized in that, The system includes: The image acquisition and processing module is used to collect raw image data of the substation using a drone equipped with a camera, and to preprocess the collected raw image data by combining heat map saliency analysis and spectral residual method based on reciprocal function to obtain hue, saturation and brightness images. The image enhancement module is used to extract the luminance component from the hue, saturation, and luminance image, and to establish an end-to-end image enhancement model based on the IAT model; the luminance component is enhanced using the image enhancement model to obtain the enhanced image; The semantic segmentation module is used to perform diffusion processing on the enhanced image based on the diffusion coefficient function, and combined with the Laplacian operator, it uses the semantic segmentation model to perform semantic segmentation processing on the diffused image to obtain the semantic segmentation result.

9. An electronic device, characterized in that, The electronic device includes: one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to perform the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the steps of the method according to any one of claims 1 to 7 are implemented when the computer program controls the device containing the computer-readable storage medium to execute during runtime.

Images