Image enhancement method and device, electronic equipment and computer readable medium

Abstract

The present disclosure relates to an image enhancement method and device for power equipment, electronic equipment and computer readable medium. The method comprises: acquiring a real-time equipment image of the power equipment; calculating the exposure degree of the real-time equipment image to determine the number of enhancement iterations; determining a target image enhancement model from a plurality of image enhancement models based on the number of enhancement iterations, the target image enhancement model being generated by a trained machine learning model and an adjustment function; and inputting the real-time equipment image into the target image enhancement model to generate an enhanced equipment image. The image enhancement method and device for power equipment, electronic equipment and computer readable medium according to the present disclosure can effectively enhance the image brightness of various power equipment, the equipment image obtained by the present solution is clearer and easier to identify, and the speed and accuracy of edge computing can be improved.

Description

Technical Field

[0001] This disclosure relates to the field of image enhancement, and more specifically, to an image enhancement method, apparatus, electronic device, and computer-readable medium for power equipment. Background Technology

[0002] The operation and maintenance of power equipment in power systems requires extremely high real-time performance and accuracy. Different types of equipment problems require specialized knowledge to handle. Currently, most power equipment inspections utilize edge services, which can significantly reduce the workload of maintenance personnel and lower the daily operation and maintenance costs of power equipment. Edge services can acquire images of power equipment in real time and analyze their status. However, power inspection equipment is often affected by factors such as weather and equipment diversity, making it difficult to effectively use the equipment images acquired by power IoT edge services for power equipment maintenance. Considering that poor-quality images are received by edge services, it is necessary to improve the quality of image data before processing and analysis for intelligent power equipment inspection. In image-based power equipment status analysis, image enhancement is an effective method to improve image quality. However, the following reasons often lead to low brightness in power equipment images. First, the height of various devices is inconsistent, causing mutual occlusion. Equipment mounted in the air can easily cast shadows on the equipment below it, resulting in localized shadows in the image. Secondly, cloudy or foggy days will result in low ambient brightness, and if the intelligent inspection equipment cannot be fully exposed, the overall image of outdoor instruments and equipment will be dark. Finally, the intelligent inspection technology itself also has certain shortcomings, such as the lack of clear regulations on the location of inspection equipment, making it difficult for the inspection robot to find a shooting position that can be fully exposed.

[0003] Current low-brightness image enhancement methods are inevitably susceptible to noise interference or may result in significant color distortion after enhancement. Therefore, there is a need for new image enhancement methods, devices, electronic equipment, and computer-readable media for power equipment.

[0004] The information disclosed in the background section is only intended to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] In view of this, the present disclosure provides an image enhancement method, apparatus, electronic device and computer-readable medium for power equipment, which can effectively enhance the image brightness of various power equipment. The equipment images obtained by the present solution are clearer and easier to identify, and can also improve the speed and accuracy of edge computing.

[0006] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part from practice of this disclosure.

[0007] According to one aspect of this disclosure, an image enhancement method for power equipment is proposed, the method comprising: acquiring a real-time image of the power equipment; calculating the exposure level of the real-time image to determine the number of enhancement iterations; determining a target image enhancement model from a plurality of image enhancement models based on the number of enhancement iterations, the target image enhancement model being generated by a trained machine learning model and an adjustment function; and inputting the real-time image of the power equipment into the target image enhancement model to generate an enhanced image of the equipment.

[0008] In one exemplary embodiment of this disclosure, the method further includes: acquiring multiple image sets of power equipment; and training and generating multiple image enhancement models based on the multiple image sets, a machine learning model, and an adjustment function.

[0009] In one exemplary embodiment of this disclosure, acquiring a real-time device image of a power device further includes: performing data normalization processing on the real-time device image.

[0010] In one exemplary embodiment of this disclosure, calculating the exposure level of the real-time device image to determine the enhancement iteration number includes: determining the exposure level based on the RGB values ​​of pixels in the real-time device image; and determining the enhancement iteration number based on the number of pixels in the real-time device image, the exposure level, and a preset constant.

[0011] In one exemplary embodiment of this disclosure, acquiring multiple image sets of power equipment includes: acquiring multiple historical images of the power equipment under multiple shooting modes; and dividing the multiple historical equipment images into multiple image sets according to the shooting state.

[0012] In one exemplary embodiment of this disclosure, generating multiple image enhancement models based on multiple image sets, a machine learning model, and an adjustment function includes: determining the number of enhancement iterations for each image set; training the machine learning model and the adjustment function based on the number of enhancement iterations and the image set; and adjusting the machine learning model and the adjustment function based on a computational loss to generate an image enhancement model corresponding to the image set.

[0013] In one exemplary embodiment of this disclosure, determining the enhancement iteration number of an image set includes: calculating the average exposure level of multiple historical device images in the image set; and determining the enhancement iteration number based on the average exposure level.

[0014] In one exemplary embodiment of this disclosure, training a machine learning model and an adjustment function based on the enhancement iteration count and the image set includes: inputting multiple historical device images from the image set into a convolutional neural network model for training to generate a parameter mapping set, wherein the enhancement iteration count is the number of parameters in the parameter mapping set; performing multiple iterative enhancements based on the parameter mapping set and the adjustment function to generate multiple enhanced historical images.

[0015] In one exemplary embodiment of this disclosure, training the machine learning model and the adjustment function based on the enhancement iteration number and the image set further includes: generating the adjustment function according to the parameters in the parameter mapping set, the intermediate image generated in the previous iteration, and the preset bias.

[0016] In one exemplary embodiment of this disclosure, adjusting the machine learning model and adjustment function based on computational loss to generate an image enhancement model corresponding to the image set includes: calculating the computational loss of the enhanced multiple historical images and the unenhanced multiple historical images based on a zero-reference loss function.

[0017] According to one aspect of this disclosure, an image enhancement device for power equipment is proposed, the device comprising: an image module for acquiring a real-time image of the power equipment; an iteration module for calculating the exposure level of the real-time image to determine the number of enhancement iterations; a model module for determining a target image enhancement model from a plurality of image enhancement models based on the number of enhancement iterations, the target image enhancement model being generated by a trained machine learning model and an adjustment function; and an enhancement module for inputting the real-time image into the target image enhancement model to generate an enhanced image of the equipment.

[0018] In one exemplary embodiment of this disclosure, it further includes: a training module for acquiring multiple image sets of power equipment; and training and generating multiple image enhancement models based on the multiple image sets, a machine learning model, and an adjustment function.

[0019] According to one aspect of this disclosure, an electronic device is proposed, comprising: one or more processors; a storage device for storing one or more programs; and, when the one or more programs are executed by the one or more processors, causing the one or more processors to implement the method as described above.

[0020] According to one aspect of this disclosure, a computer-readable medium is provided having a computer program stored thereon that, when executed by a processor, implements the method described above.

[0021] According to the image enhancement method, apparatus, electronic device, and computer-readable medium for power equipment disclosed herein, a real-time image of the power equipment is acquired; the exposure level of the real-time image is calculated to determine the enhancement iteration number; a target image enhancement model is determined from multiple image enhancement models based on the enhancement iteration number, the target image enhancement model being generated by a trained machine learning model and an adjustment function; the real-time image is input into the target image enhancement model to generate an enhanced image, which can effectively enhance the image brightness of various power equipment. The image obtained by this scheme is clearer and easier to identify, and it can also improve the speed and accuracy of edge computing.

[0022] It should be understood that the above general description and the following detailed description are merely exemplary and do not limit this disclosure. Attached Figure Description

[0023] The above and other objects, features, and advantages of this disclosure will become more apparent from the detailed description of exemplary embodiments with reference to the accompanying drawings. The drawings described below are merely some embodiments of this disclosure, and those skilled in the art will be able to obtain other drawings based on these drawings without any inventive effort.

[0024] Figure 1 This is a system block diagram illustrating an image enhancement method and apparatus for power equipment according to an exemplary embodiment.

[0025] Figure 2 This is a flowchart illustrating an image enhancement method for power equipment according to an exemplary embodiment.

[0026] Figure 3 This is a flowchart illustrating an image enhancement method for power equipment according to another exemplary embodiment.

[0027] Figure 4 This is a schematic diagram illustrating an image enhancement method for power equipment according to another exemplary embodiment.

[0028] Figure 5 This is a schematic diagram showing the average PSNR of images enhanced by different methods in the power equipment dataset.

[0029] Figure 6 This is a schematic diagram of the average SSIM of images enhanced by different methods in the power equipment dataset.

[0030] Figure 7 This is a schematic diagram of the average US of images enhanced by different methods in the power equipment dataset.

[0031] Figure 8This is a schematic diagram illustrating the average PSNR of the images enhanced by the scheme disclosed in this publication under different datasets.

[0032] Figure 9 This is a schematic diagram of the average SSIM of the images enhanced by the scheme disclosed in this publication under different datasets.

[0033] Figure 10 This is a block diagram illustrating an image enhancement device for power equipment according to an exemplary embodiment.

[0034] Figure 11 This is a block diagram illustrating an electronic device according to an exemplary embodiment.

[0035] Figure 12 This is a block diagram illustrating a computer-readable medium according to an exemplary embodiment. Detailed Implementation

[0036] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, they are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted.

[0037] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this disclosure.

[0038] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0039] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0040] It should be understood that while the terms first, second, third, etc., may be used herein to describe various components, these components should not be limited by these terms. These terms are used to distinguish one component from another. Therefore, the first component discussed below may be referred to as the second component without departing from the teachings of this disclosure. As used herein, the term "and / or" includes all combinations of any and more of the associated listed items.

[0041] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of exemplary embodiments, and the modules or processes in the drawings are not necessarily necessary for implementing this disclosure, and therefore cannot be used to limit the scope of protection of this disclosure.

[0042] The inventors of this disclosure have discovered that commonly used low-brightness image enhancement methods include histogram equalization, Retinex-based methods, and deep learning-based methods. Histogram equalization can expand gray levels with a large number of pixels and compress gray levels with fewer pixels, thereby expanding the dynamic range of pixel values, improving image contrast and gray-level variations, and making the image clearer. However, this method lacks the ability to select data and operates globally on the image, which may reduce the details of useful information in the image. Therefore, histogram equalization is more suitable for images that are generally too dark or too bright. Retinex theory reduces the influence of the incident image and preserves the reflected image as much as possible, making the image clearer. However, Retinex-based methods operate under the assumption that there is no noise or color distortion in the image, while actual captured images are inevitably affected by noise. With the continuous development of deep learning, the field of computer vision combined with deep learning has also made great progress. These methods require data-driven approaches, and the larger the amount of data, the better the performance. It can automatically learn rules by optimizing the loss function, eliminating the need for manual rule design. Furthermore, this type of method is highly portable, with some mature frameworks allowing cross-platform use. The Retinex-Net method integrates Retinex theory and deep learning, dividing the image brightness enhancement process into three steps: decomposition, adjustment, and reconstruction. In the decomposition step, Decom-Net decomposes the input image into an illumination map and a reflection map. During the adjustment process, an encoder-decoder-based enhancement network brightens the illumination map and, to some extent, eliminates noise in the reflection map. Finally, the adjusted illumination and reflection maps are reconstructed to achieve the enhancement effect. However, using Retinex-Net, the image exhibits significant color distortion after enhancement.

[0043] In view of the technical bottlenecks existing in the prior art, this disclosure provides an image enhancement method for power equipment. In this disclosure, a power equipment image is used as an example for specific technical description; however, it is understood that the method of this disclosure can also be applied to other fields, and this disclosure is not limited thereto.

[0044] This disclosure proposes a power equipment image enhancement framework for the power Internet of Things (IoT) field, which can improve the computational accuracy and speed of power IoT edge services. Based on data preprocessing of power equipment images, the framework employs the proposed method (hereinafter referred to as the EEIE method) to enhance the brightness of the power equipment images, including calculating the number of iterations, training a convolutional neural network model, iteratively adjusting the image brightness, and calculating a loss function to adjust parameters.

[0045] The image enhancement method (EEIE) disclosed herein dynamically determines the number of iterations for an image based on its average brightness. In actual datasets, although the overall image brightness is relatively low, the contrast differences between individual images are significant. Applying the same number of iterations to all images can lead to overexposure or underexposure in certain local areas. The scheme disclosed herein dynamically determines the number of iterations for image enhancement based on its average brightness, focusing on enhancing low-quality areas while maintaining high-quality areas, thereby improving the overall image quality.

[0046] The method disclosed herein can effectively improve the brightness of power equipment images without adding other noise to the images, thereby improving the computing accuracy and speed of power Internet of Things edge services. The method of this disclosure is described below with the aid of specific embodiments.

[0047] Figure 1 This is a system block diagram illustrating an image enhancement method, apparatus, electronic device, and computer-readable medium for power equipment, according to an exemplary embodiment.

[0048] like Figure 1 As shown, system architecture 10 may include terminal devices 101, 102, and 103, a network 104, and a server 105. Network 104 serves as the medium for providing communication links between terminal devices 101, 102, and 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0049] Terminal devices 101, 102, and 103 can interact with server 105 via network 104 to receive or send data. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as video monitoring applications, web browser applications, instant data transmission applications, and email clients.

[0050] Terminal devices 101, 102, and 103 can be various electronic devices with monitoring functions and supporting data transmission or calculation, including but not limited to power electronic devices, smart cameras, and smart monitoring instruments.

[0051] Terminal devices 101, 102, 103, or server 105 may, for example, acquire real-time images of power equipment; terminal devices 101, 102, 103, or server 105 may, for example, calculate the exposure level of the real-time equipment image to determine the number of enhancement iterations; terminal devices 101, 102, 103, or server 105 may, for example, determine a target image enhancement model from multiple image enhancement models based on the number of enhancement iterations, the target image enhancement model being generated by a trained machine learning model and an adjustment function; terminal devices 101, 102, 103, or server 105 may, for example, input the real-time equipment image into the target image enhancement model to generate an enhanced equipment image.

[0052] Server 105 may, for example, acquire multiple image sets of electrical equipment; server 105 may, for example, generate multiple image enhancement models based on multiple image sets, machine learning models, and adjustment functions.

[0053] Server 105 can be a single physical server, or it can be composed of multiple servers. It should be noted that the real-time monitoring method for power equipment provided in this embodiment can be executed by server 105 and / or terminal devices 101, 102, 103. Correspondingly, the real-time monitoring device for power equipment can be set in server 105 and / or terminal devices 101, 102, 103.

[0054] Figure 2 This is a flowchart illustrating an image enhancement method for power equipment according to an exemplary embodiment. The image enhancement method 20 for power equipment includes at least steps S202 to S208.

[0055] like Figure 2 As shown, in S202, a real-time image of the power equipment is acquired. This also includes performing data normalization processing on the real-time equipment image.

[0056] In S204, the exposure level of the real-time device image is calculated to determine the number of enhancement iterations. This includes: determining the exposure level based on the RGB values ​​of the pixels in the real-time device image; and determining the number of enhancement iterations based on the number of pixels in the real-time device image, the exposure level, and a preset constant.

[0057] Exposure level:

[0058] Where R(x, y), G(x, y), and B(x, y) represent the R, G, and B components of the pixel at (x, y) in the image.

[0059] The number of enhancement iterations is:

[0060]

[0061] Where num is the number of pixels in the image, and lum(x, y) represents the brightness of the pixel at position (x, y) in the image. , These are two preset constants.

[0062] In S206, a target image enhancement model is determined from multiple image enhancement models based on the number of enhancement iterations. The target image enhancement model is generated by a trained machine learning model and an adjustment function.

[0063] In step S208, the real-time device image is input into the target image enhancement model to generate an enhanced device image. The enhanced device image can also be used for target recognition or device defect detection.

[0064] According to the image enhancement method for power equipment disclosed herein, a real-time image of the power equipment is acquired; the exposure level of the real-time image is calculated to determine the enhancement iteration number; a target image enhancement model is determined from multiple image enhancement models based on the enhancement iteration number, the target image enhancement model being generated by a trained machine learning model and an adjustment function; the real-time image is input into the target image enhancement model to generate an enhanced image, which can effectively enhance the image brightness of various types of power equipment. The image obtained by this scheme is clearer and easier to identify, and it can also improve the speed and accuracy of edge computing.

[0065] It should be clearly understood that this disclosure describes how specific examples are formed and used, but the principles of this disclosure are not limited to any details of these examples. Rather, based on the teachings of this disclosure, these principles can be applied to many other embodiments.

[0066] Figure 3 This is a flowchart illustrating an image enhancement method for power equipment according to another exemplary embodiment. Figure 3 The process shown in step 30 is a detailed description of "generating a target image enhancement model from a trained machine learning model and an adjustment function".

[0067] like Figure 3 As shown, in S302, an image set of the power equipment is acquired. This includes: acquiring multiple historical images of the power equipment under multiple shooting modes; and dividing the multiple historical images of the power equipment into multiple image sets according to the shooting state.

[0068] To capture images of power equipment in a real substation environment, three shooting methods can be used: 1. Fixed-point shooting: Shooting the power equipment from a fixed direction. 2. Timed shooting: Shooting the power equipment at regular intervals. 3. Fixed-path shooting: To mimic the shooting process of a real intelligent inspection robot, shooting the power equipment along a planned route. Simultaneously, to improve the model's generalization ability, enabling the edge computer to quickly identify and address equipment problems, a large number of images were specifically taken of problematic equipment, such as insulators with oil stains, disconnect switches and coolers with foreign objects inside or outside, and capacitors with rusted metal surfaces. Furthermore, a large number of equipment images were specifically taken in low-light environments such as when lights are off or on cloudy days.

[0069] Historical device images can also be divided into multiple image sets according to different shooting conditions, such as cloudy, sunny, rainy, with lighting, and without lighting, etc.

[0070] In step S304, the number of enhancement iterations for the image set is determined. This includes: calculating the average exposure level of multiple historical device images in the image set; and determining the number of enhancement iterations based on the average exposure level. First, the number of iterations N for image enhancement during one round of training by the convolutional neural network can be determined based on the average exposure level of the images. This number is also the number of parameter mappings output by the neural network model. Then, the initial image is input into the convolutional neural network to calculate the model parameter mappings. Next, the image is enhanced N times based on the parameter mappings. Finally, the loss between the enhanced image and the initial image is calculated to adjust the model parameters, bringing the model to its optimal state.

[0071] As mentioned above, the number of iterations determines the number of parameter mappings output by the convolutional neural network model. Too few iterations will result in insufficient adjustment of pixels with insufficient brightness in the image, causing underexposure; too many iterations will over-adjust pixels with normal or high brightness in the image, causing overexposure. The average brightness of an image can reflect its contrast to a certain extent. Therefore, the number of iterations can be dynamically determined based on the average brightness of historical device images. This processing method can take into account the overall contrast of the image. If the image contrast is high, the dark areas in the image can be enhanced with fewer iterations; if the image contrast is low, the image needs to be enhanced with more iterations. The number of enhancement iterations is shown in formula (1).

[0072] (1)

[0073] Where num is the number of pixels in the image, and lum(x,y) represents the brightness of the pixel at (x,y) in the image. The calculation process is shown in formula (2). , These are two constants, used to avoid the logarithmic result tending towards negative infinity. This formula calculates the brightness value of each pixel in the original image. Then, calculate the natural logarithm of the brightness value, then average the logarithms of all brightness values, and finally calculate the natural exponent of the average value. The range of brightness values ​​is 0-255. To avoid N being too large, which would lead to too many iterations and increase computational complexity, the logarithm of the above results can be taken to base 2, limiting the range to 0-8.

[0074] Generally, the higher the average brightness of the image, the fewer the number of iterations; conversely, the lower the average brightness of the image, the more iterations. Therefore, we take the opposite of the final result and add 8. Finally, if the number of iterations for enhancement calculated based on the average brightness of the image is 0, we set the value of N to 1, resulting in formula (1).

[0075] in, It can reflect the contrast information of an image. The larger the value, the higher the contrast of the image; the smaller the value, the lower the contrast of the image.

[0076] (2)

[0077] Where R(x, y), G(x, y), and B(x, y) represent the R, G, and B components of the pixel at (x, y) in the image.

[0078] In step S306, the machine learning model and adjustment function are trained based on the enhancement iteration count and the image set. This includes: inputting multiple historical device images from the image set into a convolutional neural network model for training to generate a parameter mapping set, wherein the enhancement iteration count is the number of parameters in the parameter mapping set; and performing multiple iterative enhancements based on the parameter mapping set and the adjustment function to generate multiple enhanced historical images.

[0079] This also includes generating the adjustment function based on the parameters in the parameter mapping set, the intermediate image generated in the previous iteration, and the preset bias.

[0080] In one embodiment, the convolutional neural network may contain 7 layers. The first 6 layers each contain 32 convolutional kernels of size 3×3 with a stride of 1 and an activation function of ReLU. The last layer contains 24 convolutional kernels of size 3×3 with a stride of 1 and an activation function of Tanh. The final output is a series of parameter mappings. The image pixel estimation curve can use these parameter mappings to iteratively adjust the pixels of the input image, and finally generate the adjusted image. The adjustment function is shown in formula (3):

[0081] (3)

[0082] Where x represents the coordinates of a pixel. Let AN(x) represent the parameter mapping of the same size as the input image, AN-1(x) represent the intermediate image generated in the Nth iteration, A0(x) represent the input image, and AN(x) be the image generated after N iterations of enhancement during one round of training. M is the bias formed by normalizing the parameter mapping, used to correct potential underexposure and overexposure in the enhanced image.

[0083] In S308, the machine learning model and adjustment function are adjusted based on the computational loss to generate an image enhancement model corresponding to the image set. This includes calculating the computational loss of multiple enhanced historical images and multiple unenhanced historical images based on a zero-reference loss function.

[0084] In one embodiment, the advantage of using a zero-reference loss function to optimize model parameters is that it does not require paired low-brightness / high-brightness images as input, thus eliminating the need for manually synthesizing low-brightness images. The loss function is calculated as shown in Equation (4).

[0085] 4

[0086] Among them W sc W ep W col and W ils This represents the weights, used to address the problem of imbalanced losses. L sc It is a spatial loss, which increases the contrast of the enhanced image by maintaining the differences between adjacent regions in the original image and the enhanced image, thereby making the enhanced image clearer. Its calculation method is shown in formula (5).

[0087] (5)

[0088] Where Y and I represent the average gray values ​​of local regions in the enhanced image and the input image, respectively. K represents the number of local regions, and Ω(i) represents the four neighboring regions centered on region i.

[0089] L ep It is exposure loss, which measures the exposure level of the enhanced image. This is achieved by reducing L... ep It can reduce the occurrence of overexposure or underexposure. Its calculation is shown in formula (6).

[0090] 6

[0091] Where M represents the number of non-overlapping areas in the image, and E represents the good exposure level, which can be set based on previous experiments and the average exposure level of images in the power equipment dataset.

[0092] L col It is the color loss, which is used to correct potential color distortion in the enhanced image. Its calculation is shown in formula (7).

[0093] 7

[0094] in This represents the average intensity of the p channel of the output image, where (p,q) represents a pair of channels.

[0095] L ils This represents the brightness smoothing loss, used to maintain the monotonic relationship between adjacent pixels. Its calculation is shown in formula (8).

[0096] (8)

[0097] Where N represents the number of iterations. and Calculate the gradient in the horizontal and vertical directions respectively.

[0098] In the task of image enhancement for power equipment, to consider the diversity of power equipment and the complexity of the environment, and to ensure that the image enhancement model can achieve good enhancement results for each image of power equipment, the following steps are taken: First, images of power equipment under various shooting conditions are collected according to specified principles. Images meeting the requirements are then selected to establish multiple image sets. Each image set includes images of various types of power equipment with both normal and underexposed exposures, satisfying the model's input requirements, and the data is normalized. Next, brightness enhancement is performed on the images in the dataset. First, the number of iterations for enhancement is calculated based on the average exposure of the images. Then, a convolutional neural network model is used to calculate parameter mappings, and these parameter mappings are used to iteratively adjust the image brightness. Finally, the loss is calculated by combining the result image generated in the last iteration with the original image, thereby adjusting the model parameters. The structure of this framework is as follows: Figure 4 As shown.

[0099] In one embodiment, a total of 5609 images of power equipment were collected, encompassing various types of equipment in different environments. Then, 2068 images that met the experimental requirements were selected as the dataset. Simultaneously, the Retinex-Net method was used as a comparison method; to meet the input requirements of Retinex-Net, the images underwent low-brightness processing.

[0100] The specific experimental environment was as follows: The model experiments were conducted on a Windows 10 system using 8GB of RAM, an NVIDIA 2080Ti GPU, and PyTorch 1.0.0. The dataset contained a total of 2068 images of power equipment status, of which 1997 images were used for model training, and the remaining images were used for model testing.

[0101] The number of training samples selected at one time is 8, and the initial learning rate is 0.0001. The good exposure level E is set to 0.5. 150 iterations are performed on the preset image dataset and LOL dataset.

[0102] Experimental indicators:

[0103] Three metrics can be used to evaluate the image enhancement quality of power equipment: peak signal-to-noise ratio (PSNR), structural similarity (SSIM), and user survey (US).

[0104] Power images often differ from the original images after enhancement. The PSNR can be used to measure the distortion of the enhanced power image. The PSNR is calculated as shown in formula (7).

[0105] (7)

[0106] MASI represents the maximum possible pixel value of the image. If each pixel is represented by 8 bits, then MASI is 255. Typically, if pixel values ​​are represented by B bits, then... .

[0107] SSIM is a metric for measuring the similarity between two images. It measures the similarity between sample images x and y based on three factors: brightness, contrast, and structure, and models distortion as a combination of these three different factors. The calculation of SSIM is shown in Equation (8):

[0108] (8)

[0109] in Let x be the mean. Let y be the mean. Let x be the variance. Let y be the variance. Let x and y be the covariances. and These are two constants, used to avoid instability caused by the denominator approaching 0.

[0110] After an image undergoes brightness enhancement, noise is introduced, and the user's subjective perception can reflect the image quality to some extent. User study (US) refers to users rating the enhanced image to evaluate its quality. A high US score indicates better subjective visual quality. Here, the US score is limited to 0 to 10, and 20 people are invited to rate the enhanced image. The average of all scores is taken as the US score of the image enhancement method.

[0111] Experimental Results and Analysis

[0112] The image enhancement method for power equipment can be validated from two aspects: image enhancement effect analysis and model training effect analysis. Image enhancement effect analysis verifies whether the EEIE method can better enhance power equipment images to meet the computational requirements of power IoT edge services. Here, the method in this disclosure is compared with Retinex-Net. On the other hand, the model needs to effectively enhance power equipment images, especially images of faulty power equipment. Model training effect analysis verifies whether the image dataset can effectively train the image enhancement model, enabling it to better enhance unknown equipment images, especially those of faulty equipment. For example, the image dataset and the LOL dataset can be used for model training separately.

[0113] Image Enhancement Effectiveness Analysis

[0114] The experiments used a power equipment image dataset to train and test the proposed method and the Retinex-Net method. Their PSNR and SSIM were then compared. The experiments included 1997 training instances and underwent 150 iterations. Figure 5 and Figure 6 The PSNR and SSIM results obtained by the two methods are shown. Figure 7 The US results are displayed.

[0115] like Figure 5 and Figure 6 As shown, the scheme disclosed in this paper has higher PSNR and SSIM than Retinex-Net, indicating that after EEIE enhances the power equipment image, the background noise of the image is less and the image distortion is less noticeable. Figure 7 As shown, the EEIE method achieves a higher US score than the Retinex-Net method, indicating that the image enhanced by EEIE has better subjective quality, thus reflecting better overall image quality. Therefore, EEIE can effectively enhance images of power equipment, meeting the computational requirements of edge services in the power IoT.

[0116] Model training effectiveness analysis

[0117] Images of a variety of power equipment were collected and a dataset was constructed. To verify whether the dataset of this scheme can effectively train the image augmentation model and promote fault diagnosis in the power Internet of Things edge service, the image augmentation model was trained using the power image dataset of this scheme and the LOL dataset under the same experimental environment and parameter settings. Then the average PSNR and SSIM of the augmented images were compared. Figure 8 and Figure 9 The comparison between PSNR and SSIM is shown separately.

[0118] like Figure 8 and Figure 9 As shown, the model trained on the power equipment image dataset of this scheme achieves higher PSNR and SSIM for the enhanced images. This indicates that the dataset of this scheme can effectively train the model, enabling it to quickly identify equipment problems when enhancing unknown power equipment images, especially those with faults. This improves the accuracy of fault diagnosis in the power Internet of Things edge service, allowing it to take timely and accurate maintenance measures for power equipment.

[0119] Those skilled in the art will understand that all or part of the steps of the above embodiments are implemented as a computer program executed by a CPU. When the computer program is executed by the CPU, it performs the functions defined by the methods provided in this disclosure. The program can be stored in a computer-readable storage medium, such as a read-only memory, a magnetic disk, or an optical disk.

[0120] Furthermore, it should be noted that the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of this disclosure, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0121] The following are embodiments of the apparatus disclosed herein, which can be used to execute embodiments of the method disclosed herein. For details not disclosed in the apparatus embodiments of this disclosure, please refer to the embodiments of the method disclosed herein.

[0122] Figure 10 This is a block diagram illustrating an image enhancement device for power equipment according to an exemplary embodiment. Figure 10 As shown, the image enhancement device 100 for power equipment includes: an image module 1002, a multiplication module 1004, a model module 1006, an enhancement module 1008, and a training module 1010.

[0123] The image module 1002 can be installed in an IoT edge device or a server to acquire real-time device images of power equipment.

[0124] The iteration module 1004 can be set in an IoT edge device or a server to calculate the exposure level of the real-time device image to determine the number of enhancement iterations;

[0125] The model module 1006 can be set in an IoT edge device or a server, and is used to determine a target image enhancement model from multiple image enhancement models based on the number of enhancement iterations. The target image enhancement model is generated by a trained machine learning model and an adjustment function.

[0126] The enhancement module 1008 can be set in an IoT edge device or a server, and is used to input the real-time device image into the target image enhancement model to generate an enhanced device image.

[0127] The training module 1010 can be set in the server to acquire multiple image sets of power equipment; and to train and generate multiple image enhancement models based on multiple image sets, machine learning models, and adjustment functions.

[0128] According to the image enhancement device for power equipment disclosed herein, a real-time image of the power equipment is acquired; the exposure level of the real-time image is calculated to determine the enhancement iteration number; a target image enhancement model is determined from multiple image enhancement models based on the enhancement iteration number, the target image enhancement model being generated by a trained machine learning model and an adjustment function; the real-time image is input into the target image enhancement model to generate an enhanced image, which can effectively enhance the image brightness of various types of power equipment. The image obtained by this scheme is clearer and easier to identify, and it can also improve the speed and accuracy of edge computing.

[0129] Figure 11 This is a block diagram illustrating an electronic device according to an exemplary embodiment.

[0130] The following reference Figure 11 To describe an electronic device 1100 according to such an embodiment of the present disclosure. Figure 11 The electronic device 1100 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0131] like Figure 11 As shown, the electronic device 1100 is presented in the form of a general-purpose computing device. The components of the electronic device 1100 may include, but are not limited to: at least one processing unit 1110, at least one storage unit 1120, a bus 1130 connecting different system components (including storage unit 1120 and processing unit 1110), a display unit 1140, etc.

[0132] The storage unit stores program code that can be executed by the processing unit 1110, causing the processing unit 1110 to perform the steps described in this specification according to various exemplary embodiments of this disclosure. For example, the processing unit 1110 can perform actions such as... Figure 2 , Figure 3 The steps are shown in the figure.

[0133] The storage unit 1120 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 11201 and / or a cache storage unit 11202, and may further include a read-only memory unit (ROM) 11203.

[0134] The storage unit 1120 may also include a program / utility 11204 having a set (at least one) program module 11205, such program module 11205 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0135] Bus 1130 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0136] Electronic device 1100 can also communicate with one or more external devices 1100' (e.g., keyboard, pointing device, Bluetooth device, etc.), enabling users to communicate with devices that interact with electronic device 1100, and / or any device that allows electronic device 1100 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 1150. Furthermore, electronic device 1100 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 1160. Network adapter 1160 can communicate with other modules of electronic device 1100 via bus 1130. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 1100, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0137] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software, or by combining software with necessary hardware. Therefore, as... Figure 12As shown, the technical solution according to the embodiments of this disclosure can be embodied in the form of a software product. The software product can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, mobile hard drive, etc.) or on a network, and includes several instructions to cause a computing device (such as a personal computer, server, or network device, etc.) to execute the above-described method according to the embodiments of this disclosure.

[0138] The software product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0139] The computer-readable storage medium may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable storage medium may also be any readable medium other than a readable storage medium, capable of transmitting, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0140] Program code for performing the operations of this disclosure can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0141] The aforementioned computer-readable medium carries one or more programs that, when executed by a device, cause the computer-readable medium to perform the following functions: acquire a real-time device image of a power device; calculate the exposure level of the real-time device image to determine the number of enhancement iterations; determine a target image enhancement model from multiple image enhancement models based on the number of enhancement iterations, the target image enhancement model being generated by a trained machine learning model and an adjustment function; and input the real-time device image into the target image enhancement model to generate an enhanced device image.

[0142] Those skilled in the art will understand that the above modules can be distributed in the device as described in the embodiments, or they can be modified accordingly and placed in one or more devices that are unique to this embodiment. The modules in the above embodiments can be combined into one module, or they can be further divided into multiple sub-modules.

[0143] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, mobile terminal, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0144] Exemplary embodiments of this disclosure have been specifically shown and described above. It should be understood that this disclosure is not limited to the detailed structures, arrangements, or implementations described herein; rather, this disclosure is intended to cover various modifications and equivalent arrangements contained within the spirit and scope of the appended claims.

Claims

1. An image enhancement method for power equipment, characterized in that, include: Acquire real-time equipment images of power equipment; The exposure level of the real-time device image is calculated to determine the number of enhancement iterations; Acquire multiple image sets of power equipment; Multiple image enhancement models are generated based on training with multiple image sets, machine learning models, and adjustment functions. The target image enhancement model is determined from multiple image enhancement models based on the number of enhancement iterations, and the target image enhancement model is generated by a trained machine learning model and an adjustment function. The real-time device image is input into the target image enhancement model to generate an enhanced device image; Multiple image enhancement models are generated based on training with multiple image sets, machine learning models, and adjustment functions, including: Determine the number of enhancement iterations for the image set; The machine learning model and adjustment function are trained based on the number of enhancement iterations and the image set; The machine learning model and adjustment function are adjusted based on computational loss to generate an image enhancement model corresponding to the image set; The number of iterations is increased as shown in formula (1); ⑴； Where num is the number of pixels in the image, and lum(x,y) represents the brightness of the pixel at position (x,y) in the image. , They are two constants; Training the machine learning model and adjustment function based on the enhancement iteration number and the image set includes: Multiple historical device images from the image set are input into a convolutional neural network model for training to generate a parameter mapping set, wherein the number of enhancement iterations is the number of parameters in the parameter mapping set; Based on the parameter mapping set and the adjustment function, multiple iterations of enhancement are performed to generate multiple enhanced historical images.

2. The method as described in claim 1, characterized in that, Acquiring real-time equipment images of power equipment also includes: The real-time device image is subjected to data normalization processing.

3. The method as described in claim 1, characterized in that, Calculating the exposure level of the real-time device image to determine the number of enhancement iterations includes: The exposure level is determined based on the pixel RGB values ​​in the real-time device image; The enhancement iteration number is determined based on the number of pixels, exposure level, and preset constant in the real-time device image.

4. The method as described in claim 1, characterized in that, Acquire multiple image sets of power equipment, including: Acquire multiple historical images of power equipment under various shooting methods; The images from the historical devices are divided into multiple image sets according to the shooting status.

5. The method as described in claim 1, characterized in that, Determine the number of augmentation iterations for the image set, including: Calculate the average exposure level of multiple historical device images in the image set; The number of enhancement iterations is determined based on the average exposure level.

6. The method as described in claim 1, characterized in that, Training the machine learning model and adjustment function based on the enhancement iteration number and the image set further includes: The adjustment function is generated based on the parameters in the parameter mapping set, the intermediate image generated in the previous iteration, and the preset bias.

7. The method as described in claim 1, characterized in that, The machine learning model and adjustment function are adjusted based on computational loss to generate an image enhancement model corresponding to the image set, including: The computational loss is calculated based on the zero-reference loss function for the enhanced historical images and the unenhanced historical images.

8. An image enhancement device for power equipment, characterized in that, include: The image module is used to acquire real-time images of the power equipment. The iteration module is used to calculate the exposure level of the real-time device image to determine the number of enhancement iterations; The training module is used to acquire multiple image sets of power equipment; Multiple image enhancement models are generated based on training with multiple image sets, machine learning models, and adjustment functions. A model module is used to determine a target image enhancement model from multiple image enhancement models based on the number of enhancement iterations, wherein the target image enhancement model is generated by a trained machine learning model and an adjustment function; An enhancement module is used to input the real-time device image into the target image enhancement model to generate an enhanced device image; Multiple image enhancement models are generated based on training with multiple image sets, machine learning models, and adjustment functions, including: Determine the number of enhancement iterations for the image set; The machine learning model and adjustment function are trained based on the number of enhancement iterations and the image set; The machine learning model and adjustment function are adjusted based on computational loss to generate an image enhancement model corresponding to the image set; The number of iterations is increased as shown in formula (1); ⑴； Where num is the number of pixels in the image, and lum(x,y) represents the brightness of the pixel at position (x,y) in the image. , They are two constants; Training the machine learning model and adjustment function based on the enhancement iteration number and the image set includes: Multiple historical device images from the image set are input into a convolutional neural network model for training to generate a parameter mapping set, wherein the number of enhancement iterations is the number of parameters in the parameter mapping set; Based on the parameter mapping set and the adjustment function, multiple iterations of enhancement are performed to generate multiple enhanced historical images.

9. An electronic device, characterized in that, include: One or more processors; Storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-7.

10. A computer-readable medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-7.

Images