Zero-sample blind image restoration method, system, equipment and medium
Through the zero-sample blind image restoration method, the noise of the power system image is acquired and detected, and a random noise set is established to replace the offset noise block. The inverted deterministic diffusion model is used to achieve efficient restoration of the power system image, solving the problems of insufficient adaptability and efficiency in existing technologies.
Patent Information
- Application Number
- CN202510740788.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-05
- Publication Date
- 2025-10-17
AI Technical Summary
Existing image restoration methods in power systems have problems such as dependence on training data, insufficient generalization ability, and poor adaptability, making it difficult to meet the needs of complex and changeable actual degradation scenarios.
A zero-sample blind image restoration method is adopted. By obtaining the input noise of the degraded image to be restored, a first detection operation is performed to identify local noise blocks, a random noise set is established and the offset noise blocks are replaced, and the image restoration is performed using the inverse deterministic diffusion model.
It improves the adaptability and efficiency of image restoration, can effectively repair images in complex environments, and enhance image clarity and visual effects. It is particularly suitable for image processing in power systems.
Smart Images
Figure CN120807356A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of power system image processing, and in particular to a zero-sample blind image restoration method, system, device and medium. BACKGROUND
[0002] In power equipment inspection and operation safety supervision, various devices such as surveillance balls, unmanned aerial vehicles and fixed cameras are usually used to collect images of operation scenes and their auxiliary equipment. However, due to complex and changeable environment, such as shaking of the shooting device, low-visibility weather (such as rain, fog and dust), aging of the imaging device and other reasons, the image is prone to degradation problems such as blur, occlusion and noise, which seriously affects the accuracy of subsequent defect identification and intelligent analysis. Image restoration technology is one of the key technologies to solve such problems.
[0003] Existing image restoration methods mainly include training-based methods and non-training methods. The former relies on large-scale labeled image datasets for modeling, learning the mapping relationship between degraded images and clear images, including supervised and unsupervised modes. Supervised learning requires paired samples to be input, and the cost of data acquisition is high, and when the actual degradation type is inconsistent with the training data, the restoration effect decreases significantly; unsupervised learning reduces the dependence on labeled data, but the restoration accuracy is limited and there is also a problem of insufficient generalization ability, which requires retraining the model for different degradation types.
[0004] Non-training image restoration methods are more flexible and practical because they do not rely on task-specific training data, but existing methods are mostly non-blind solutions that require prior knowledge of the type and parameters of the degradation process, making it difficult to meet the needs of the actual degradation scene in the power system, which is variable and uncontrollable. To meet the requirements of high adaptability, high efficiency and low deployment cost for image restoration in power inspection, a truly non-training blind image restoration solution that can automatically adapt to multiple degradation conditions is urgently needed. SUMMARY
[0005] In view of the above existing problems, the present application is proposed.
[0006] Therefore, the present application provides a zero-sample blind image restoration method, system, device and medium, which can solve the image restoration problem under multiple degradation conditions and improve the adaptability and efficiency of image restoration.
[0007] To solve the above technical problems, the present application provides the following technical solutions:
[0008] In a first aspect, the present application provides a zero-sample blind image restoration method, comprising:
[0009] Obtaining a degraded image to be restored, and calculating the input noise of the degraded image to be restored;
[0010] performing a first detection operation on the input noise;
[0011] The first detection operation is used to detect a local noise block that produces deviation in the input noise;
[0012] establishing a random noise set, obtaining a random noise that is most similar to the local noise block, and replacing the local noise block that produces deviation with the random noise that is similar to the local noise block;
[0013] The obtaining of the random noise that is most similar to the local noise block is performed by establishing a minimum difference function;
[0014] The minimum difference function is used to obtain a random noise in the random noise set that has the minimum difference from the local noise block;
[0015] According to the noise data after replacement, the degraded image is restored.
[0016] As a preferred scheme of the zero-sample blind image restoration method, the first detection operation on the input noise comprises:
[0017] A sliding window is preset, and the sliding window is used to divide the input noise of the degraded image to be restored into a plurality of noise blocks that have the same size as the sliding window;
[0018] The obtained noise blocks are subjected to the first detection operation, and noise blocks that do not satisfy the first detection operation are recorded as local noise blocks that produce deviation.
[0019] As a preferred scheme of the zero-sample blind image restoration method, the first detection operation comprises:
[0020] A deviation detection strategy is preset;
[0021] The deviation detection strategy is established according to analysis of a data distribution rule that is consistent with the input noise of the degraded image to be restored.
[0022] This preferred scheme can more accurately identify which noise blocks are local noise blocks that produce deviation, thereby improving the accuracy of noise replacement and further improving the effect of image restoration. By presetting the deviation detection strategy, the input noise of the degraded image to be restored can be analyzed and processed more deeply, so that only the local noise blocks that produce deviation are replaced, unnecessary noise replacement operations are avoided, and the efficiency and accuracy of image restoration are improved.
[0023] As a preferred scheme of the zero-sample blind image restoration method, the establishment of the random noise set, the obtaining of the random noise that is most similar to the local noise block, and the replacement of the local noise block that produces deviation with the random noise that is similar to the local noise block comprise:
[0024] establish a random noise set, the distribution of which is the same as the data distribution of the input noise of the image to be restored;
[0025] establish a minimum difference function, and obtain a random noise in the random noise set with the minimum difference from the local noise block according to the established minimum difference function;
[0026] replace the random noise in the random noise set with the minimum difference from the local noise block with the offset local noise block.
[0027] As a preferred scheme of the zero-sample blind image restoration method, the method further comprises:
[0028] input the noise data after replacement into an inverse deterministic diffusion model;
[0029] the output of the inverse deterministic diffusion model is the restored degraded image.
[0030] As a preferred scheme of the zero-sample blind image restoration method, the method further comprises:
[0031] classify and save the noise blocks satisfying the first detection operation and the noise blocks not satisfying the first detection operation;
[0032] construct a binary mask according to the noise blocks not satisfying the first detection operation.
[0033] As a preferred scheme of the zero-sample blind image restoration method, the method further comprises:
[0034] establish an inverse deterministic diffusion model, and calculate the input noise of the image to be restored according to the inverse deterministic diffusion model;
[0035] a jump term is introduced in the inverse diffusion process of the inverse deterministic diffusion model.
[0036] In a second aspect, the present application provides a zero-sample blind image restoration system, comprising:
[0037] a data acquisition module, configured to acquire a degraded image to be restored and calculate the input noise of the degraded image to be restored;
[0038] a detection module, configured to perform a first detection operation on the input noise;
[0039] the first detection operation is used to detect the local noise block with offset in the input noise.
[0040] The replacement module is configured to establish a random noise set, obtain random noise most similar to the local noise block, and replace the local noise block with the random noise most similar to the local noise block to generate an offset local noise block.
[0041] The random noise most similar to the local noise block is obtained by establishing a minimum difference function.
[0042] The minimum difference function is configured to obtain random noise in the random noise set with a minimum difference from the local noise block.
[0043] The restoration module is configured to restore the degraded image according to the noise data after the replacement is completed.
[0044] In a third aspect, the present application provides an electronic device including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method described above when executing the computer program.
[0045] In a fourth aspect, the present application provides a computer readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the steps of the method described above.
[0046] Compared with the prior art, the present application has the following beneficial effects: the present application provides a zero-sample blind image restoration method, obtains a degraded image to be restored, and calculates input noise of the degraded image to be restored; performs a first detection operation on the input noise; establishes a random noise set, obtains random noise most similar to a local noise block, and replaces the local noise block with the random noise most similar to the local noise block to generate an offset local noise block; and restores the degraded image according to the noise data after the replacement is completed. The steps of obtaining the degraded image to be restored and calculating the input noise can accurately locate the noise information in the image and provide a basis for subsequent processing. The first detection operation step can efficiently identify the noise type in the image and provide accurate guidance for subsequent noise replacement. The steps of establishing the random noise set and replacing the local noise block with the random noise most similar to the local noise block improve the flexibility and accuracy of image restoration by utilizing the diversity of random noise. The step of restoring the degraded image according to the noise data after the replacement is completed realizes effective repair of the degraded image and improves the clarity and visual effect of the image.
[0047] In summary, the method has good adaptability in processing linear blur, random noise, local occlusion, and complex nonlinear degradation, and is particularly suitable for improving the usability and recognition accuracy of power images in complex inspection environments. BRIEF DESCRIPTION OF DRAWINGS
[0048] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.
[0049] Figure 1 A method flow chart of a zero sample blind image restoration method provided by an embodiment of the present application.
[0050] Figure 2 A zero sample blind image restoration method process schematic diagram of a zero sample blind image restoration method provided by an embodiment of the present application.
[0051] Figure 3 An internal structure diagram of an electronic device of a zero sample blind image restoration method provided by an embodiment of the present application. DETAILED DESCRIPTION
[0052] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the specific embodiments of the present application will be described in detail below with reference to the drawings in the specification. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should be within the scope of protection of the present application.
[0053] Embodiment 1, refer to Figures 1-3 As a first embodiment of the present application, the embodiment provides a zero sample blind image restoration method, comprising:
[0054] In the prior art, there are some problems, such as limited image restoration effect, insufficient restoration accuracy, poor generalization ability and strong dependence on specific degradation types. These problems lead to difficulties in meeting the requirements of high adaptability, high efficiency and low deployment cost for image restoration in practical applications, especially in the field of power system image processing.
[0055] The present application provides a method that can effectively solve the above-mentioned problems. Next, how to realize the zero sample blind image restoration method will be described in detail in combination with multiple embodiments.
[0056] Figure 1 A method flow chart of a zero sample blind image restoration method is shown, comprising:
[0057] S101, obtaining a to-be-restored degraded image and calculating the input noise of the to-be-restored degraded image;
[0058] It should be noted that the existing image restoration method always lacks the establishment of a model or a training data set, but there are not enough pairs of clear images and degraded images for supervised learning in actual application, and the imaging conditions and degradation types of different devices are significantly different, resulting in poor generalization ability of the trained model. To solve the above problems, the present application provides a zero-shot blind image restoration method, which can realize image restoration without training.
[0059] In an optional embodiment, the input noise of the degraded image to be restored can be calculated in different ways, for example, by using an advanced noise estimation algorithm that can accurately estimate the noise distribution in the degraded image based on the statistical characteristics and prior knowledge of the image. The specific steps can include:
[0060] First, the input degraded image is preprocessed, such as denoising, contrast enhancement, etc., to improve the accuracy of noise estimation.
[0061] Then, the noise estimation algorithm is used to analyze the preprocessed image and extract the statistical characteristics of the image, such as mean, variance, histogram, etc.
[0062] Next, combine prior knowledge, such as common noise types, noise intensity ranges, etc., to construct a noise model.
[0063] Finally, through an iterative optimization algorithm, the parameters of the noise model are continuously adjusted until the noise distribution output by the model matches the noise distribution in the actual degraded image, thereby obtaining accurate noise estimation results.
[0064] In an optional embodiment, the input noise of the degraded image to be restored can also be calculated by establishing a model, i.e., establishing a noise estimation network based on deep learning. Through the learning of a large number of degraded image samples, the network can automatically extract noise features from the image and achieve accurate noise estimation. With the strong learning ability of the deep learning model, it can adapt to different types of noise and degradation conditions, improving the generalization performance of noise estimation. Through the combination of the two models, the calculation accuracy of the input noise of the degraded image to be restored can be further improved, thereby improving the performance of the entire zero-shot blind image restoration system.
[0065] It should be noted that the above operation cannot completely avoid errors in noise estimation, therefore, in actual application, the present application proposes an optimization strategy to reduce the influence of noise estimation errors on image restoration effect.
[0066] In the embodiment of the present application, the degraded image to be restored is obtained, and the input noise of the degraded image to be restored is calculated, which includes:
[0067] An inverse deterministic diffusion model is established, and the input noise of the degraded image to be restored is calculated according to the inverse deterministic diffusion model.
[0068] A jump term is introduced in the reverse diffusion process of the reverse deterministic diffusion model.
[0069] In an optional embodiment, the reverse deterministic diffusion model can be a probability-based generative model that gradually restores a clear image from noisy data by simulating a reverse diffusion process.
[0070] In an embodiment of the present application, the following optimization problem is solved by gradient descent to obtain the input noise of the degraded image Y
[0071]
[0072] where G DDIM (z) is computationally expensive in the diffusion model, the present application introduces a jump term δ t in the reverse diffusion process, thereby achieving faster convergence.
[0073] In an optional embodiment, in order to introduce the jump term δ t , some time steps can be skipped in the reverse diffusion process.
[0074] Specifically, assuming that the time steps of the original diffusion process are T, a new time sequence {t0, t1,..., tK} can be defined, where K < T, and ti represents the time point after jumping.
[0075] Therefore, the reverse diffusion process can be represented as:
[0076] z ti+1 = z ti + δ ti
[0077] where δ ti is the jump term at time point ti, which can be obtained by some strategy or learning.
[0078] It should be noted that obtaining the to-be-restored degraded image and calculating the input noise of the to-be-restored degraded image provides accurate basic data for subsequent noise detection and processing. By introducing a jump term in the reverse deterministic diffusion model, not only the computational efficiency is improved, but also the accuracy of the input noise estimation is ensured, which is crucial for subsequent identification of noise types in the image and execution of accurate noise replacement. In addition, the application range of this method is wide, especially suitable for processing power images in complex inspection environments, which can significantly improve the usability and recognition accuracy of images.
[0079] S102, a first detection operation is performed on the input noise;
[0080] In the embodiment of the present invention, the first detection operation is used to detect a local noise block that generates an offset in the input noise.
[0081] In an optional embodiment, the first detection operation can be implemented using a preset detection algorithm that automatically analyzes the data characteristics of the input noise and identifies local noise blocks that cause offsets. These offset local noise blocks are often caused by interference during image acquisition or transmission, and their presence may affect image quality and clarity.
[0082] In an optional embodiment, the prediction algorithm can use machine learning or deep learning technology to identify noise characteristics by training a large amount of image data and automatically detect shifted local noise blocks. This method has high accuracy and adaptability and can handle different types of noise and degradation.
[0083] Specifically, the first detection operation may include the following steps:
[0084] First, preprocess the input noise, such as filtering and denoising, to improve detection accuracy;
[0085] Then, the detection algorithm is used to analyze the preprocessed noise and extract features;
[0086] Next, based on a preset threshold or classifier, determine which noise blocks have generated the offset;
[0087] Finally, the detected offset noise blocks are marked or classified to provide a basis for subsequent processing. Through this method, the offset noise blocks in the image can be effectively identified and accurate guidance can be provided for subsequent noise replacement.
[0088] However, the above-mentioned method of using an algorithm or establishing a model cannot simply and directly realize the detection of the offset noise block in the input noise. In practical applications, in order to improve the detection efficiency and accuracy, the present invention proposes an optimization strategy.
[0089] In an embodiment of the present invention, performing a first detection operation on the input noise includes:
[0090] A sliding window is preset, and the sliding window is used to divide the input noise of the degraded image to be restored into a number of noise blocks of the same size as the sliding window;
[0091] The obtained noise blocks are subjected to a first detection operation, and the noise blocks that do not satisfy the first detection operation are recorded as local noise blocks that generate offsets.
[0092] In an embodiment of the present invention, the first detection operation includes:
[0093] Preset offset detection strategy;
[0094] The offset detection strategy is established according to a data distribution rule of input noise to be analyzed for restoring a degraded image.
[0095] In the embodiment of the present application, the first detection operation further includes:
[0096] The noise blocks satisfying the first detection operation are classified and saved from the noise blocks not satisfying the first detection operation.
[0097] A binary mask is constructed according to the noise blocks not satisfying the first detection operation.
[0098] Specifically, first, a sliding window method is used to process wherein the size of each block is k x k x c, and then each block is flattened into a one-dimensional vector with a size of m = ck 2 wherein k represents a spatial window size, and c represents a channel number.
[0099] A classical statistical hypothesis test is applied to determine whether each flattened block follows a standard normal distribution.
[0100] After testing all the noise blocks, a binary mask M is constructed, wherein M (x, y, v) = 1 indicates that the noise at position (x, y) of the vth channel belongs to a noise block that does not pass the normality test. n×m×c
[0101] It should be noted that the input noise distribution rule selected by the present application is a standard normal distribution, and the offset detection strategy is the distribution rule of the standard normal distribution.
[0102] It should be further noted that the first detection operation on the input noise can accurately distinguish between normal noise and local noise blocks with offset, providing a key basis for noise replacement in subsequent steps. Through the preset sliding window and offset detection strategy, the present application can efficiently process a large amount of noise data, ensuring that each noise block is strictly detected. This accurate detection operation not only improves the accuracy of noise processing, but also enhances the stability and reliability of the entire zero-sample blind image restoration system. In addition, the operation of constructing a binary mask makes the subsequent noise replacement step more intuitive and efficient, further improving the efficiency and quality of image restoration.
[0103] S103, a set of random noises is established, the random noise most similar to the local noise block is obtained, and the random noise similar to the local noise block is replaced with the local noise block with offset.
[0104] It should be noted that because the first detection operation is completed, the noise block with offset will be identified, and therefore the noise block with offset needs to be repaired or replaced.
[0105] In the embodiment of the present application, the most similar random noise to the local noise block is obtained by establishing a minimization difference function.
[0106] In the embodiment of the present application, the minimization difference function is used to obtain the random noise in the random noise set with the smallest difference from the local noise block.
[0107] In an optional embodiment, the minimization difference function can be realized by calculating the similarity or distance between the local noise block and each random noise in the random noise set.
[0108] Specifically, the Euclidean distance, Manhattan distance, cosine similarity, and other measurement methods can be used to measure the difference between the local noise block and the random noise.
[0109] Then, the random noise with the smallest difference is selected as the most similar random noise to replace the local noise block that generates the offset.
[0110] In this way, it can be ensured that the replaced noise block is more coordinated with other parts of the original image, thereby improving the quality of image restoration.
[0111] In addition, the step of establishing a random noise set can be realized by collecting a large number of natural noise samples or using a noise generation algorithm to ensure the diversity and representativeness of the random noise. In the replacement process, the replacement strategy can also be adjusted according to actual needs, such as step-by-step replacement, weighted replacement, and other methods, to further optimize the image restoration effect.
[0112] Finally, through this method, the offset noise block in the degraded image can be effectively repaired, and the clarity and visual effect of the image can be improved.
[0113] In the embodiment of the present application, the random noise set is established, the most similar random noise to the local noise block is obtained, and the random noise similar to the local noise block is replaced by the local noise block that generates the offset, comprising:
[0114] The random noise set is established, and the distribution rule of the random noise set is the same as the data distribution rule of the input noise of the degraded image to be restored;
[0115] The minimization difference function is established, and the random noise in the random noise set with the smallest difference from the local noise block is obtained according to the established minimization difference function;
[0116] The random noise in the random noise set with the smallest difference from the local noise block is obtained, and the local noise block that generates the offset is replaced by the random noise.
[0117] In the embodiment of the present application, the random noise set is established to conform to the standard normal distribution rule.
[0118] For each noise block that does not pass the normality test where (i,j) is the coordinate of the center of the block, and generate a set of S random noise samples Z of size k x k from
[0119] select the noise vector in Z that is closest to the original noise block as the matching item, where
[0120]
[0121] by collecting all define then the revised noise sample z * is:
[0122]
[0123] where denotes the element-wise multiplication (Hadamard product).
[0124] It should be noted that the establishment of a random noise set, the acquisition of the random noise most similar to the local noise block, and the replacement of the random noise similar to the local noise block with the local noise block that has been offset can significantly improve the accuracy and efficiency of image restoration. First, through the carefully designed random noise set, it can be ensured that the replaced noise block is consistent with other parts of the original image in statistical characteristics, thereby avoiding image distortion or artifacts caused by improper noise replacement. Secondly, the use of the minimum difference function to select the most similar random noise for replacement can maximize the retention of useful information in the original image while removing or weakening the influence of the offset noise block. Finally, the implementation of this replacement strategy not only improves the quality of image restoration, but also provides more reliable basic data for subsequent image processing steps (such as image enhancement, feature extraction, etc.). In summary, the random noise replacement method proposed in the present application has important application value in the zero-sample blind image restoration system and can significantly improve the usability and recognition accuracy of images.
[0125] S104, according to the noise data after replacement is completed, the degraded image is restored.
[0126] In the embodiments of the present application, according to the noise data after replacement is completed, the degraded image is restored, including:
[0127] inputting the noise data after replacement into an inverse deterministic diffusion model;
[0128] The output of the inverse deterministic diffusion model is the restored degraded image.
[0129] Specifically, the restored image y is obtained by applying a deterministic diffusion model DDIM (inverted relationship with the aforementioned reverse deterministic diffusion model) on the corrected noise sample * = G DDIM (z * ).
[0130] Figure 2 The process diagram of the zero-sample blind image restoration method of the present application. Among them, the upper left corner: the original degraded image, this image shows a worker in the power operation scene, but due to factors such as camera shake, rain and fog obstruction or imaging blur, the image quality has decreased significantly. As an input image, it shows the state before processing, and intuitively reflects the specific performance of image degradation.
[0131] The upper right corner: noise space representation, which is a gray image block, representing the noise space representation obtained by reverse mapping through the DDIM model. It corresponds to a specific area in the original degraded image (marked by a blue box). It shows the form of the degraded image in the noise space, providing a basis for subsequent noise detection and correction.
[0132] The middle row: noise correction process, the image after preliminary processing may have improved part of the image quality through some preprocessing steps.
[0133] The left two: the image after further processing, it can be seen that the image clarity has improved, and the details are more abundant.
[0134] The left three: the final restored image, compared with the original degraded image, the image quality has been significantly improved, and the details and structures are clearer.
[0135] The right one: noise space representation in the middle processing stage, showing the changes of noise in the correction process.
[0136] The right two: the final corrected noise space representation, compared with the initial noise space representation, showing the result of effective noise correction.
[0137] The "noise correction" label at the bottom right corner indicates that the two images on the right are the results of noise correction. It clearly points out that these image blocks show the specific effect of noise correction, helping to understand the key steps of the entire repair process.
[0138] Original degraded image input: starting from the image in the upper left corner, this is the input image that needs to be restored.
[0139] Reverse DDIM to get input noise: reverse map the degraded image to the noise space through the DDIM model to get the noise space representation in the upper right corner.
[0140] Detecting local noise blocks deviating from standard normal distribution: using the sliding window method and statistical hypothesis testing, identify the areas in the noise space that deviate from the standard normal distribution, and construct a binary mask to mark these areas.
[0141] Correcting noise blocks: for noise blocks that do not pass the normality test, a set of random noise is sampled from the standard normal distribution, and the sample closest to the original noise block is selected for replacement, which is reflected in the two rightmost image blocks in the middle row.
[0142] Generating the restored image: input the corrected noise samples into the DDIM model again to generate a restored image with consistent structure but improved quality, and the final result is shown in the left three columns of images in the middle row.
[0143] In summary, the present application proposes a zero-shot blind image restoration method, which acquires a degraded image to be restored and calculates the input noise of the degraded image to be restored; performs a first detection operation on the input noise; establishes a random noise set, obtains random noise similar to the local noise block, and replaces the local noise block with similar random noise to generate an offset; and restores the degraded image according to the noise data after replacement. The step of obtaining the degraded image to be restored and calculating the input noise can accurately locate the noise information in the image, providing a basis for subsequent processing. The first detection operation step can efficiently identify the noise type in the image, providing accurate guidance for subsequent noise replacement. The step of establishing a random noise set and replacing the local noise block that generates an offset improves the flexibility and accuracy of image restoration using the diversity of random noise. The step of restoring the degraded image according to the noise data after replacement realizes effective repair of the degraded image and improves the clarity and visual effect of the image.
[0144] In summary, the method shows good adaptability in handling linear blur, random noise, local occlusion and complex nonlinear degradation, and is particularly suitable for improving the usability and recognition accuracy of power images in complex inspection environments.
[0145] In a preferred embodiment, compared with existing methods, the method does not rely on known degradation models and their parameter forms, and can directly perform adaptive image restoration based on the input degraded observation image, has stronger universality and environmental adaptability, and is particularly suitable for image quality limited restoration tasks in complex scenes such as power inspection.
[0146] To verify the actual effect of the method, the present application carries out systematic tests on multiple typical image restoration tasks, covering structural degradation (such as Gaussian deblurring, 8 times super-resolution) and non-structural degradation (such as additive noise, JPEG artifact removal, rain and raindrop elimination) and other common types. The peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) are used as evaluation indexes of image fidelity, and the perceptual similarity (LPIPS) is introduced to evaluate the visual consistency of the image.
[0147] In particular, to improve the generalization ability and application value of the method in the power scene, 5,358 image samples from the real power operation and maintenance environment are introduced in the experiments of the present application, combined with synthetic degradation for extended evaluation, and the degradation types include motion blur, Gaussian blur, raindrop interference, rainwater shielding, JPEG compression artifact and extreme super-resolution requirement. As shown in Table 1, the method shows better restoration ability than the existing zero-sample blind restoration method on all tasks, especially in the rain and fog shielding and blurred areas in the power grid image, which can effectively reconstruct the structural details, verifying the feasibility and engineering value of the method in the power industry inspection image processing.
[0148] Table 1: Quantitative comparison with the fully blind zero-sample method on various image restoration tasks.
[0149]
[0150]
[0151] The present application is always better than or comparable to the most advanced method in various image restoration tasks. Although there is no prior knowledge of degradation, the present application produces reasonable reconstruction and tends to maintain higher fidelity compared with other competitive methods.
[0152] In the embodiment, a zero-sample blind image restoration system is also provided, which comprises:
[0153] A data acquisition module is configured to acquire a degraded image to be restored and calculate input noise of the degraded image to be restored.
[0154] A detection module is configured to perform a first detection operation on the input noise.
[0155] The first detection operation is configured to detect a local noise block that is offset in the input noise.
[0156] A replacement module is configured to establish a random noise set, acquire a random noise most similar to the local noise block, and replace the local noise block that is offset with the random noise most similar to the local noise block.
[0157] The random noise most similar to the local noise block is acquired by establishing a minimum difference function.
[0158] minimizing the difference function is used to obtain the random noise in the random noise set that has the smallest difference with the local noise block;
[0159] a restoration module, configured to restore the degraded image according to the noise data after the replacement is completed.
[0160] The above modules can be embedded in or independent of the processor in the electronic device in hardware form, or stored in the memory in the electronic device in software form, so as to be called and executed by the processor to perform the operations corresponding to the above modules.
[0161] The embodiment also provides an electronic device, which can be a terminal, and an internal structure diagram of the electronic device can be as shown in Figure 3 The electronic device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. The processor of the electronic device is configured to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The communication interface of the electronic device is configured to perform wired or wireless communication with an external terminal. The wireless communication can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The computer program is executed by the processor to implement a zero-sample blind image restoration method. The display screen of the electronic device can be a liquid crystal display screen or an electronic ink display screen. The input device of the electronic device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the shell of the electronic device, or an external keyboard, touchpad or mouse, etc.
[0162] The embodiment also provides a computer readable storage medium having a computer program stored thereon, and the computer program is executed by the processor to implement the following steps:
[0163] obtaining a degraded image to be restored and calculating input noise of the degraded image to be restored;
[0164] performing a first detection operation on the input noise;
[0165] The first detection operation is configured to detect a local noise block that produces a deviation in the input noise;
[0166] establishing a random noise set, obtaining a random noise most similar to the local noise block, and replacing the local noise block that produces the deviation with the random noise most similar to the local noise block;
[0167] The random noise most similar to the local noise block is obtained by establishing a minimizing difference function;
[0168] minimizing the difference function is used to obtain the random noise in the random noise set that has the minimum difference with the local noise block;
[0169] According to the replaced noise data, the degraded image is restored.
[0170] It should be noted that the above examples are only used to illustrate the technical solutions of the present application but not limit the present application. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced equivalently without departing from the spirit and scope of the present application, and all of them should be covered in the scope of the claims of the present application.
[0171] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can adopt a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can adopt a computer program product in the form of being implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program codes. The solutions in the embodiments of the present application can be implemented in various computer languages.
[0172] The present application is described with reference to flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a means for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that carries out the functions specified in one or more flows and / or blocks.
[0173] These computer program instructions can also be stored in a computer readable storage medium that can direct the computer or other programmable data processing devices to work in a specific manner, so that the instructions stored in the computer readable storage medium produce a product including instruction apparatus, which implements the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that carries out the functions specified in one or more flows and / or blocks.
[0174] These computer program instructions can also be loaded into a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 Figure 1
[0175] Although preferred embodiments of the application have been described herein, it will be apparent to those skilled in the art that various modifications and changes can be made to the embodiments without departing from the spirit and scope of the application. Accordingly, it is intended that all claims be interpreted to include all such modifications and changes as fall within the true spirit and scope of the application.
[0176] Obviously, numerous modifications and variations of the present application are possible in light of the above teachings. It is therefore to be understood that within the scope of the appended claims and their equivalents, the application can be practiced otherwise than as specifically described.
Claims
1. A zero-sample blind image restoration method, characterized in that: include: Acquire a degraded image to be restored, and calculate the input noise of the degraded image to be restored; performing a first detection operation on the input noise; The first detection operation is used to detect a local noise block that generates an offset in the input noise; Establishing a random noise set, obtaining random noise that is most similar to the local noise block, and replacing the offset local noise block with the random noise that is similar to the local noise block; The random noise that is most similar to the local noise block is obtained by establishing a minimization difference function; The minimization difference function is used to obtain the random noise with the smallest difference from the local noise block in the random noise set; The degraded image is restored based on the noise data after replacement.
2. The zero-sample blind image restoration method according to claim 1, wherein: The performing a first detection operation on the input noise includes: A sliding window is preset, wherein the sliding window is used to divide the input noise of the degraded image to be restored into a plurality of noise blocks of the same size as the sliding window; The obtained noise blocks are subjected to a first detection operation, and the noise blocks that do not satisfy the first detection operation are recorded as local noise blocks that generate offsets.
3. The zero-sample blind image restoration method according to claim 2, wherein: The first detection operation includes: Preset offset detection strategy; The offset detection strategy is established based on analyzing the data distribution law that the input noise of the degraded image to be restored conforms to.
4. The zero-sample blind image restoration method according to claim 3, wherein: The establishing of a random noise set, obtaining random noise most similar to the local noise block, and replacing the offset local noise block with the random noise similar to the local noise block comprises: Establishing a random noise set, wherein the distribution law of the random noise set is the same as the data distribution law of the input noise of the degraded image to be restored; Establishing a minimization difference function, and obtaining the random noise with the smallest difference from the local noise block in the random noise set according to the established minimization difference function; The random noise with the smallest difference with the local noise block in the obtained random noise set is used to replace the offset local noise block.
5. The zero-sample blind image restoration method according to claim 4, wherein: Restoring the degraded image according to the noise data after replacement includes: The replaced noise data is input into the inverted deterministic diffusion model; The output of the inverted deterministic diffusion model is the restored degraded image.
6. The zero-sample blind image restoration method according to claim 5, characterized in that: The first detection operation further includes: Classifying and saving noise blocks that meet the first detection operation and noise blocks that do not meet the first detection operation; A binary mask is constructed from the noise blocks that do not satisfy the first detection operation.
7. The zero-sample blind image restoration method according to claim 6, wherein: The acquiring of the degraded image to be restored and calculating the input noise of the degraded image to be restored comprises: Establishing an inverse deterministic diffusion model, and calculating the input noise of the degraded image to be restored according to the inverse deterministic diffusion model; A jump term is introduced into the reverse diffusion process of the inverse deterministic diffusion model.
8. A zero-sample blind image restoration system, applying the method according to any one of claims 1 to 7, characterized in that: include: A data acquisition module, configured to acquire a degraded image to be restored and calculate the input noise of the degraded image to be restored; a detection module, configured to perform a first detection operation on the input noise; The first detection operation is used to detect a local noise block that generates an offset in the input noise; a replacement module, configured to establish a random noise set, obtain random noise that is most similar to the local noise block, and replace the shifted local noise block with the random noise that is similar to the local noise block; The random noise that is most similar to the local noise block is obtained by establishing a minimization difference function; The minimization difference function is used to obtain the random noise with the smallest difference from the local noise block in the random noise set; The restoration module is used to restore the degraded image based on the noise data after replacement.
9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the zero-sample blind image restoration method according to any one of claims 1 to 7 are implemented.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of a zero-sample blind image restoration method according to any one of claims 1 to 7 are implemented.