A method, system, device and medium for blurring image sharpening

Abstract

The application discloses a method, system, device and medium for blurring image sharpening, which comprises the following steps: obtaining an initial blurred image of a target area; performing edge feature extraction on the initial blurred image to obtain an edge feature image; performing blur decoupling processing based on the edge feature image and the initial blurred image to obtain a primary decoupling image; performing coupling enhancement processing based on the primary decoupling image and the edge feature image to obtain a target feature image; and fusing the target feature image and the initial blurred image to obtain a target clear image of the target area. The method solves the problem of low image usability caused by poor image sharpness of the images directly collected by the road monitoring system in the prior art.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method, system, device and medium for sharpening blurred images. Background Technology

[0002] Currently, road surveillance images face serious blurring issues: motion blur caused by high-speed vehicles or camera shake in dynamic scenes, out-of-focus blur due to camera focusing errors or shallow depth of field, and image blurring caused by sensor noise and hardware limitations in video transmission. To improve the clarity and usability of road surveillance system images, a technology capable of effectively restoring the quality of blurred images is urgently needed. Summary of the Invention

[0003] To overcome the problem of poor image clarity and low image usability caused by the direct acquisition of images by existing road monitoring systems, this application provides a method, system, device, and medium for blurry image sharpening.

[0004] Firstly, in order to solve the above-mentioned technical problems, this application provides a method for sharpening blurred images, including:

[0005] Obtain the initial blurred image of the target region;

[0006] Edge features are extracted from the initial blurred image to obtain an edge feature image;

[0007] A primary decoupling image is obtained by performing fuzzy decoupling processing based on the edge feature image and the initial blurred image.

[0008] The target feature image is obtained by performing coupling enhancement processing based on the primary decoupled image and the edge feature image;

[0009] The target feature image and the initial blurred image are fused to obtain a clear image of the target region.

[0010] Furthermore, edge features are extracted from the initial blurred image to obtain an edge feature image, including:

[0011] The initial blurred image is reconstructed at low resolution to obtain the reconstructed image;

[0012] The initial blurred image is convolved to obtain the guiding image;

[0013] Edge feature images are obtained by performing trilinear interpolation on the reconstructed image and the guiding image.

[0014] Furthermore, the initial blurred image is reconstructed at low resolution to obtain a reconstructed image, including:

[0015] The initial blurred image is downsampled to obtain a low-resolution blurred image;

[0016] Feature extraction is performed on low-resolution blurred images to obtain low-level feature images;

[0017] The low-level feature image is reconstructed using a bilateral network to obtain the reconstructed image.

[0018] Furthermore, based on the edge feature image and the initial blurred image, fuzzy decoupling processing is performed to obtain a primary decoupled image, including:

[0019] The initial blurred image is subjected to multi-scale encoding processing to obtain a multi-scale feature image;

[0020] Based on the preset high-frequency decoding rules, multi-scale feature images and edge feature images are processed to obtain primary high-frequency feature images;

[0021] Based on the preset low-frequency decoding rules, the multi-scale feature image and the initial blurred image are processed to obtain the primary low-frequency feature image;

[0022] A primary decoupled image is formed based on the primary high-frequency feature image and the primary low-frequency feature image.

[0023] Furthermore, coupling enhancement processing is performed based on the primary decoupled image and the edge feature image to obtain the target feature image, including:

[0024] Coupled learning processing is performed based on the primary decoupled image and the edge feature image to obtain the intermediate feature image, which includes the intermediate high-frequency feature image and the intermediate low-frequency feature image.

[0025] Feature enhancement processing is performed based on intermediate feature images and primary decoupled images to obtain target feature images, which include target high-frequency feature images and target low-frequency feature images.

[0026] Furthermore, the primary decoupling image includes a primary high-frequency feature image and a primary low-frequency feature image; based on the primary decoupling image and the edge feature image, coupling learning processing is performed to obtain the intermediate feature image, including:

[0027] Based on the preset high-frequency coupling learning rules, the primary high-frequency feature image, primary low-frequency feature image, and edge feature image are processed to obtain the intermediate high-frequency feature image.

[0028] Based on the preset low-frequency coupling learning rules, the primary high-frequency feature image, primary low-frequency feature image and edge feature image are processed to obtain the intermediate low-frequency feature image.

[0029] An intermediate feature image is formed based on intermediate high-frequency feature images and intermediate low-frequency feature images.

[0030] Furthermore, the primary decoupling image includes a primary high-frequency feature image and a primary low-frequency feature image; feature enhancement processing is performed based on the intermediate feature image and the primary decoupling image to obtain the target feature image, including:

[0031] Based on the preset high-frequency feature enhancement rules, the intermediate high-frequency feature image and the primary high-frequency feature image are processed to obtain the target high-frequency feature image;

[0032] The intermediate low-frequency feature image and the primary low-frequency feature image are processed based on the preset low-frequency feature enhancement rules to obtain the target low-frequency feature image.

[0033] A target feature image is formed based on the target's high-frequency feature image and target's low-frequency feature image.

[0034] Secondly, this application also provides a system for sharpening blurred images, comprising:

[0035] The acquisition module is used to acquire the initial blurred image of the target area;

[0036] The edge feature extraction module is used to extract edge features from the initial blurred image to obtain an edge feature image;

[0037] The fuzzy decoupling module is used to perform fuzzy decoupling processing based on the edge feature image and the initial fuzzy image to obtain the primary decoupling image;

[0038] The coupling enhancement module is used to perform coupling enhancement processing based on the primary decoupled image and the edge feature image to obtain the target feature image;

[0039] The fusion module is used to fuse the target feature image and the initial blurred image to obtain a clear image of the target region.

[0040] Thirdly, this application also provides a computing device, including a memory, a processor, and a program stored in the memory and running on the processor, wherein the processor executes the program to implement the steps of the above-described method for sharpening a blurred image.

[0041] Fourthly, this application also provides a computer-readable storage medium storing instructions that, when executed on a terminal device, cause the terminal device to perform the steps of a blurry image sharpening method.

[0042] The beneficial effects of this application are as follows: First, edge features are extracted from the initial blurred image of the target region, enabling the obtained edge feature image to highlight key features such as texture details and color. Then, blur decoupling processing is performed based on the edge feature image and the initial blurred image, allowing the resulting primary decoupling image to improve deblurring performance while preserving key features. Second, coupling enhancement processing is performed based on the primary decoupling image and the edge feature image to eliminate redundant feature components in the image, amplify and refine key features, and obtain a target feature image with clear key features. Finally, the target feature image and the initial blurred image are fused to clearly enhance the key features in the initial blurred image, ensuring that the obtained clear target image of the target region meets visual clarity requirements, thereby improving the usability of the clear target image. Attached Figure Description

[0043] Figure 1 This is a flowchart illustrating an exemplary embodiment of a method for sharpening a blurred image, as shown in this application.

[0044] Figure 2 This is a logic diagram of edge feature extraction processing in an exemplary embodiment of this application;

[0045] Figure 3 This is a logic diagram of fuzzy decoupling processing in an exemplary embodiment of this application;

[0046] Figure 4 This is a coupled learning processing logic diagram in an exemplary embodiment of this application;

[0047] Figure 5 This is a feature enhancement processing logic diagram in an exemplary embodiment of this application;

[0048] Figure 6 This is a schematic flowchart illustrating the application of the provided blurry image sharpening method in an exemplary embodiment of this application;

[0049] Figure 7 Comparison of deblurring effects from existing technologies;

[0050] Figure 8 This image shows a comparison of the blurring effect of the image sharpening method of this application with that of existing technologies.

[0051] Figure 9 This is a schematic diagram illustrating the structure of a blurred image sharpening system, which is an exemplary embodiment of this application. Detailed Implementation

[0052] The following embodiments are further explanations and supplements to this application and do not constitute any limitation on this application.

[0053] With the development of deep learning technology, convolutional neural networks have shown great potential in the field of blurry image sharpening. However, traditional deep learning-based image deblurring methods typically employ an end-to-end mapping approach, directly predicting the corresponding sharp image from the blurry image using a neural network. However, in the global image restoration process, directly modeling this highly nonlinear "blur-sharp" mapping relationship leads to model redundancy, causing a mismatch between model capacity and task complexity, severely impairing deblurring performance. Ultimately, this results in difficulty in restoring detailed textures and color features, failing to meet the needs of modern intelligent road monitoring systems.

[0054] To address the aforementioned problems, embodiments of this application provide a method, system, device, and medium for sharpening blurred images, which will be described in detail below.

[0055] The method for sharpening blurred images provided in this application can be specifically executed by a server. It should be noted that the server can be an independent server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms. No limitation is imposed here.

[0056] Please see Figure 1 , Figure 1 A method for sharpening a blurred image is illustrated in an exemplary embodiment of this application, such as... Figure 1 As shown, this application provides a method for sharpening a blurred image, including:

[0057] S11, Obtain the initial blurred image of the target area;

[0058] S12, extract edge features from the initial blurred image to obtain an edge feature image;

[0059] S13, perform fuzzy decoupling processing based on edge feature image and initial fuzzy image to obtain primary decoupling image;

[0060] S14, based on the primary decoupled image and the edge feature image, a coupling enhancement process is performed to obtain the target feature image;

[0061] S15, fuse the target feature image and the initial blurred image to obtain a clear image of the target region.

[0062] The blurry image sharpening method provided in this application first extracts edge features from the initial blurry image of the target region, enabling the obtained edge feature image to highlight key features such as texture details and color. Then, blur decoupling processing is performed on the edge feature image and the initial blurry image, allowing the resulting primary decoupling image to improve deblurring performance while preserving key features. Next, coupling enhancement processing is performed on the primary decoupling image and the edge feature image to eliminate redundant feature components in the image, amplify and refine key features, and obtain a target feature image with clear key features. Finally, the target feature image and the initial blurry image are fused to further enhance the clarity of key features in the initial blurry image, ensuring that the obtained sharp image of the target region meets visual clarity requirements, thereby improving the usability of the sharp image.

[0063] In an exemplary embodiment provided in this application, a target feature image and an initial blurred image are fused to obtain a clear target image of the target region, using the following formula:

[0064] ;

[0065] in, Indicates a clear image of the target. Represents the initial blurred image. Represents the high-frequency feature image of the target. Represents the low-frequency feature image of the target. This indicates the addition of residuals. This forms a global residual structure.

[0066] Optionally, edge features are extracted from the initial blurred image to obtain an edge feature image, including:

[0067] The initial blurred image is reconstructed at low resolution to obtain the reconstructed image;

[0068] The initial blurred image is convolved to obtain the guiding image;

[0069] An edge feature image is obtained by performing trilinear interpolation on the reconstructed image and the guiding image.

[0070] The formula for calculating edge feature images is as follows:

[0071] ;

[0072] in, Represents edge feature image, Indicates the guide image Perform trilinear interpolation. This indicates that the image has been reshaped.

[0073] In the embodiment provided in this application, firstly, a reconstructed image is obtained by performing low-resolution reshaping on the initial blurred image to amplify the edge differences in texture and color, thereby enhancing the ability to distinguish edge information in the image. Secondly, the initial blurred image is convolved to highlight the boundaries of objects in the image, resulting in a guiding image. Then, trilinear interpolation is performed based on the reconstructed image and the guiding image, so that the resulting edge feature image has significant differences in texture and color, and obvious object boundaries, thereby highlighting key features such as texture details and color, achieving accurate restoration of image texture details and color, thus reducing the difficulty of subsequent enhancement of key features such as texture details and color in the image. This not only improves the deblurring efficiency of the initial blurred image, but also improves the sharpness of the final target clear image, thereby improving the usability of the target clear image.

[0074] Optionally, the initial blurred image is reconstructed at a low resolution to obtain a reconstructed image, including:

[0075] The initial blurred image is downsampled to obtain a low-resolution blurred image;

[0076] Feature extraction is performed on low-resolution blurred images to obtain low-level feature images;

[0077] The low-level feature image is reconstructed using a bilateral network to obtain the reconstructed image;

[0078] The formula for reshaping an image is as follows:

[0079] ;

[0080] in, This represents the reconstructed image, also known as the affine bilateral grid coefficients. d=6 represents the grid depth, and c=6 represents that each grid cell in the two-sided network contains 6 coefficients. Indicates the grid depth; , indicating a horizontal spatial position; Indicates vertical spatial position. The depth dimension of the mesh (i.e., the first dimension of the bilateral network) represents the depth dimension of the mesh. (one grid layer) Represents the first [unit] in each grid cell of the bilateral network. One coefficient, This represents a low-level feature image.

[0081] In the embodiment provided in this application, firstly, by processing the initial blurred image... Downsampling is performed to obtain a low-resolution blurred image. This reduces subsequent computational complexity. Secondly, two convolutional blocks are used to process the low-resolution blurred image. Feature extraction is performed to extract low-level features from the image, resulting in a learnable low-level feature image. Finally, a bilateral network is used to reconstruct the low-level feature image to reconstruct high-frequency details in the image and amplify the edge differences in texture and color. This results in a reconstructed image that enhances the ability to distinguish edge information, facilitating accurate restoration of image texture details and colors in subsequent steps. This improves the sharpness of the resulting target image and enhances its usability.

[0082] The goal of the bilateral network reshaping operation is to rearrange the 36 channels of the low-level feature image in a 6×6 structure to form a new four-dimensional tensor.

[0083] In this embodiment, considering that the pixel intensities of edge regions in blurred images are often highly similar in spatial dimension, making it difficult for conventional convolutional neural networks to effectively distinguish their subtle differences, this embodiment uses low-resolution reshaping to map the initial two-dimensional blurred image into three-dimensional space. In this three-dimensional space, Euclidean distance imparts edge perception capability. In this way, edge differences are amplified, achieving accurate restoration of image texture details and colors, and promoting the reconstruction of edge features, thereby enhancing the ability to distinguish edge information in the reshaped image.

[0084] Please see Figure 2 , Figure 2 This is a logic diagram of edge feature extraction processing in an exemplary embodiment of this application, such as... Figure 2 As shown, the specific logic for edge feature extraction is as follows:

[0085] Initial blurred image at full resolution A downsampling process is performed to obtain a low-resolution blurred image;

[0086] Feature extraction is performed on low-resolution blurred images to obtain low-level feature images;

[0087] The low-level feature image is reconstructed using a bilateral network to obtain the reconstructed image. (Affine bilateral grid coefficients);

[0088] The initial blurred image is obtained through two convolutional layers and a ReLU activation function. Processing is performed to obtain the guide image. ;

[0089] Trilinear interpolation is performed based on the reconstructed image and the guide image, by searching for grid positions in the reconstructed image. Then, trilinear interpolation is used to extract edge features, resulting in an edge feature image. (High-frequency feature output image generated by the high-frequency reconstruction unit).

[0090] Optionally, blur decoupling processing is performed based on the edge feature image and the initial blurred image to obtain a primary decoupled image, including:

[0091] The initial blurred image is subjected to multi-scale encoding processing to obtain a multi-scale feature image;

[0092] Based on the preset high-frequency decoding rules, multi-scale feature images and edge feature images are processed to obtain primary high-frequency feature images;

[0093] Based on the preset low-frequency decoding rules, the multi-scale feature image and the initial blurred image are processed to obtain the primary low-frequency feature image;

[0094] A primary decoupled image is formed based on the primary high-frequency feature image and the primary low-frequency feature image.

[0095] In the embodiment provided in this application, a multi-scale feature image is obtained by performing multi-scale encoding processing on the initial blurred image. Based on preset high-frequency and low-frequency decoding rules, the multi-scale feature image, edge feature image, and initial blurred image are then processed to obtain complementary primary high-frequency feature images with high-frequency texture residual features and primary low-frequency feature images with low-frequency color residual features, forming a primary decoupled image. In this way, the obtained primary high-frequency and primary low-frequency feature images can autonomously learn the texture and color features within their respective domains. This differentiated learning method reduces the complexity of learning, thereby improving the deblurring performance of blurred textures and colors during subsequent sharpening based on the primary decoupled image. This improves the sharpness of the final target sharp image, thus enhancing its usability.

[0096] In an exemplary embodiment provided in this application, the preset high-frequency decoding rule is as follows: after sharpening the multi-scale feature image, it is input into the first decoder D1 for processing to learn the high-frequency texture features in the multi-scale feature image and generate a primary high-frequency feature image with high-frequency texture residual features.

[0097] The preset low-frequency decoding rule is as follows: the multi-scale feature image and the initial blurred image are directly input into the second decoder D2 for processing, so as to learn the low-frequency color features corresponding to the basic information such as color, texture details, and contours in the multi-scale feature image, and generate a primary low-frequency feature image with low-frequency color residual features.

[0098] Please see Figure 3 , Figure 3This is a logic diagram of fuzzy decoupling processing in an exemplary embodiment of this application, such as... Figure 3 As shown, the specific logic of fuzzy decoupling processing is as follows:

[0099] First, the initial blurred image is processed by an encoder. Multi-scale encoding is performed to obtain multi-scale feature images in the form of multi-head attention. This is shared by the first encoder D1 and the second decoder D2. The specific processing procedure in the encoder can be represented as follows:

[0100] ;

[0101] ;

[0102] in, This represents the scale feature map output by the j-th layer scale network of the encoder. This represents the convolution operation at the j-th layer. This indicates downsampling, and n represents the number of encoder layers.

[0103] Secondly, the multi-scale feature image and the edge feature image obtained by the high-frequency reconstruction unit are input into the first decoder D1 for processing.

[0104] For each decoding layer of the first decoder D1, perform the following steps:

[0105] ;

[0106] in, This represents the high-frequency texture features output by the first decoder D1 at the j-th decoding layer. This represents aligned convolution, where G represents the edge feature image. This represents the scale feature map output by the j-th layer scale network of the sharpened encoder, where G's channels and scale are adapted to... Consistent;

[0107] Multiple high-frequency texture features are residually connected to obtain a primary high-frequency feature image.

[0108] Next, the multi-scale feature image and the initial blurred image are input into the second decoder D2 for processing.

[0109] For each decoding layer of the second decoder D2, perform the following steps:

[0110] ;

[0111] in, This represents the low-frequency color characteristics output by the second decoder D2 at the j-th decoding layer. This indicates bilateral filtering. Represents the initial blurred image. Indicates downsampling, This represents the scale feature map output by the j-th layer scale network of the encoder;

[0112] By performing residual connections on multiple low-frequency color features, a primary low-frequency feature image is obtained.

[0113] In summary, the first decoder D2 and the second decoder D2 share parameters. The main goal of the parameter sharing strategy between the two decoders is to initially decouple the fuzzy features while maintaining a parameter count comparable to that of the traditional single-decoder method.

[0114] Optionally, coupling enhancement processing is performed based on the primary decoupled image and the edge feature image to obtain the target feature image, including:

[0115] Coupled learning processing is performed based on the primary decoupled image and the edge feature image to obtain the intermediate feature image, which includes the intermediate high-frequency feature image and the intermediate low-frequency feature image.

[0116] Feature enhancement processing is performed based on intermediate feature images and primary decoupled images to obtain target feature images, which include target high-frequency feature images and target low-frequency feature images.

[0117] In the embodiment provided in this application, coupling learning processing based on the primary decoupled image and the edge feature image can eliminate redundant feature components of texture and color in the image, resulting in an intermediate feature image. Feature enhancement processing based on the intermediate feature image and the primary decoupled image can amplify and refine key features such as texture and color in the image, resulting in a target feature image with clear key features. This facilitates subsequent enhancement of key features of the initial blurred image based on the target feature image, ensuring that the obtained clear target image of the target region meets the visual clarity requirements, thereby improving the usability of the clear target image.

[0118] Optionally, the primary decoupling image includes a primary high-frequency feature image and a primary low-frequency feature image; based on the primary decoupling image and the edge feature image, coupling learning processing is performed to obtain an intermediate feature image, including:

[0119] Based on the preset high-frequency coupling learning rules, the primary high-frequency feature image, primary low-frequency feature image, and edge feature image are processed to obtain the intermediate high-frequency feature image.

[0120] The formula for calculating intermediate-level high-frequency feature images is as follows:

[0121] ;

[0122] in; Represents a mid-to-high frequency feature image. Represents the primary high-frequency feature image. This represents a primary low-frequency feature image. Represents edge feature image, This indicates that the simplified spatial attention layer is used to... High-frequency attention mask generated from learned high-frequency feature maps This indicates that the channel attention layer (CA) is used to... The low-frequency feature map learned is normalized to [0, 1] using the Sigmoid function to generate a low-frequency attention mask;

[0123] Based on the preset low-frequency coupling learning rules, the primary high-frequency feature image, primary low-frequency feature image and edge feature image are processed to obtain the intermediate low-frequency feature image.

[0124] The formula for calculating the mid-to-low frequency feature image is as follows:

[0125] ;

[0126] in, Represents a mid-to-low frequency feature image;

[0127] An intermediate feature image is formed based on intermediate high-frequency feature images and intermediate low-frequency feature images.

[0128] In the embodiment provided in this application, the initial high-frequency feature image, the primary low-frequency feature image, and the edge feature image are processed based on preset high-frequency coupling learning rules and low-frequency coupling learning rules. This can eliminate redundant high-frequency texture feature components and low-frequency color feature components in the image, and reduce invalid features in the obtained intermediate high-frequency feature image and intermediate low-frequency feature image. This can reduce the enhancement and interference of invalid features in the subsequent image sharpening process based on the intermediate high-frequency feature image and intermediate low-frequency feature image, thereby improving the accuracy of the final target sharp image and further improving the usability of the target sharp image.

[0129] In this embodiment, through and Redundant and blurry components were cleverly removed.

[0130] Please see Figure 4 , Figure 4 This is a coupled learning processing logic diagram in an exemplary embodiment of this application, such as... Figure 4 As shown, the specific logic of Coupled Learning Processing (CLM) is as follows:

[0131] Based on the preset high-frequency coupling learning rules, the primary high-frequency feature image is processed. Primary low-frequency feature images Edge feature images obtained by high-frequency reconstruction unit processing Processing is performed to obtain intermediate-frequency feature images. ;

[0132] Based on the preset low-frequency coupling learning rule, the primary high-frequency feature image is processed. Primary low-frequency feature images and edge feature images Processing is performed to obtain a mid-to-low frequency feature image. .

[0133] Figure 4 Each processing node in the process is represented by an icon of a different color, including: Convolution, Sigmoid activation function, Transposed Convolution, Strided Convolution, Parametric ReLU (PReLU), Pixel-wise Summation, and Pixel-wise Product.

[0134] Coupled Learning Processing (CLM) is used to eliminate redundant feature components and generate more realistic color features from the decoupled primary high-frequency feature image t1 and primary low-frequency feature image t2, thereby reducing the learning difficulty and improving network performance. CLM uses a divide-and-conquer approach, dividing into two branches to extract features of different frequencies. Specifically, CLM encodes the mixture relationships (complementary and redundant components) and returns the intermediate high-frequency feature image through coupled learning. and mid-to-low frequency feature images The fine joint representation is obtained. Because early designs of dual decoders allowed the network to decompose the intrinsic information of encoded features into fuzzy features in different domains without explicit constraints, this led to significant differences in fuzziness intensity between different domains. Therefore, this embodiment designs an asymmetric structure for the CLM to handle different fuzziness intensities of the dual decoders separately. In this way, the number of network parameters can be effectively reduced while generating more realistic color features and extracting high-frequency feature images from mid-range. and mid-to-low frequency feature images Eliminating redundant components in the accurate joint representation enables the reduction of subsequent processing based on intermediate-frequency feature images. and mid-to-low frequency feature images The process of enhancing and interfering with invalid features during image sharpening aims to improve the accuracy of the final sharp image and further enhance its usability.

[0135] Optionally, the primary decoupling image includes a primary high-frequency feature image and a primary low-frequency feature image; feature enhancement processing is performed based on the intermediate feature image and the primary decoupling image to obtain the target feature image, including:

[0136] Based on the preset high-frequency feature enhancement rules, the intermediate high-frequency feature image and the primary high-frequency feature image are processed to obtain the target high-frequency feature image;

[0137] The formula for calculating the high-frequency feature image of the target is as follows:

[0138] ;

[0139] in, Represents the high-frequency feature image of the target. This represents a 3×3 convolutional layer. Represents the primary high-frequency feature image. Represents a mid-to-high frequency feature image. This represents the learnable operators obtained from the channel attention layer;

[0140] The intermediate low-frequency feature image and the primary low-frequency feature image are processed based on the preset low-frequency feature enhancement rules to obtain the target low-frequency feature image.

[0141] The formula for calculating the low-frequency feature image of the target is as follows:

[0142] ;

[0143] in, Represents the low-frequency feature image of the target. This represents a primary low-frequency feature image. Represents a mid-to-low frequency feature image;

[0144] A target feature image is formed based on the target's high-frequency feature image and target's low-frequency feature image.

[0145] In the embodiment provided in this application, firstly, the intermediate high-frequency feature image and the primary high-frequency feature image are processed based on preset high-frequency feature enhancement rules. This process amplifies and refines the key high-frequency texture features in the image, resulting in a target high-frequency feature image with clear key high-frequency texture features. Secondly, the intermediate low-frequency feature image and the primary low-frequency feature image are processed based on preset low-frequency feature enhancement rules. This process amplifies and refines the key low-frequency color features in the image, resulting in a target low-frequency feature image with clear key low-frequency color features. This facilitates subsequent enhancement of the initial blurred image's high-frequency texture and low-frequency color based on the target high-frequency feature image and the target low-frequency feature image. The resulting clear target image of the target region meets visual clarity requirements at both the high-frequency and low-frequency thresholds, further improving the usability of the clear target image.

[0146] Please see Figure 5 , Figure 5 This is a feature enhancement processing logic diagram in an exemplary embodiment of this application, such as... Figure 5 As shown, the specific logic of feature enhancement processing is as follows:

[0147] Based on preset high-frequency feature enhancement rules, the mid-level high-frequency feature image is processed. and primary high-frequency feature images The process yields a high-frequency feature image of the target with high-frequency texture features. ;

[0148] Based on the preset low-frequency feature enhancement rules, the mid-to-low-frequency feature image is processed. and primary low-frequency feature images The process yields a target low-frequency feature image with low-frequency color characteristics. .

[0149] Figure 5 The various processing nodes are represented by icons of different colors, including: Convolution, Adaptive Average Pooling, Parametric Corrected Linear Unit (PReLU), Pixel-wise Summation, Pixel-wise Product, and Sigmoid activation function.

[0150] In an exemplary embodiment provided in this application, the blurry image sharpening method of this application is implemented using PyTorch 1.7. Based on the blurry image sharpening method of this application, a blurry network model can be constructed, in which the following steps are performed:

[0151] Edge features are extracted from the initial blurred image to obtain an edge feature image;

[0152] A primary decoupling image is obtained by performing fuzzy decoupling processing based on the edge feature image and the initial blurred image.

[0153] The target feature image is obtained by performing coupling enhancement processing based on the primary decoupled image and the edge feature image;

[0154] The target feature image and the initial blurred image are fused to obtain a clear image of the target region.

[0155] Fuzzy network model training settings: The Adam (Adaptive Moment Estimation) optimizer was used for a total of 381 epochs. The parameters in the Adam optimizer included a first coefficient β1 = 0.9, a second coefficient β2 = 0.9, and weight decay of 0. The initial learning rate of the model was 2e. -4 As the cosine annealing strategy gradually decreases to 1e -7 Where e can specifically take the value 10. During training, the patch size is set to 256×256, and the patch size is set to 4. In this embodiment, the performance / memory / computational complexity evaluation of the fuzzy network model is performed on a 12G NVIDIA 3060 GPU.

[0156] Please see Figure 6 , Figure 6 This is a schematic flowchart illustrating the application of the provided blurry image sharpening method in an exemplary embodiment of this application. The specific steps are as follows:

[0157] High-frequency feature reconstruction unit: extracts edge features from the initial blurred image to obtain an edge feature image;

[0158] Fuzzy feature decoupling: Based on the edge feature image and the initial fuzzy image, fuzzy decoupling processing is performed to obtain the primary high-frequency feature image and the primary low-frequency feature image, forming the primary decoupled image;

[0159] Coupled learning module: Based on the intermediate feature image and the primary decoupled image, feature enhancement processing is performed to obtain the target high-frequency feature image and the target low-frequency feature image, forming the target feature image;

[0160] Feature enhancement module: Based on the primary decoupled image and edge feature image, coupling learning processing is performed to obtain intermediate high-frequency feature image and intermediate low-frequency feature image, forming intermediate feature image;

[0161] Finally, the target feature image and the initial blurred image are fused to obtain the target deblurred image (target clear image) of the target region.

[0162] Thus, the method of this application establishes a coupled learning framework through a convolutional neural network, which can autonomously learn the differences and intrinsic interaction between low-frequency features (content and color information) and high-frequency features (edge ​​and texture detail information), and eliminate redundant features. This effectively solves the problems of lost texture details and color distortion in the restored image, thereby obtaining a clear target image with texture details and color features that meet the requirements after image restoration while maintaining high efficiency, thereby improving the usability of the clear target image.

[0163] Please see Figure 7 and Figure 8 The image shown is a comparison of the blurring effect of the proposed image sharpening method with that of existing technologies. Figure 7 and Figure 8 As shown, Figure 7 Figure (a) shows the initial blurred image acquired. Figure 7 Figure (b) shows the image obtained by sharpening the initial blurred image using the existing MRDNet (Multi-scale Residual Deblurring Network). Figure 8 Figure (c) shows the image obtained by sharpening the initial blurred image using the existing TBLNet (Two-Stage Bilateral Learning Network). Figure 8 Figure (d) shows the image obtained by sharpening the initial blurred image using the blurry image sharpening method of this application. Analysis of Figures (a), (b), (c), and (d) shows that, compared with existing sharpening methods, the blurred image sharpening method of this application produces the sharpest image.

[0164] Please see Figure 9 , Figure 9 An exemplary embodiment of this application illustrates a system for sharpening blurred images, such as... Figure 9 As shown, this application provides a blurred image sharpening system 800, including:

[0165] The acquisition module 901 is used to acquire the initial blurred image of the target area;

[0166] The edge feature extraction module 902 is used to extract edge features from the initial blurred image to obtain an edge feature image;

[0167] The fuzzy decoupling module 903 is used to perform fuzzy decoupling processing based on the edge feature image and the initial fuzzy image to obtain the primary decoupling image;

[0168] The coupling enhancement module 904 is used to perform coupling enhancement processing based on the primary decoupled image and the edge feature image to obtain the target feature image;

[0169] The fusion module 905 is used to fuse the target feature image and the initial blurred image to obtain a clear image of the target region.

[0170] The blurry image sharpening system 900 of this application first uses an edge feature extraction module 902 to extract edge features from the initial blurry image of the target region acquired by the acquisition module 901, so that the obtained edge feature image can highlight key features such as texture details and color. Then, a blur decoupling module 903 performs blur decoupling processing based on the edge feature image and the initial blurry image, so that the obtained primary decoupling image can improve the deblurring performance while retaining key features. Second, a coupling enhancement module 904 performs coupling enhancement processing based on the primary decoupling image and the edge feature image to eliminate redundant feature components in the image, amplify and refine key features in the image, and obtain a target feature image with clear key features. Then, a fusion module 905 fuses the target feature image and the initial blurry image to enhance the clarity of key features in the initial blurry image, so that the obtained clear target image of the target region can meet the visual clarity requirements, thereby improving the usability of the clear target image.

[0171] Optionally, the edge feature extraction module 902 is specifically used for:

[0172] The initial blurred image is reconstructed at low resolution to obtain the reconstructed image;

[0173] The initial blurred image is convolved to obtain the guiding image;

[0174] Edge feature images are obtained by performing trilinear interpolation on the reconstructed image and the guiding image.

[0175] Optionally, the edge feature extraction module 902 is specifically used for:

[0176] The initial blurred image is downsampled to obtain a low-resolution blurred image;

[0177] Feature extraction is performed on low-resolution blurred images to obtain low-level feature images;

[0178] The low-level feature image is reconstructed using a bilateral network to obtain the reconstructed image.

[0179] Optionally, the fuzzy decoupling module 903 is specifically used for:

[0180] The initial blurred image is subjected to multi-scale encoding processing to obtain a multi-scale feature image;

[0181] Based on the preset high-frequency decoding rules, multi-scale feature images and edge feature images are processed to obtain primary high-frequency feature images;

[0182] Based on the preset low-frequency decoding rules, the multi-scale feature image and the initial blurred image are processed to obtain the primary low-frequency feature image;

[0183] A primary decoupled image is formed based on the primary high-frequency feature image and the primary low-frequency feature image.

[0184] Optionally, the coupling enhancement module 904 is specifically used for:

[0185] Coupled learning processing is performed based on the primary decoupled image and the edge feature image to obtain the intermediate feature image, which includes the intermediate high-frequency feature image and the intermediate low-frequency feature image.

[0186] Feature enhancement processing is performed based on intermediate feature images and primary decoupled images to obtain target feature images, which include target high-frequency feature images and target low-frequency feature images.

[0187] Optionally, the primary decoupling image includes a primary high-frequency feature image and a primary low-frequency feature image; the coupling enhancement module 904 is specifically used for:

[0188] Based on the preset high-frequency coupling learning rules, the primary high-frequency feature image, primary low-frequency feature image, and edge feature image are processed to obtain the intermediate high-frequency feature image.

[0189] Based on the preset low-frequency coupling learning rules, the primary high-frequency feature image, primary low-frequency feature image and edge feature image are processed to obtain the intermediate low-frequency feature image.

[0190] An intermediate feature image is formed based on intermediate high-frequency feature images and intermediate low-frequency feature images.

[0191] Optionally, the primary decoupling image includes a primary high-frequency feature image and a primary low-frequency feature image; the coupling enhancement module 904 is specifically used for:

[0192] Based on the preset high-frequency feature enhancement rules, the intermediate high-frequency feature image and the primary high-frequency feature image are processed to obtain the target high-frequency feature image;

[0193] The intermediate low-frequency feature image and the primary low-frequency feature image are processed based on the preset low-frequency feature enhancement rules to obtain the target low-frequency feature image.

[0194] A target feature image is formed based on the target's high-frequency feature image and target's low-frequency feature image.

[0195] It should be noted that the blurred image sharpening system and the blurred image sharpening method provided in the above embodiments belong to the same concept. The specific ways in which each module and unit performs its operations have been described in detail in the method embodiments and will not be repeated here. In practical applications, the blurred image sharpening system provided in the above embodiments can be assigned to different functional modules as needed, that is, the internal structure of the system can be divided into different functional modules to complete all or part of the functions described above. This is not a limitation here.

[0196] A computing device according to an embodiment of this application includes a memory, a processor, and a program stored in the memory and running on the processor. When the processor executes the program, it implements some or all of the steps of the above-described method for sharpening a blurred image.

[0197] The computing device can be a computer, and the corresponding program is computer software. The parameters and steps of the computing device described above in this application can be referred to the parameters and steps in the embodiment of the blurry image sharpening method above, and will not be repeated here.

[0198] This application embodiment provides a computer-readable storage medium storing instructions that, when executed, perform the steps of the aforementioned method for sharpening a blurred image.

[0199] The computer-readable storage medium may be a transient computer-readable storage medium or a non-transitory computer-readable storage medium.

[0200] The technical solutions of this disclosure can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes one or more instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of this disclosure. The aforementioned computer-readable storage medium can be a non-transitory computer-readable storage medium, including: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks, and other media capable of storing program code; it can also be a transient computer-readable storage medium.

[0201] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0202] Those skilled in the art will recognize that this application can be implemented as a system, method, or computer program product. Therefore, this disclosure can be implemented in the following forms: it can be entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software, generally referred to herein as a "module" or "system." Furthermore, in some embodiments, this application can also be implemented as a computer program product contained in one or more computer-readable media, which contains computer-readable program code. Computer-readable storage media can be, for example, but not limited to—electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof.

[0203] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0204] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A method of deblurring an image, characterized by, The method comprises the following steps: obtaining an initial blurred image of a target region; down-sampling the initial blurred image to obtain a low-resolution blurred image; extracting features from the low-resolution blurred image to obtain a low-level feature image; remodeling the low-level feature image through a bilateral network to obtain a remodeled image, and the corresponding remodeling formula is as follows: ； wherein, represents a remodeled image, which can also be referred to as an affine bilateral grid coefficient, , d = 6 represents a grid depth, and c = 6 represents that each grid cell in the bilateral network contains 6 coefficients, represents a grid depth; represents a horizontal spatial position; represents a vertical spatial position, represents a depth dimension of the grid (i.e., the th grid layer of the bilateral network), represents the th coefficient within each grid cell in the bilateral network, represents a low-level feature image; performing convolution processing on the initial blurred image to obtain a guide image; performing trilinear interpolation processing on the remodeled image and the guide image to obtain an edge feature image; performing blur decoupling processing on the edge feature image and the initial blurred image to obtain a primary decoupled image; performing coupling enhancement processing on the primary decoupled image and the edge feature image to obtain a target feature image; fusing the target feature image and the initial blurred image to obtain a target clear image of the target region.

2. The method of claim 1, wherein, The method comprises the following steps: performing multi-scale coding processing on the initial blurred image to obtain a multi-scale feature image; processing the multi-scale feature image and the edge feature image based on a preset high-frequency decoding rule to obtain a primary high-frequency feature image; processing the multi-scale feature image and the initial blurred image based on a preset low-frequency decoding rule to obtain a primary low-frequency feature image; forming a primary decoupled image based on the primary high-frequency feature image and the primary low-frequency feature image.

3. The method according to claim 1 or 2, characterized in that, The method comprises the following steps: performing coupling learning processing on the primary decoupled image and the edge feature image to obtain an intermediate feature image, the intermediate feature image comprising an intermediate high-frequency feature image and an intermediate low-frequency feature image; performing feature enhancement processing on the intermediate feature image and the primary decoupled image to obtain a target feature image, the target feature image comprising a target high-frequency feature image and a target low-frequency feature image.

4. The method of claim 3, wherein, The primary decoupled image comprises a primary high-frequency feature image and a primary low-frequency feature image; the method comprises the following steps: processing the primary high-frequency feature image, the primary low-frequency feature image, and the edge feature image based on a preset high-frequency coupling learning rule to obtain an intermediate high-frequency feature image; processing the primary high-frequency feature image, the primary low-frequency feature image, and the edge feature image based on a preset low-frequency coupling learning rule to obtain an intermediate low-frequency feature image; forming an intermediate feature image based on the intermediate high-frequency feature image and the intermediate low-frequency feature image.

5. The method of claim 3, wherein, The primary decoupled image comprises a primary high-frequency feature image and a primary low-frequency feature image; the method comprises the following steps: processing the intermediate high-frequency feature image and the primary high-frequency feature image based on a preset high-frequency feature enhancement rule to obtain a target high-frequency feature image; The primary low-frequency feature image and the intermediate low-frequency feature image are processed based on a preset low-frequency feature enhancement rule to obtain a target low-frequency feature image; A target feature image is formed based on the target high-frequency feature image and the target low-frequency feature image.

6. A system for deblurring an image, characterized in that The method comprises the following steps: An initial blur image of a target region is obtained; An edge feature extraction module is configured to down-sample the initial blur image to obtain a low-resolution blur image; A low-level feature image is obtained by performing feature extraction on the low-resolution blur image; A remolding image is obtained by performing bilateral network remolding on the low-level feature image, and a corresponding remolding formula is as follows: ； wherein, represents a remodeled image, which can also be referred to as an affine bilateral grid coefficient, , d = 6 represents the grid depth, and c = 6 represents that each grid cell in the bilateral network contains 6 coefficients, represents the grid depth; represents a horizontal spatial position; represents a vertical spatial position, represents the depth dimension of the grid (i.e., the th grid layer of the bilateral network), represents the th coefficient within each grid cell in the bilateral network, represents a low-level feature map; A guide image is obtained by performing convolution processing on the initial blur image; A three-linear interpolation processing is performed on the remolding image and the guide image to obtain an edge feature image; A blur decoupling module is configured to perform blur decoupling processing on the edge feature image and the initial blur image to obtain a primary decoupling image; A coupling enhancement module is configured to perform coupling enhancement processing on the primary decoupling image and the edge feature image to obtain a target feature image; A fusion module is configured to fuse the target feature image and the initial blur image to obtain a target clear image of the target region.

7. A computing device comprising a memory, a processor, and a program stored on the memory and running on the processor, wherein, The processor executes the program to implement the steps of the blur image sharpening method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores instructions, and when the instructions run on the terminal device, the terminal device executes the steps of the blur image sharpening method according to any one of claims 1 to 5.

Images