Image restoration methods, apparatus, computer equipment, and readable storage media

Abstract

This application relates to an image restoration method, apparatus, computer device, and readable storage medium. The method involves acquiring a second illumination artifact image and an initial artifact-removed image determined based on a first illumination artifact image. The resolution of the second illumination artifact image is lower than that of the first illumination artifact image. A target spatial weight map is generated by performing image structure difference analysis on the second illumination artifact image and the initial artifact-removed image. Based on the detail information of the first illumination artifact image and the target spatial weight map, image restoration is performed on the initial artifact-removed image to generate a target restored image. This effectively improves the clarity and detail fidelity of the artifact-removed image, ensures the effective restoration and preservation of detail information, and further guarantees the image quality and structural integrity of the artifact-removed image.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image restoration method, apparatus, computer device, and readable storage medium. Background Technology

[0002] Image artifacts (such as glare, shadows, and reflections) are common quality degradation phenomena during imaging. They are usually caused by abnormal reflections, scattering, or uneven lighting in the optical system. In images, they manifest as spots, halos, streaks, hazy areas, or brightness imbalances, severely impairing contrast, detail sharpness, and overall image quality. Therefore, image artifact removal has become an important task in computer vision and image processing.

[0003] In recent years, deep learning methods have made significant progress in artifact removal thanks to their end-to-end feature learning capabilities. A typical workflow involves inputting the original image with artifacts into a deep learning model such as a convolutional neural network, directly outputting the artifact-removed result. However, with the increasing number of high-resolution images, direct processing incurs enormous memory and computational overhead, making it difficult to meet the needs of practical applications. Therefore, the traditional approach usually involves first downscaling the high-resolution image before feeding it into the model for artifact removal, obtaining a low-resolution result, and then directly upsampling it back to its original size. However, this simple upscaling method easily leads to image blurring and loss of detail, especially at high magnification, making it difficult to guarantee the image quality and structural integrity of the artifact-removed image.

[0004] There is currently no effective solution to the problem that traditional techniques cannot guarantee the image quality and structural integrity of the image after artifact removal. Summary of the Invention

[0005] Therefore, it is necessary to provide an image restoration method, apparatus, computer device, and readable storage medium to address the aforementioned technical problems.

[0006] In a first aspect, this application provides an image restoration method, the method comprising:

[0007] Obtain a second illumination artifact image and an initial artifact-free image determined based on a first illumination artifact image; the resolution of the second illumination artifact image is lower than the resolution of the first illumination artifact image.

[0008] By performing image structure difference analysis on the second illumination artifact image and the initial artifact-removed image, a target space weight map is generated.

[0009] Based on the detailed information of the first illumination artifact image and the target spatial weight map, image restoration is performed on the initial artifact-removed image to generate the target restored image.

[0010] In one embodiment, the step of generating a target spatial weight map by performing image structure difference analysis on the second illumination artifact image and the initial artifact-removed image includes:

[0011] Obtain the first gradient feature map corresponding to the second illumination artifact image and the second gradient feature map corresponding to the initial artifact-removed image;

[0012] A target space weight map is generated by performing gradient difference space weight mapping on the first gradient feature map and the second gradient feature map.

[0013] In one embodiment, generating a target space weight map by performing gradient difference space weight mapping processing on the first gradient feature map and the second gradient feature map includes:

[0014] Calculate the gradient difference value between the first gradient feature map and the second gradient feature map to obtain the gradient difference map;

[0015] Based on a preset gradient-weight function, the gradient difference map is subjected to spatial weight mapping to generate the target spatial weight map.

[0016] In one embodiment, the step of performing spatial weight mapping processing on the gradient difference map according to a preset gradient-weight function to generate the target spatial weight map includes:

[0017] For each gradient difference value contained in the gradient difference map, the gradient difference value is weighted according to the preset gradient-weight function to obtain the target weight of the pixel position corresponding to the gradient difference value.

[0018] The target space weight map is obtained by constructing an image based on the target weights of the pixel positions corresponding to each gradient difference value.

[0019] In one embodiment, the preset gradient-weight function includes an upper threshold for gradient difference and a lower threshold for gradient difference; the step of performing weight mapping processing on the gradient difference value according to the preset gradient-weight function to obtain the target weight of the pixel position corresponding to the gradient difference value includes:

[0020] If the gradient difference value is less than the lower limit threshold of the gradient difference, then the target weight of the pixel position corresponding to the gradient difference value is set as the first weight;

[0021] If the gradient difference value is greater than the upper limit threshold of the gradient difference, then the target weight of the pixel position corresponding to the gradient difference value is set as the second weight;

[0022] If the gradient difference value is greater than or equal to the lower threshold of gradient difference and less than or equal to the upper threshold of gradient difference, then the gradient difference value is subjected to linear weight mapping processing according to the upper threshold of gradient difference and the lower threshold of gradient difference to obtain a third weight, and the third weight is determined as the target weight of the pixel position corresponding to the gradient difference value.

[0023] The third weight is less than the first weight and greater than the second weight.

[0024] In one embodiment, the step of performing image restoration on the initial artifact-removed image based on the detail information of the first illumination artifact image and the target spatial weight map to generate a target restored image includes:

[0025] Based on the target space weight map, the detailed information is weighted to obtain compensation information;

[0026] Based on the compensation information, the initial artifact-removed image is restored to obtain the target restored image.

[0027] In one embodiment, the method for obtaining detailed information of the first illumination artifact image includes:

[0028] The first illumination artifact image is downsampled to obtain the second illumination artifact image;

[0029] The second illumination artifact image is upsampled to obtain a third illumination artifact image; the resolution of the third illumination artifact image is the same as the resolution of the first illumination artifact image.

[0030] Based on the third illumination artifact image, the first illumination artifact image is subjected to detail separation processing to obtain the detail information of the first illumination artifact image.

[0031] In one embodiment, the method for obtaining the initial artifact-free image includes:

[0032] The second illumination artifact image is processed using an illumination artifact removal model to obtain the initial artifact-removed image.

[0033] Secondly, this application provides an image restoration apparatus, the apparatus comprising:

[0034] The acquisition module is used to acquire a second illumination artifact image and an initial artifact-free image determined based on a first illumination artifact image; the resolution of the second illumination artifact image is lower than the resolution of the first illumination artifact image;

[0035] The analysis module is used to generate a target space weight map by performing image structure difference analysis on the second illumination artifact image and the initial artifact-removed image;

[0036] The recovery module is used to perform image recovery on the initial artifact-removed image based on the detailed information of the first illumination artifact image and the target spatial weight map, and generate a target recovered image.

[0037] Thirdly, this application provides a computer device including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method described above.

[0038] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described above.

[0039] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described above.

[0040] The aforementioned image restoration method, apparatus, computer equipment, and readable storage medium acquire a second illumination artifact image determined based on a first illumination artifact image and an initial artifact-free image. By analyzing the image structure differences between the second illumination artifact image and the initial artifact-free image, regions altered by artifact removal processing can be accurately identified, thus clearly distinguishing between regions affected by illumination artifacts and normal regions unaffected by them. Furthermore, a target spatial weight map is generated based on the image structure difference analysis results, assigning corresponding weights to different pixel positions, thereby laying the foundation for subsequent differential detail compensation of the initial artifact-free image. Further, based on the detail information of the first illumination artifact image and the target spatial weight map, image restoration is performed on the initial artifact-free image to generate a target restored image. This effectively improves the clarity and detail fidelity of the artifact-free image, ensuring the effective restoration and preservation of detail information, and further guaranteeing the image quality and structural integrity of the artifact-free image. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is a diagram illustrating the application environment of an image restoration method in one embodiment;

[0043] Figure 2 This is a flowchart illustrating an image restoration method in one embodiment;

[0044] Figure 3 This is a flowchart illustrating the steps for generating a target space weighted map in one embodiment;

[0045] Figure 4 This is a flowchart illustrating the target image restoration generation step in one embodiment;

[0046] Figure 5 This is a flowchart illustrating the steps for obtaining detailed information in one embodiment;

[0047] Figure 6 This is a schematic diagram of the first, second, and third illumination artifact images in one embodiment;

[0048] Figure 7 This is a schematic diagram illustrating detailed information in one embodiment;

[0049] Figure 8 This is a schematic diagram of the processing flow of an image restoration method in a specific embodiment;

[0050] Figure 9 This is a structural block diagram of an image restoration device in one embodiment;

[0051] Figure 10 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0053] The typical process for artifact removal involves inputting the original image with artifacts into a deep learning model such as a convolutional neural network and directly outputting the artifact-removed result. However, with the increasing number of high-resolution images, direct processing incurs huge memory and computational overhead, making it difficult to meet the needs of practical applications. In traditional solutions, the high-resolution image containing lighting artifacts is first downscaled to a low-resolution image to obtain a low-resolution image with lighting artifacts. Next, a deep learning model is used to process this low-resolution image to generate a low-resolution image without lighting artifacts. Then, by subtracting the model's input (the low-resolution image with lighting artifacts) from its output (the low-resolution image without lighting artifacts), a low-resolution image with no lighting artifacts is obtained. Next, the low-resolution pure illumination artifact image is enlarged to its original resolution. Finally, the enlarged high-resolution pure illumination artifact image is subtracted from the original high-resolution image with illumination artifacts to obtain a high-resolution image without illumination artifacts. This method is based on the assumption that illumination artifacts are low-frequency information. However, in practical applications, the above assumption is not entirely valid. Real illumination artifacts also contain a certain amount of high-frequency information. If the low-resolution pure illumination artifact image is directly enlarged, the high-frequency components of the illumination artifacts will be missing in the high-resolution pure illumination artifact image. This results in residual illumination artifacts in the result of subtracting the original high-resolution image with illumination artifacts from the high-resolution pure illumination artifact image, affecting the image's clarity and quality.

[0054] Therefore, this application provides an image restoration method, which aims to improve the clarity and detail fidelity of the image after artifact removal, ensure the effective recovery and preservation of detail information, and guarantee the image quality and structural integrity of the image after artifact removal.

[0055] The image restoration method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104, or it can be located in the cloud or on other network servers. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, and tablets. Server 104 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0056] In one embodiment, such as Figure 2 As shown, Figure 2This is a flowchart illustrating an image restoration method in one embodiment. This embodiment uses the method applied to a terminal as an example for illustration. It is understood that this method can also be applied to a server, and further to a system including both a terminal and a server, and implemented through interaction between the terminal and the server. In this embodiment, the image restoration method includes the following steps:

[0057] Step S201: Obtain the second illumination artifact image and the initial artifact removal image determined based on the first illumination artifact image.

[0058] The first illumination artifact image refers to the original image containing illumination artifacts received or directly acquired through an image acquisition device.

[0059] The image acquisition device may include, but is not limited to, mobile terminals such as cameras, mobile phones, and tablets, smart wearable devices (such as smartwatches, smart glasses, or AR, XR, and VR devices), or other imaging devices. This imaging device can be installed on various types of vehicles, such as passenger cars, SUVs, commercial vehicles such as trucks, tractors, and large buses, as well as special vehicles such as fire trucks and ambulances. The imaging device can also be installed on robots. Specifically, robots include, but are not limited to, industrial robots equipped with robotic arms, AGVs, robot dogs, water robots, drones, service robots that provide convenience or assistance, humanoid robots, etc.

[0060] Lighting artifacts are caused by poor lighting conditions or limitations of the equipment itself. Specific manifestations can include, but are not limited to, flare, shadows, and reflections. For example, flare is easily produced when strong light sources such as sunlight or artificial light enter the lens directly, or when there is high contrast in the scene (such as a bright sky and a dark ground). At night, on a stage, or in a laboratory, when facing multiple, high-intensity lights (such as streetlights, spotlights, or lasers), multiple reflections can create complex flare patterns. When shooting targets such as water surfaces, ice surfaces, glass curtain walls, or metal surfaces, the reflected strong light enters the lens at a specific angle, equivalent to a direct light source, thus producing flare. An unclean lens (e.g., contaminated with fingerprints, oil, dust, etc.) or a poor-quality filter / lens hood will exacerbate light scattering and reflection, potentially inducing localized flare even in non-backlit conditions.

[0061] The second illumination artifact image is obtained by downsampling the first illumination artifact image; the resolution of the second illumination artifact image is lower than that of the first illumination artifact image. Downsampling is used to merge multiple adjacent pixels into one pixel, thereby reducing the image resolution. This embodiment does not specifically limit the downsampling implementation method, as long as it can reduce the image resolution.

[0062] The initial artifact-removed image is obtained by processing the second illumination artifact image; the resolution of the initial artifact-removed image is the same as that of the second illumination artifact image. In an exemplary embodiment, the initial artifact-removed image can be obtained by processing the second illumination artifact image using an illumination artifact removal model. The illumination artifact removal model refers to a trained image processing model used to identify and suppress illumination artifacts (such as glare, shadows, reflections, etc.) in an image; the illumination artifact removal model can be, but is not limited to, built based on a deep learning architecture; it should be noted that the specific structure and training method of the illumination artifact removal model can be set with reference to existing related technologies, and are not specifically limited here.

[0063] It should be noted that because high-resolution images contain a large amount of pixel data, directly processing this data requires significant computing resources and storage space. Therefore, this embodiment downsamples the high-resolution first illumination artifact image into a low-resolution second illumination artifact image, and then performs artifact removal operations on this second image to generate an initial artifact-removed image. This approach preserves the core structural features of the image while reducing the amount of data processed, effectively lowering memory usage and computational complexity, making it suitable for resource-constrained applications. Specifically, the resolution of the downsampled second illumination artifact image can be set according to the requirements of the illumination artifact removal model, or the first illumination artifact image can be downsampled according to a preset downsampling ratio to obtain the second illumination artifact image. Typically, the second illumination artifact image and the initial artifact-removed image have the same resolution.

[0064] However, downsampling the high-resolution first illumination artifact image to a low-resolution second illumination artifact image results in the loss of detailed information (such as edges and textures) in the first illumination artifact image. Consequently, the initial artifact-removed image obtained after processing the second illumination artifact image also lacks relevant detailed information. To compensate for this deficiency, this embodiment proposes to perform image analysis on the initial artifact-removed image and the second illumination artifact image to ensure the image quality and structural integrity of the final artifact-removed image.

[0065] Step S202: By performing image structure difference analysis on the second illumination artifact image and the initial artifact-removed image, a target space weight map is generated.

[0066] It should be noted that by analyzing the image structure differences between the second illumination artifact image and the initial artifact-removed image, the regions that have changed due to the artifact removal process can be accurately identified. This allows for a clear distinction between regions affected by illumination artifacts and normal regions unaffected by them. Furthermore, based on the results of the image structure difference analysis, a target spatial weight map is generated, which assigns differentiated weights to different pixel locations. This enables subsequent image restoration processes to perform differentiated detail compensation on the initial artifact-removed image according to the weights of each pixel location, ensuring the accuracy and effectiveness of image restoration.

[0067] The image structure difference analysis can be performed on the differences in pixel value size and / or pixel value variation between the second illumination artifact image and the initial artifact-removed image. The target space weight map includes the weights corresponding to different pixel positions; these weights quantify the importance of the corresponding pixel position. It can be understood that a high weight indicates that the corresponding pixel position is more important to the overall image detail, while a low weight indicates that the corresponding pixel position is less important to the overall image detail.

[0068] Understandably, the target spatial weight map is used to regulate the intensity of detail compensation at different pixel locations during image restoration. In an exemplary embodiment, higher weight values ​​can be assigned to pixel locations in important regions (such as regions unaffected by lighting artifacts) to ensure the detail and sharpness of these important regions; while lower weights can be assigned to pixel locations in unimportant regions (such as regions affected by lighting artifacts) to suppress artifact edge residue and avoid adverse effects on the overall image quality.

[0069] In an exemplary embodiment, the method for generating a target spatial weight map by performing image structure difference analysis on the second illumination artifact image and the initial artifact-removed image can be as follows: performing image structure difference analysis on the second illumination artifact image and the initial artifact-removed image to obtain image structure difference results; and performing spatial weight mapping processing on the image structure difference results to generate a target spatial weight map. The image structure difference results are used to characterize the changes in image structure caused by the artifact removal process. By mapping the image structure difference results to the target spatial weight map, the importance of each pixel location to the image restoration process can be accurately quantified. This guides the image restoration process to focus on important regions with high weights and ensures that these important regions can achieve sufficient detail restoration, thereby improving the quality of the final image.

[0070] Step S203: Based on the detailed information of the first illumination artifact image and the target spatial weight map, perform image restoration on the initial artifact-removed image to generate the target restored image.

[0071] The detailed information of the first illumination artifact image may include, but is not limited to, high-frequency components such as edges and textures. In an exemplary embodiment, the method for obtaining the detailed information of the first illumination artifact image may include, but is not limited to, extracting detailed features from the first illumination artifact image based on a detail feature extraction model to obtain the detailed information of the first illumination artifact image. The detail feature extraction model may include, but is not limited to, feature extraction models such as neural networks, and is not specifically limited here.

[0072] The resolution of the target restored image is the same as that of the first illumination artifact image. This means that the target restored image, while removing illumination artifacts, fully preserves the detailed information of the first illumination artifact image, thus ensuring image quality, sharpness, and structural integrity, achieving high-quality image restoration.

[0073] In an exemplary embodiment, the method for generating the target restored image by performing image restoration on the initial artifact-removed image based on the detail information of the first illumination artifact image and the target spatial weight map can be as follows: upsampling the initial artifact-removed image to obtain an upsampled artifact-removed image; and performing detail compensation on the upsampled artifact-removed image based on the detail information of the first illumination artifact image and the target spatial weight map to generate the target restored image. The resolution of the upsampled artifact-removed image is the same as the resolution of the first illumination artifact image. Upsampling can be implemented, but is not limited to, interpolation algorithms (such as bilinear interpolation), and is not specifically limited here.

[0074] It is understandable that upsampling is an interpolation estimation process based on neighboring pixels. Although it can improve image resolution, it results in blurred edges, smoothed textures, and decreased image clarity in the upsampled artifact-free image due to the loss of detail information during the downsampling of the first illumination artifact image. Therefore, this embodiment introduces detail information from the first illumination artifact image and a target spatial weight map to compensate for the details in the upsampled artifact-free image. This effectively compensates for the lost detail information, improves image clarity, and thus ensures the image quality and structural integrity of the recovered target image.

[0075] For example, a first illumination artifact image is acquired through an image acquisition device. The first illumination artifact image is downsampled to obtain a second illumination artifact image with a lower resolution than the first illumination artifact image. The second illumination artifact image is then processed to remove artifacts, resulting in an initial artifact-removed image. Image structure difference analysis is performed between the second illumination artifact image and the initial artifact-removed image to obtain image structure difference results. Spatial weight mapping is applied to the image structure difference results to generate a target spatial weight map. The initial artifact-removed image is then upsampled to obtain an upsampled artifact-removed image. Based on the detail information of the first illumination artifact image and the target spatial weight map, detail compensation is performed on the upsampled artifact-removed image to generate a target restored image. The resulting target restored image can effectively remove illumination artifacts while fully preserving the detail information (such as edges, textures, and other high-frequency components) in the first illumination artifact image. Furthermore, image restoration based on the target spatial weight map can effectively suppress artifact edge residue problems, further ensuring the structural integrity and image quality of the target restored image.

[0076] In this embodiment, by analyzing the image structure differences between the second illumination artifact image and the initial artifact-free image, regions altered by the artifact removal process can be accurately identified. This clearly distinguishes between regions affected by illumination artifacts and normal regions unaffected by them. Based on the image structure difference analysis, a target spatial weight map is generated, assigning corresponding weights to different pixel locations. This lays the foundation for subsequent differential detail compensation of the initial artifact-free image. Furthermore, based on the detail information of the first illumination artifact image and the target spatial weight map, image restoration is performed on the initial artifact-free image to generate the target restored image. This effectively improves the clarity and detail fidelity of the artifact-free image, ensuring the effective restoration and preservation of detail information, and further guaranteeing the image quality and structural integrity of the artifact-free image.

[0077] In one embodiment, such as Figure 3 As shown, Figure 3 This is a flowchart illustrating the target space weight map generation steps in one embodiment; the target space weight map is generated by performing image structure difference analysis on the second illumination artifact image and the initial artifact-removed image, including the following steps:

[0078] Step S301: Obtain the first gradient feature map corresponding to the second illumination artifact image and the second gradient feature map corresponding to the initial artifact removal image.

[0079] The first gradient feature map includes the first gradient values ​​corresponding to different pixel positions in the second illumination artifact image; the first gradient feature map is used to characterize the image structure information of the second illumination artifact image. The second gradient feature map includes the second gradient values ​​corresponding to different pixel positions in the initial artifact-removed image; the second gradient feature map is used to characterize the image structure information of the initial artifact-removed image. The image structure information may include, but is not limited to, edge, texture, and contour information.

[0080] It is understandable that regions with larger gradient values ​​in the gradient feature map correspond to edge or contour regions (high-frequency components) in the image, while regions with smaller gradient values ​​correspond to flat regions (low-frequency components) in the image.

[0081] In an exemplary embodiment, the method for obtaining the first gradient feature map corresponding to the second illumination artifact image and the second gradient feature map corresponding to the initial artifact removal image may be: performing gradient feature extraction on the second illumination artifact image and the initial artifact removal image respectively to obtain the first gradient feature map corresponding to the second illumination artifact image and the second gradient feature map corresponding to the initial artifact removal image.

[0082] Gradient feature extraction can be implemented using, but is not limited to, the Sobel operator or the Laplacian operator; no specific limitation is made here. For example, the Sobel operator can be used to convolve the second illumination artifact image and the initial artifact-removed image to obtain the corresponding first gradient feature map and second gradient feature map.

[0083] Step S302: Generate a target space weight map by performing gradient difference space weight mapping on the first gradient feature map and the second gradient feature map.

[0084] It should be noted that by analyzing the gradient difference between the first and second gradient feature maps before and after artifact removal, the regions and degrees of image structural changes can be determined. For example, if the gradient value of a certain region in the second gradient feature map is significantly smaller than the gradient value of the corresponding region in the first gradient feature map, it indicates that this region is likely affected by lighting artifacts. After artifact removal, this region has become a relatively flat region. Understandably, the gradient difference at the same pixel location before and after artifact removal can be used to characterize the degree of image structural change. Furthermore, based on the gradient difference between the first and second gradient feature maps, a target space weight map is generated, which can adjust the detail compensation intensity at different pixel locations.

[0085] In an exemplary embodiment, the corresponding weight can be determined based on the magnitude of the gradient difference value. Specifically, for pixel locations with small gradient difference values, it can be determined that the image structure at that pixel location has not changed significantly and may belong to a region unaffected by lighting artifacts. Therefore, a higher weight can be assigned to it to preserve and enhance the detail and sharpness of that pixel location. Conversely, for pixel locations with large gradient difference values, it can be determined that the image structure at that pixel location has changed and may belong to a region affected by lighting artifacts. Therefore, a lower weight can be assigned to it to suppress artifact edge residue and avoid adversely affecting the overall image quality.

[0086] For example, gradient features are extracted from the second illumination artifact image and the initial artifact-removed image respectively to obtain a first gradient feature map corresponding to the second illumination artifact image and a second gradient feature map corresponding to the initial artifact-removed image; gradient difference analysis is performed on the first gradient feature map and the second gradient feature map to obtain the corresponding gradient difference; spatial weight mapping is performed on the gradient difference between the first gradient feature map and the second gradient feature map to generate a target spatial weight map.

[0087] In this embodiment, by performing gradient difference analysis on the first gradient feature map corresponding to the second illumination artifact image and the second gradient feature map corresponding to the initial artifact-removed image, the changes in image structure before and after artifact removal can be accurately determined. Based on this, spatial weight mapping is performed on the gradient difference values ​​corresponding to each pixel position to generate a target spatial weight map, so that different pixel positions are assigned differentiated weights according to their degree of image structure change: specifically, regions with stable structures (small gradient difference values) are given higher weights to preserve original details and clarity; regions with significantly changed structures (large gradient difference values) are given lower weights to suppress illumination artifact residues. Based on the target spatial weight map, differentiated detail compensation can be performed on different pixel positions in the subsequent image restoration stage to avoid image quality degradation caused by introducing unnecessary compensation, ensuring that high-quality image output can be achieved in the end.

[0088] In one embodiment, a target space weight map is generated by performing gradient difference space weight mapping on the first gradient feature map and the second gradient feature map, including the following steps:

[0089] Step 1: Calculate the gradient difference between the first gradient feature map and the second gradient feature map to obtain the gradient difference map.

[0090] The first gradient feature map includes the first gradient values ​​corresponding to different pixel positions in the second illumination artifact image. The second gradient feature map includes the second gradient values ​​corresponding to different pixel positions in the initial artifact-removed image.

[0091] The gradient difference map includes the gradient difference values ​​at different pixel locations before and after artifact removal. It can be understood that the gradient difference value at each pixel location is equal to the difference between the first gradient value and the second gradient value at that pixel location.

[0092] Gradient difference maps are used to characterize changes in image structure before and after artifact removal. These changes include both the regions of structural change and the degree of change. The degree of structural change can be quantified by the gradient difference value; a larger gradient difference value indicates a greater degree of structural change, and vice versa.

[0093] In an exemplary embodiment, the method for calculating the gradient difference value between the first gradient feature map and the second gradient feature map to obtain the gradient difference map can be as follows: subtract the first gradient value and the second gradient value of the first gradient feature map and the second gradient feature map at the same pixel position to obtain the gradient difference value corresponding to each pixel position; construct the image based on the gradient difference value corresponding to each pixel position to obtain the gradient difference map. For example, the first gradient feature map is denoted as... The second gradient feature map is denoted as The first gradient feature map Second gradient feature map Subtracting them yields the gradient difference map. .

[0094] Step 2: Based on the preset gradient-weight function, perform spatial weight mapping on the gradient difference map to generate the target spatial weight map.

[0095] The preset gradient-weight function is used to perform differential weight mapping on the gradient difference values ​​in the gradient difference map. The preset gradient-weight function can be, but is not limited to, a piecewise linear function; no specific limitation is made here.

[0096] Understandably, pixels with smaller gradient differences have not undergone significant changes in image structure and may belong to regions unaffected by lighting artifacts, while pixels with larger gradient differences have experienced changes in image structure and may belong to regions affected by lighting artifacts. By using a predefined gradient-weight function, the gradient difference values ​​of pixels with smaller gradient differences in the gradient difference map can be mapped to higher weights, while the gradient difference values ​​of pixels with larger gradient differences can be mapped to lower weights. This allows different pixel locations to be assigned differentiated weights based on the degree of change in their image structure.

[0097] In an exemplary embodiment, step 2 involves performing spatial weight mapping on the gradient difference map according to a preset gradient-weight function to generate a target spatial weight map, including the following steps:

[0098] Step 2.1: For each gradient difference value contained in the gradient difference map, perform weight mapping processing on the gradient difference value according to the preset gradient-weight function to obtain the target weight of the pixel position corresponding to the gradient difference value.

[0099] The target weight is used to characterize the importance of the corresponding pixel position in the image restoration stage. The higher the target weight, the more attention the pixel position needs to be paid and the more detail compensation is required during the restoration process. Conversely, the lower the target weight, the more limited the contribution of the pixel position to the overall image detail. The restoration intensity can be appropriately reduced to avoid introducing unnecessary compensation and causing a decline in image quality.

[0100] Step 2.2: Construct the image based on the target weights of the pixel positions corresponding to each gradient difference value to obtain the target space weight map.

[0101] In an exemplary embodiment, the method for constructing a target spatial weight map based on the target weights of the pixel positions corresponding to each gradient difference value can be as follows: constructing an initial spatial weight map based on the target weights of the pixel positions corresponding to each gradient difference value; and upsampling the initial spatial weight map to obtain the target spatial weight map. The resolution of the target spatial weight map is the same as the resolution of the first illumination artifact image, ensuring that subsequent pixel-level image restoration operations are performed at the same resolution.

[0102] For example, the first gradient value and the second gradient value at the same pixel position in the first gradient feature map and the second gradient feature map are subtracted to obtain the gradient difference value corresponding to each pixel position; an image is constructed based on the gradient difference value corresponding to each pixel position to obtain a gradient difference map; according to a preset gradient-weight function, the gradient difference values ​​at different pixel positions in the gradient difference map are subjected to differential weight mapping processing to obtain the target weight of each pixel position; an image is constructed based on the target weight of each pixel position to obtain an initial spatial weight map; the initial spatial weight map is upsampled to obtain a target spatial weight map.

[0103] In this embodiment, a gradient difference map is obtained by calculating the gradient difference value between the first gradient feature map and the second gradient feature map, which can determine the changes in image structure before and after artifact removal. By using a preset gradient-weight function, differential spatial weight mapping is performed on each gradient difference value in the gradient difference map, so that different pixel positions are assigned differential weights according to their degree of image structure change. This ensures that differential detail compensation can be performed on different pixel positions in the subsequent image restoration stage, so as to avoid the image quality degradation caused by introducing unnecessary compensation and ensure that high-quality image output can be achieved in the end.

[0104] In one embodiment, the preset gradient-weight function includes an upper gradient difference threshold and a lower gradient difference threshold. It should be noted that the upper and lower gradient difference thresholds need to be set according to the actual image restoration requirements, and are not specifically limited here.

[0105] In one embodiment, the gradient difference values ​​are weighted according to a preset gradient-weight function to obtain the target weights of the pixel positions corresponding to the gradient difference values, including the following steps:

[0106] Step 1: If the gradient difference value is less than the lower limit threshold of the gradient difference, then set the target weight of the pixel position corresponding to the gradient difference value as the first weight.

[0107] Step 2: If the gradient difference value is greater than the upper limit threshold of the gradient difference, then set the target weight of the pixel position corresponding to the gradient difference value as the second weight.

[0108] Step 3: If the gradient difference value is greater than or equal to the lower threshold of gradient difference and less than or equal to the upper threshold of gradient difference, then perform linear weight mapping on the gradient difference value according to the upper threshold of gradient difference and the lower threshold of gradient difference to obtain the third weight, and determine the third weight as the target weight of the pixel position corresponding to the gradient difference value.

[0109] The third weight is less than the first weight but greater than the second weight.

[0110] The first weight can be 1, the second weight can be 0, and the third weight is between 0 and 1.

[0111] The preset gradient-weight function also includes a linear mapping function; the expression for the linear mapping function can be:

[0112] ;

[0113] in, For the target weight, This is the upper limit threshold for gradient difference. This is the lower limit threshold for gradient difference. This represents the gradient difference value. .

[0114] Understandably, regarding satisfying The gradient difference value x, and the target weight at the pixel position corresponding to the gradient difference value x. , and gradient difference value They are negatively correlated. That is, the larger the gradient difference value x, the greater the target weight at the pixel position corresponding to the gradient difference value x. The smaller the value, the less likely it is to introduce unnecessary compensation, effectively suppressing illumination artifacts and ensuring the overall image quality.

[0115] In this embodiment, by performing differential weight mapping on the gradient difference values, the target weight of the corresponding pixel position is obtained; then, based on the target weight of each pixel position, the subsequent image restoration process can be accurately guided, and differential detail compensation matching the degree of structural change is implemented for different regions: high weight regions are enhanced with stronger details, while low weight regions are moderately suppressed, thereby effectively removing lighting artifacts while taking into account the detail fidelity, structural integrity and visual naturalness of the image.

[0116] In one embodiment, such as Figure 4 As shown, Figure 4 This is a flowchart illustrating the target restoration image generation steps in one embodiment; based on the detailed information of the first illumination artifact image and the target spatial weight map, image restoration is performed on the initial artifact-removed image to generate the target restoration image, including the following steps:

[0117] Step S401: Based on the target space weight map, the detailed information is weighted to obtain the compensation information.

[0118] The target space weight map includes the weights corresponding to different pixel positions. It can be understood that a high weight indicates that the corresponding pixel position is more important to the overall image detail, while a low weight indicates that the corresponding pixel position is less important to the overall image detail.

[0119] The target spatial weight map is used to adjust the detail compensation intensity at different pixel locations during image restoration. It should be noted that the target spatial weight map can be generated using the method described in any of the above embodiments, and will not be repeated here.

[0120] The compensation information includes the detail compensation amount corresponding to each pixel position. In an exemplary embodiment, the product of the detail information corresponding to each pixel position and the target weight is determined as the detail compensation amount corresponding to each pixel position.

[0121] Understandably, because the target weights at different pixel locations differ in two-dimensional space, the amount of detail compensation obtained at different pixel locations also varies: important regions (such as those unaffected by lighting artifacts) receive stronger detail enhancement due to their higher weights, while unimportant regions (such as those affected by lighting artifacts) are moderately suppressed due to their lower weights. This differentiated detail compensation can effectively remove lighting artifacts while accurately restoring and enhancing key details of the image, while avoiding the introduction of noise into unimportant regions, thus ensuring the image quality, structural integrity, and naturalness of the final output image.

[0122] Step S402: Perform image restoration on the initial artifact-removed image based on the compensation information to obtain the target restored image.

[0123] In an exemplary embodiment, the method for obtaining the target restored image by performing image restoration on the initial artifact-removed image based on the compensation information may be: upsampling the initial artifact-removed image to obtain the upsampled artifact-removed image; and fusing the compensation information with the upsampled artifact-removed image to obtain the target restored image.

[0124] For example, let the initial artifact-removed image be... The initial artifact-removed image is upsampled to obtain the upsampled artifact-removed image. Let the target space weight map be... Details are as follows According to the target space weight map For details After weighted processing, the compensation information is obtained as follows: Regarding compensation information Compared with the upsampled artifact-free image The target image is obtained by performing fusion processing. .in," "" indicates that the values ​​at corresponding positions in the image are multiplied pixel by pixel.

[0125] Understandably, since the resolution of the initial artifact-restored image is lower than that of the first illumination artifact image, to ensure that the final output target restoration image has the same resolution as the first illumination artifact image, the initial artifact-restored image needs to be upsampled to obtain an upsampled artifact-restored image with the same resolution as the first illumination artifact image. However, since the upsampling process only fills pixels through interpolation and other methods, it does not restore the high-frequency details lost due to downsampling, resulting in a lack of image detail information. To address this, this embodiment utilizes the detail information extracted from the first illumination artifact image and combines it with the target spatial weight map for weighted processing to generate compensation information. Subsequently, the compensation information is fused into the upsampled artifact-restored image to achieve differentiated detail compensation; based on the effective removal of illumination artifacts, the details of important areas (such as edges and textures) are accurately restored and enhanced, ultimately generating a high-resolution, high-quality target restoration image that balances clarity, realism, and structural integrity.

[0126] In one embodiment, such as Figure 5 As shown, Figure 5 This is a flowchart illustrating the detailed information acquisition steps in one embodiment; the method for acquiring detailed information of the first illumination artifact image includes the following steps:

[0127] Step S501: Downsample the first illumination artifact image to obtain the second illumination artifact image.

[0128] The resolution of the second lighting artifact image is lower than that of the first lighting artifact image. Understandably, the second lighting artifact image only retains the low-frequency components (such as large outlines and overall tone) of the first lighting artifact image, while losing high-frequency components (such as edge and texture details).

[0129] Step S502: Upsample the second illumination artifact image to obtain the third illumination artifact image.

[0130] The resolution of the third illumination artifact image is the same as that of the first illumination artifact image. The third illumination artifact image refers to the image obtained by downsampling and upsampling the first illumination artifact image. Although the third illumination artifact image has the same resolution as the first illumination artifact image, it does not contain effective detailed information due to information loss during the sampling process.

[0131] Step S503: Based on the third illumination artifact image, perform detail separation processing on the first illumination artifact image to obtain the detail information of the first illumination artifact image.

[0132] In an exemplary embodiment, the method for obtaining the detail information of the first lighting artifact image by performing detail separation processing on the first lighting artifact image based on the third lighting artifact image may be: subtracting the first lighting artifact image from the third lighting artifact image to obtain the detail information of the first lighting artifact image.

[0133] For example, see Figure 6 and Figure 7 ,right Figure 6 The first lighting artifact image shown is downsampled to obtain a second lighting artifact image with a lower resolution than the first lighting artifact image; the second lighting artifact image is upsampled to obtain a third lighting artifact image with a resolution equal to the first lighting artifact image; then, the first lighting artifact image and the third lighting artifact image are subtracted to obtain the image shown below. Figure 7 The detailed information of the first lighting artifact image shown.

[0134] It should be noted that during downsampling, reducing image resolution by merging multiple pixels inevitably leads to the loss of some details, such as high-frequency components like edges and textures. When upsampling (i.e., image magnification), since the original detail information has been lost, the newly added pixel values ​​are usually estimated based on the relationships between existing pixels (such as the average of neighboring pixels). Although interpolation algorithms can mitigate image distortion to some extent, they cannot completely recover the detail information lost during downsampling; this can result in problems such as blurred edges and unclear textures in the upsampled image (i.e., the third illumination artifact image). This embodiment, by subtracting the first illumination artifact image from the third illumination artifact image, can cancel out low-frequency components in the image, thereby effectively separating the high-frequency components lost during downsampling (contrast in edge contours, differences in fine textures), laying the foundation for subsequent image restoration.

[0135] In one specific embodiment, taking glare in artifacts as an example, see... Figure 8 The first illumination artifact image with glare was acquired using an image acquisition device. (Example, resolution 4096×3072), for the first illumination artifact image Downsampling is performed to obtain the second illumination artifact image. (Resolution: 512×512); for the second illumination artifact image Upsampling is performed to obtain the third illumination artifact image. At this time compared to It will become blurry, the difference between the two This can represent the first lighting artifact image. The detailed information and specific calculation formula are as follows: .

[0136] The second illumination artifact image Input an illumination artifact removal model (such as U-Net) for artifact removal processing to obtain an initial artifact-removed image. Initial artifact-removed image With the second lighting artifact image They have the same resolution.

[0137] Then the second illumination artifact image is calculated. The gradient feature map is used to obtain the first gradient feature map. Calculate the initial artifact-free image The gradient feature map is used to obtain the second gradient feature map. The first gradient feature map Second gradient feature map The second illumination artifact image can be processed using, but is not limited to, the Sobel operator or the Laplacian operator. and the initial artifact-removed image It is obtained by convolution. It can be understood that the image gradient can reflect the structural information of the image. The area with a large gradient value corresponds to the edge or contour of the object in the image (high frequency component), and the area with a small gradient value corresponds to the flat area in the image (low frequency component).

[0138] By comparing the difference in image gradients before and after artifact removal, we can understand whether the image structure has changed. For example, if the gradient value of a certain region after artifact removal is significantly smaller than the gradient value of the corresponding region before artifact removal, then this region is likely a strong edge region of glare in the original image, which has been transformed into a flat region through artifact removal. In the subsequent process of restoring the target image from the low-resolution initial artifact-removed image to a high-resolution image, it is necessary to reduce the compensation for detail information in this region; otherwise, glare edge residue will occur. Therefore, this embodiment introduces a target spatial weight map to control the intensity of detail compensation in different regions during the image restoration process. Specifically:

[0139] By calculating the first gradient feature map Second gradient feature map The difference between them is used to obtain a gradient difference map. :

[0140] ;

[0141] Based on the preset gradient-weight function, the gradient difference map is processed. Perform spatial weight mapping to generate an initial spatial weight graph. Upsample the initial spatial weight map to obtain the target spatial weight map. Target space weight map Image with first lighting artifact They have the same resolution.

[0142] in, This represents the preset gradient-weight function, which can be, but is not limited to, a piecewise linear function. For example, a lower bound threshold for gradient difference can be set empirically. and gradient difference upper limit threshold When gradient difference map The gradient difference value x is less than When, map it to 1, when the gradient difference map The gradient difference value x is greater than When, map it to 0, when the gradient difference map The gradient difference value x in the middle is between and In the middle, it is mapped to a weight value w according to the following formula:

[0143] ;

[0144] It should be noted that the initial spatial weight graph The value in is between 0 and 1.

[0145] Finally, the initial artifact-removed image Upsampling is performed to obtain the upsampled artifact-free image. Upsampled artifact-free image Image with first lighting artifact With the same resolution. The artifact-free image after upsampling at this time. The original image lacks detailed information, which needs to be obtained through detailed information. Compensation is performed, while also incorporating the target space weight map. By suppressing high-frequency details in the glare areas, the final image of the target reconstruction is obtained. :

[0146] ;

[0147] in," "" indicates that the values ​​at corresponding positions in the image are multiplied pixel by pixel.

[0148] By introducing a target space weight map during the image restoration stage It can control the degree of detail compensation in each region: in regions where the original image details need to be preserved, the corresponding weight value is 1 or close to 1, and vice versa, the weight value is 0 or close to 0. Thus, while effectively removing lighting artifacts, it can fully preserve the details (such as edges, textures and other high-frequency components) in the first lighting artifact image, ensuring the structural integrity and image quality of the target restored image.

[0149] The aforementioned image restoration method, on the one hand, significantly reduces memory usage and computational overhead by downsampling the original high-resolution first illumination artifact image and performing artifact removal processing in the low-resolution space. On the other hand, it utilizes detailed information extracted from the original high-resolution image and combines it with a target space weight map generated based on the gradient difference between the images before and after artifact removal to restore the initial artifact-removed image. This achieves accurate detail restoration of important regions (such as edges and textures), effectively avoiding detail loss caused by downsampling. Simultaneously, it effectively suppresses the residual artifacts such as glare and reflections in edge regions, preventing excessive enhancement from introducing noise or distortion. Overall, this image restoration method ensures efficient computation while maintaining image detail fidelity and structural integrity, making it suitable for resource-constrained applications with high image quality requirements.

[0150] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0151] Based on the same inventive concept, this application also provides an image restoration apparatus for implementing the image restoration method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more image restoration apparatus embodiments provided below can be found in the limitations of the image restoration method described above, and will not be repeated here.

[0152] In one exemplary embodiment, such as Figure 9 As shown, an image restoration device is provided, including: an acquisition module 901, an analysis module 902, and a restoration module 903, wherein:

[0153] The acquisition module 901 is used to acquire a second illumination artifact image and an initial artifact-free image determined based on the first illumination artifact image; the resolution of the second illumination artifact image is lower than the resolution of the first illumination artifact image;

[0154] Analysis module 902 is used to generate a target space weight map by performing image structure difference analysis on the second illumination artifact image and the initial artifact removal image;

[0155] The recovery module 903 is used to perform image recovery on the initial artifact-removed image based on the detailed information of the first illumination artifact image and the target spatial weight map, and generate a target recovered image.

[0156] The aforementioned image restoration device acquires a second illumination artifact image determined based on a first illumination artifact image and an initial artifact-removed image. By analyzing the image structure differences between the second illumination artifact image and the initial artifact-removed image, it can accurately identify regions changed due to artifact removal processing, thus clearly distinguishing between regions affected by illumination artifacts and normal regions unaffected by them. Based on the results of the image structure difference analysis, a target spatial weight map is generated, which assigns corresponding weights to different pixel positions, laying the foundation for subsequent differential detail compensation of the initial artifact-removed image. Furthermore, based on the detail information of the first illumination artifact image and the target spatial weight map, image restoration is performed on the initial artifact-removed image to generate a target restored image. This effectively improves the clarity and detail fidelity of the artifact-removed image, ensuring the effective restoration and preservation of detail information, and further guaranteeing the image quality and structural integrity of the artifact-removed image.

[0157] In one embodiment, the analysis module 902 is further configured to:

[0158] Obtain the first gradient feature map corresponding to the second illumination artifact image and the second gradient feature map corresponding to the initial artifact-removed image;

[0159] A target space weight map is generated by performing gradient difference space weight mapping on the first gradient feature map and the second gradient feature map.

[0160] In one embodiment, the analysis module 902 is further configured to:

[0161] Calculate the gradient difference between the first gradient feature map and the second gradient feature map to obtain the gradient difference map;

[0162] Based on the preset gradient-weight function, the gradient difference map is processed by spatial weight mapping to generate the target spatial weight map.

[0163] In one embodiment, the analysis module 902 is further configured to:

[0164] For each gradient difference value contained in the gradient difference map, the gradient difference value is weighted according to the preset gradient-weight function to obtain the target weight of the pixel position corresponding to the gradient difference value.

[0165] The target space weight map is obtained by constructing an image based on the target weights of the pixel positions corresponding to the gradient difference values.

[0166] In one embodiment, the preset gradient-weight function includes an upper threshold for gradient difference and a lower threshold for gradient difference; the analysis module 902 is further configured to:

[0167] If the gradient difference value is less than the lower limit threshold of the gradient difference, then the target weight of the pixel position corresponding to the gradient difference value is set as the first weight.

[0168] If the gradient difference value is greater than the upper limit threshold of the gradient difference, then the target weight of the pixel position corresponding to the gradient difference value is set as the second weight.

[0169] If the gradient difference value is greater than or equal to the lower threshold of gradient difference and less than or equal to the upper threshold of gradient difference, then the gradient difference value is subjected to linear weight mapping based on the upper threshold of gradient difference and the lower threshold of gradient difference to obtain the third weight, and the third weight is determined as the target weight of the pixel position corresponding to the gradient difference value.

[0170] The third weight is less than the first weight but greater than the second weight.

[0171] In one embodiment, the recovery module 903 is further configured to:

[0172] Based on the target space weight map, the detailed information is weighted to obtain the compensation information;

[0173] Based on the compensation information, the initial artifact-removed image is restored to obtain the target restored image.

[0174] In one embodiment, the acquisition module 901 is further configured to:

[0175] The first illumination artifact image is downsampled to obtain the second illumination artifact image;

[0176] The second illumination artifact image is upsampled to obtain the third illumination artifact image; the resolution of the third illumination artifact image is the same as that of the first illumination artifact image.

[0177] Based on the third illumination artifact image, detail separation processing is performed on the first illumination artifact image to obtain the detail information of the first illumination artifact image.

[0178] In one embodiment, the acquisition module 901 is further configured to:

[0179] The second illumination artifact image is processed using an illumination artifact removal model to obtain the initial artifact-free image.

[0180] Each module in the aforementioned image restoration device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0181] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 10 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores image restoration-related data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements an image restoration method.

[0182] Those skilled in the art will understand that Figure 10 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0183] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0184] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0185] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0186] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0187] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchain. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, data processing logic devices based on quantum computing, artificial intelligence (AI) processors, etc., and are not limited to these.

[0188] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0189] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. An image restoration method, characterized in that, The method includes: Obtain a second illumination artifact image and an initial artifact-free image determined based on a first illumination artifact image; the resolution of the second illumination artifact image is lower than the resolution of the first illumination artifact image. By performing image structure difference analysis on the second illumination artifact image and the initial artifact-removed image, a target space weight map is generated. Based on the detailed information of the first illumination artifact image and the target spatial weight map, image restoration is performed on the initial artifact-removed image to generate the target restored image.

2. The method according to claim 1, characterized in that, The step of generating a target spatial weight map by performing image structure difference analysis on the second illumination artifact image and the initial artifact-removed image includes: Obtain the first gradient feature map corresponding to the second illumination artifact image and the second gradient feature map corresponding to the initial artifact-removed image; A target space weight map is generated by performing gradient difference space weight mapping on the first gradient feature map and the second gradient feature map.

3. The method according to claim 2, characterized in that, The step of generating a target space weight map by performing gradient difference space weight mapping on the first gradient feature map and the second gradient feature map includes: Calculate the gradient difference value between the first gradient feature map and the second gradient feature map to obtain the gradient difference map; Based on a preset gradient-weight function, the gradient difference map is subjected to spatial weight mapping to generate the target spatial weight map.

4. The method according to claim 3, characterized in that, The step of performing spatial weight mapping processing on the gradient difference map according to a preset gradient-weight function to generate the target spatial weight map includes: For each gradient difference value contained in the gradient difference map, the gradient difference value is weighted according to the preset gradient-weight function to obtain the target weight of the pixel position corresponding to the gradient difference value. The target space weight map is obtained by constructing an image based on the target weights of the pixel positions corresponding to each gradient difference value.

5. The method according to claim 4, characterized in that, The preset gradient-weight function includes an upper threshold for gradient difference and a lower threshold for gradient difference; the step of performing weight mapping processing on the gradient difference value according to the preset gradient-weight function to obtain the target weight of the pixel position corresponding to the gradient difference value includes: If the gradient difference value is less than the lower limit threshold of the gradient difference, then the target weight of the pixel position corresponding to the gradient difference value is set as the first weight; If the gradient difference value is greater than the upper limit threshold of the gradient difference, then the target weight of the pixel position corresponding to the gradient difference value is set as the second weight; If the gradient difference value is greater than or equal to the lower threshold of gradient difference and less than or equal to the upper threshold of gradient difference, then the gradient difference value is subjected to linear weight mapping processing according to the upper threshold of gradient difference and the lower threshold of gradient difference to obtain a third weight, and the third weight is determined as the target weight of the pixel position corresponding to the gradient difference value. The third weight is less than the first weight and greater than the second weight.

6. The method according to claim 1, characterized in that, The step of performing image restoration on the initial artifact-removed image based on the detail information of the first illumination artifact image and the target spatial weight map to generate the target restored image includes: Based on the target space weight map, the detailed information is weighted to obtain compensation information; Based on the compensation information, the initial artifact-removed image is restored to obtain the target restored image.

7. The method according to any one of claims 1 to 6, characterized in that, The method for obtaining detailed information of the first illumination artifact image includes: The first illumination artifact image is downsampled to obtain the second illumination artifact image; The second illumination artifact image is upsampled to obtain a third illumination artifact image; the resolution of the third illumination artifact image is the same as the resolution of the first illumination artifact image. Based on the third illumination artifact image, the first illumination artifact image is subjected to detail separation processing to obtain the detail information of the first illumination artifact image.

8. The method according to claim 7, characterized in that, The method for obtaining the initial artifact-free image includes: The second illumination artifact image is processed using an illumination artifact removal model to obtain the initial artifact-removed image.

9. An image restoration device, characterized in that, The device includes: The acquisition module is used to acquire a second illumination artifact image and an initial artifact-free image determined based on a first illumination artifact image; the resolution of the second illumination artifact image is lower than the resolution of the first illumination artifact image; The analysis module is used to generate a target space weight map by performing image structure difference analysis on the second illumination artifact image and the initial artifact-removed image; The recovery module is used to perform image recovery on the initial artifact-removed image based on the detailed information of the first illumination artifact image and the target spatial weight map, and generate a target recovered image.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

12. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

Images