Image restoration method based on regional degradation constraint graph and closed loop re-degradation checking, computer device and storage medium

Abstract

The application discloses an image restoration method based on a region degradation constraint graph and a closed-loop re-degradation verification. Firstly, an image to be restored is acquired and divided into multiple image regions, and a degradation description vector of each region is extracted by using a degradation perception model; a region degradation constraint graph is constructed by combining the spatial relationship between the regions, structural similarity and degradation similarity, a restoration routing weight is generated based on the graph, and a preliminary restoration image is obtained according to the output of multiple restoration expert branches selected according to the restoration routing weight; the preliminary restoration image is subjected to re-degradation simulation, consistency verification is performed with the image to be restored, a description and a restoration strategy are updated in a closed loop according to the result, and finally a restoration image and a region restoration credibility map are output. The method is suitable for the restoration of complex mixed degradation images, improves the restoration precision and robustness, and can be widely applied to image enhancement, monitoring repair, remote sensing restoration and industrial detection scenes.

Description

Technical Field

[0001] This invention belongs to the field of image restoration, and specifically relates to an image restoration method based on region degradation constraint maps and closed-loop re-degradation verification. Background Technology

[0002] Image restoration is a key technology in computer vision and digital image processing, widely used in remote sensing images, surveillance videos, and image enhancement. In practical applications, images are typically affected by various degradation factors such as noise, blurring, compression artifacts, low light, fog, and rain / snow occlusion. These degradation factors may be independent or the result of multiple degradations acting together.

[0003] Most existing image restoration methods are based on a whole-image restoration strategy, which assumes uniform image degradation and uses a single restoration strategy for image processing. However, this approach often fails to achieve ideal restoration accuracy when dealing with the interplay of multiple degradation factors. In practical applications, images often exhibit significant regional differences; some areas may be heavily affected by noise, while others may be affected by blurring or compression artifacts. Whole-image restoration methods typically lack detailed modeling of these regional differences, resulting in restoration results that often fall short of high-quality restoration requirements.

[0004] For example, deep learning-based image restoration methods typically employ convolutional neural networks or generative adversarial networks for image inpainting. While these methods perform well in some simple degradation scenarios, their restoration effectiveness is easily limited when faced with complex degradation patterns. Especially when images are affected by multiple degradation factors, existing image restoration methods often lack adaptive dynamic adjustment mechanisms for degradation differences in different regions, and cannot effectively handle degradation differences between different regions, leading to problems such as over-enhancement or false textures in the restoration results.

[0005] To overcome these problems, increasing research has focused on region-level image restoration methods. These methods improve restoration accuracy by dividing the image into regions and processing the degradation of each region individually. However, most existing region restoration methods ignore the degradation correlation between regions or rely solely on a single feature to describe the relationship between regions. This limits the effectiveness of region restoration.

[0006] Therefore, how to effectively utilize the degradation correlation, spatial adjacency, structural consistency and semantic consistency between image regions, adopt adaptive restoration strategies for different regions, and jointly optimize the restoration process through inter-regional constraints has become a major challenge in the field of image restoration. Summary of the Invention

[0007] Purpose of the invention: In order to overcome the above shortcomings, the purpose of this invention is to provide an image restoration method based on region degradation constraint map and closed-loop re-degradation verification, which aims to solve the problems of poor image restoration effect in the prior art, especially in complex multi-degradation coupled scenarios, where the restoration results have problems such as over-enhancement and false textures.

[0008] To address this problem, this invention provides a novel image restoration method. This method introduces a region degradation constraint map to describe the regional constraint relationships between different regions of an image, determined by degradation similarity, spatial correlation, structural similarity, and semantic consistency. Simultaneously, it incorporates a closed-loop re-degradation verification mechanism to correct the degradation description and restoration strategy based on the consistency error between the simulated degraded image and the image to be restored, thereby improving the stability and reliability of the region-level restoration results.

[0009] Among them, the regional degradation constraint graph is not simply used for regional modeling, but affects the recovery route weights; the recovery route weights are not generated solely based on the single-region degradation vector, but are generated from the graph-enhanced degradation features after graph propagation; the re-degradation consistency error is not only used to evaluate the final image, but also to reverse-correct the degradation description vector and further change the edge weights and routes; the confidence graph is not an additional output, but is used to control the conservative recovery strength of low-confidence regions.

[0010] Technical Solution: To achieve the above objectives, this invention provides an image restoration method based on region degradation constraint maps and closed-loop re-degradation verification, comprising: S1): Obtain the image to be restored, which contains at least one degradation factor; S2): Divide the image to be restored into multiple initial image units, merge homogeneous regions to obtain multiple image regions, extract the spatial location, local structure and semantic information of each image region, establish the spatial adjacency relationship between image regions, and obtain a set of candidate adjacency edges for constructing the region degradation constraint graph; S3): Extract local degradation features from the image region using a degradation perception model to obtain the corresponding region degradation description vector; S4): Combining the spatial adjacency relationships, structural similarities, semantic consistency, and degradation similarities of each image region, a region degradation constraint map is constructed to characterize the region degradation association and recovery constraints; S5): Obtain graph augmentation degradation features and generate restoration route weights for each image region based on the graph augmentation degradation features; input the image region content into multiple restoration expert branches to obtain multiple candidate restoration features; then perform weighted fusion of multiple candidate restoration features based on the restoration route weights to obtain a preliminary restored image; S6): Generate the re-degradation parameters for the corresponding image region, and perform re-degradation simulation on the preliminary restored image based on the re-degradation parameters. Then, fuse the degradation simulation results to obtain the simulated degradation image. Perform consistency verification between the simulated degradation image and the image to be restored to obtain the consistency error. S7): Based on the consistency error, perform closed-loop updates on the region degradation description vector, region degradation constraint map, and recovery route weights, and regenerate the recovery results based on the updated recovery route weights, outputting the final recovery image and region recovery confidence map.

[0011] The image restoration method based on region degradation constraint graph and closed-loop re-degradation verification, wherein the specific process of obtaining the candidate adjacent edge set for constructing the region degradation constraint graph in step S2) is as follows: S201): Calculate the structural consistency score between adjacent initial image units according to the features of the divided initial image units. When the structural consistency score is greater than or equal to the preset structural consistency threshold, merge the corresponding adjacent initial image units into the same image region to obtain multiple structurally consistent image regions. S202): Record the spatial location information of each image region in the image to be restored; S203): Extract the local structural and semantic information of each image region. S204): Based on the spatial adjacency relationship, center point distance, structural similarity and semantic consistency between image regions, establish candidate adjacency relationships between image regions to obtain a set of candidate adjacency edges for constructing a region degradation constraint graph.

[0012] The image restoration method based on regional degradation constraint map and closed-loop re-degradation verification, wherein the specific method of the regional degradation description vector in S3) is as follows: S301): Input each image region into the degradation perception model and extract the local degradation features corresponding to the image region; S302): Local degradation features are input into multiple degradation prediction heads to obtain the region degradation description vector corresponding to each image region; S303): The region degradation description vector includes at least one of noise intensity, fuzz scale, fuzz direction, compression artifact intensity, low illumination level, fog level, probability of rain and snow occlusion, and degradation uncertainty, wherein the degradation uncertainty is output by the uncertainty prediction head in the degradation perception model, and the degradation uncertainty includes at least one of degradation parameter prediction variance, logarithmic variance, or confidence inverse quantization value.

[0013] Furthermore, in the image restoration method based on region degradation constraint maps and closed-loop re-degradation verification, the specific process of constructing a region degradation constraint map to characterize region degradation associations and restoration constraints is as follows: S401): Treat each image region as a node in the region degradation constraint graph, and use the corresponding region degradation description vector, spatial location, local structural information, and semantic information as node features; S402): Establish candidate edges based on the spatial adjacency relationship between different image regions; S403): Based on the degradation description vector similarity, texture continuity, edge direction consistency, brightness distribution similarity, and semantic category consistency between different image regions, the degradation similarity, structural similarity, and semantic consistency are quantized respectively, and the edge weight of the candidate edge is calculated accordingly. S404): Construct a region degradation constraint graph based on nodes, node features, candidate edges, and edge weights.

[0014] Furthermore, in the image restoration method based on region degradation constraint maps and closed-loop re-degradation verification, the region degradation constraint map is represented as follows: in, This represents a set of nodes, where each node corresponds to an image region. This represents a set of edges used to connect image regions that exhibit degenerate associations; Represents the adjacency weight matrix; No. The node and the first Edge weights between nodes Determined in the following ways: in, Represents the row normalization function. Representing the normalized degraded similarity between two image regions, based on the region degradation description vector. The cosine similarity between them is determined; They represent the first The image region and the first Region degradation description vectors for each image region; This represents the normalized spatial correlation between two image regions. They represent the first The image region and the first Spatial location information of each image region; This represents the normalized structural similarity between two image regions. They represent the first The image region and the first Local structural features of an image region; This indicates the normalized semantic consistency between two image regions. They represent the first The image region and the first Semantic information of each image region; Let represent the weight coefficients for degenerate similarity, spatial relevance, structural similarity, and semantic consistency, respectively, and satisfy the following: and The similarity function The output values ​​are normalized to the same range to avoid bias in edge weight calculation due to differences in units between different feature measures.

[0015] Furthermore, the specific process for obtaining the preliminary restored image is as follows: S501): Input the node features in the region degradation constraint graph into the graph information propagation module to obtain the graph enhancement degradation features of each image region; S502): Input the image enhancement degradation features into the recovery route generation module to obtain the recovery route weights corresponding to each image region; S503: Input the image content of each image region in the image to be restored into the shared feature encoding module to obtain the corresponding region input features, and input the region input features into multiple restoration expert branches based on deep neural networks to obtain multiple candidate restoration features; S504: Based on the recovery route weights corresponding to each image region, multiple candidate recovery features are weighted and fused to obtain the region recovery features of the corresponding image region; S505: Input the region restoration features corresponding to each image region into the image reconstruction module, and fuse them according to the spatial coordinates or region mask of each image region to obtain a preliminary restored image.

[0016] More preferably, the restoration expert branch constructed based on the deep neural network includes at least one of the following: denoising restoration expert branch, deblurring restoration expert branch, compression artifact removal restoration expert branch, defogging restoration expert branch, rain and snow occlusion removal restoration expert branch, and low-light enhancement expert branch; The recovery route weights are used to control the degree of participation of different recovery expert branches in the corresponding image regions, and the first... The recovery routing weights corresponding to each image region satisfy the following: in, Indicates the first The image region for the first Recovery route weights for each recovery expert branch. Indicates the number of expert branches for recovery; the first The region restoration features of each image region are obtained through the following methods: in, express Region restoration features of the image region Indicates the first Input features of each image region Indicates the first A recovery expert branch built on a deep neural network.

[0017] Furthermore, in step S7), based on the consistency error, a closed-loop update is performed on the region degradation description vector, the region degradation constraint map, and the recovery route weights. The recovery result is then regenerated based on the updated recovery route weights, and the final recovered image and region recovery confidence map are output. The specific process is as follows: S701): Calculate the local consistency error corresponding to each image region based on the consistency error. When the local consistency error of the target image region is greater than the preset error threshold, correct the region degradation description vector corresponding to the target image region. S702): Recalculate the edge weights between the target image region and other image regions based on the corrected region degradation description vector, and update the region degradation constraint graph; S703): Regenerate the recovery route weights based on the updated region degradation constraint map and adjust the recovery intensity of the corresponding image regions; S704): Based on the adjusted recovery route weights and recovery strength, the recovery results are regenerated to obtain the final recovered image; S705): Generate a region restoration confidence map based on local consistency error, degradation estimation uncertainty, and structural changes before and after restoration; S706): When the confidence level of a certain image region in the region restoration confidence map is lower than the preset confidence threshold, reduce the texture generation intensity, edge enhancement intensity and detail compensation intensity of the image region, and retain more original observation information from the image to be restored, so as to reduce false textures and over-restoration.

[0018] It also includes a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements an image restoration method based on a region degradation constraint map and closed-loop re-degradation verification.

[0019] It also includes a computer-readable storage medium storing a computer program that, when executed by a processor, implements an image restoration method based on a region degradation constraint map and closed-loop re-degradation verification.

[0020] As can be seen from the above technical solution, the present invention has the following beneficial effects: The image restoration method based on regional degradation constraint graphs and closed-loop re-degradation verification described in this invention can achieve accurate region-level restoration. In this scheme, by introducing regional degradation constraint graphs, the regional constraint relationships between different regions of the image, which are jointly determined by degradation similarity, spatial correlation, structural similarity, and semantic consistency, can be effectively described. Thus, restoration route weights are adaptively generated according to the specific degradation characteristics of each region and the consistency constraints between regions, avoiding the problem that traditional whole-image unified restoration strategies are difficult to adapt to regional degradation differences, and improving restoration accuracy and detail preservation capabilities.

[0021] Multi-degradation factor processing capability: This invention can simultaneously process multiple degradation factors in an image (such as noise, blur, compression artifacts, low illumination, etc.), and optimize the restoration process through a closed-loop re-degradation verification mechanism, avoiding over-enhancement and the generation of false textures, thereby making the restoration result more natural and realistic.

[0022] Improving the credibility of recovery results: This invention effectively ensures the high accuracy and reliability of recovery results by quantifying the credibility of the recovery results for each region and combining it with a closed-loop feedback update mechanism for consistency error, thus avoiding the problem of inaccurate assessment of recovery quality in traditional methods.

[0023] Broad application prospects: The image restoration method of the present invention has strong versatility and is applicable to multiple fields such as image enhancement, monitoring restoration, remote sensing image restoration, and low-light image enhancement. It can achieve high-quality image restoration in complex and variable degradation scenarios, and has strong market potential and wide application value.

[0024] In summary, this invention introduces a region degradation constraint graph composed of degradation similarity, spatial correlation, structural similarity, and semantic consistency, and combines it with a closed-loop re-degradation verification mechanism to iteratively update the region degradation description vector, constraint edge weights, and recovery route weights. This improves the stability and reliability of existing image restoration techniques in scenarios with multiple degradation coupling and significant differences in region degradation. Attached Figure Description

[0025] Figure 1 The flowchart shows the image restoration method based on region degradation constraint map and closed-loop re-degradation verification. Figure 2 A schematic diagram illustrating the process of constructing a regional degradation constraint map. Detailed Implementation

[0026] The present invention will be further explained below with reference to the accompanying drawings and specific embodiments. Example

[0027] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0028] 1. An image restoration method based on region degradation constraint maps and closed-loop re-degradation verification, characterized in that it includes: S1): Obtain an image to be restored containing at least one degradation factor, including noise, blur, compression artifacts, low light, fogging, and rain / snow occlusion; S2): Divide the image to be restored into multiple initial image units, merge homogeneous regions to obtain multiple image regions, extract the spatial location, local structure and semantic information of each image region, establish the spatial adjacency relationship between image regions, and obtain a set of candidate adjacency edges for constructing the region degradation constraint graph; S3): Extract local degradation features from the image region using a degradation perception model to obtain the corresponding region degradation description vector; S4): Combining the spatial adjacency relationships, structural similarities, semantic consistency, and degradation similarities of each image region, a region degradation constraint map is constructed to characterize the region degradation association and recovery constraints; S5): Obtain graph augmentation degradation features and generate restoration route weights for each image region based on the graph augmentation degradation features; input the image region content into multiple restoration expert branches to obtain multiple candidate restoration features; then perform weighted fusion of multiple candidate restoration features based on the restoration route weights to obtain a preliminary restored image; S6): Generate the re-degradation parameters for the corresponding image region, and perform re-degradation simulation on the preliminary restored image based on the re-degradation parameters. Then, fuse the degradation simulation results to obtain the simulated degradation image. Perform consistency verification between the simulated degradation image and the image to be restored to obtain the consistency error. S7): Based on the consistency error, perform closed-loop updates on the region degradation description vector, region degradation constraint map, and recovery route weights, and regenerate the recovery results based on the updated recovery route weights, outputting the final recovery image and region recovery confidence map.

[0029] In a preferred embodiment of this invention, the image to be restored originates from natural image acquisition devices, surveillance cameras, remote sensing imaging devices, industrial inspection cameras, vehicle-mounted cameras, and mobile terminal cameras. Because the imaging process in real-world scenes is easily affected by factors such as lighting conditions, lens shake, sensor noise, compression encoding, weather conditions, and shooting distance, the acquired images often do not contain only a single degradation factor, but rather multiple degradation factors simultaneously, and the distribution of different degradation factors in the image space is not uniform. For example, dark areas of the image may contain strong noise, the edges of moving targets may be blurred, the sky and distant areas may be fogged, and local areas of the image may also be affected by compression artifacts or rain / snow occlusion.

[0030] In one specific implementation, the acquired image to be restored is denoted as... Image height is The image width is The number of image channels is [value missing]. For a regular color image, [value missing]. The image to be recovered can be a single image or a single frame selected from a video sequence. To facilitate subsequent processing, the image to be recovered undergoes preprocessing operations such as size normalization, pixel value normalization, and color space conversion. For example, the image pixel values ​​are normalized to... Within the range.

[0031] Furthermore, in one specific embodiment, the image to be restored also carries auxiliary information related to image acquisition, including the type of shooting device, exposure time, ISO value, focal length, compression quality parameters, shooting time, weather conditions, scene category, etc. The aforementioned auxiliary information is not a necessary condition for the implementation of this invention, but when such information is present, it can serve as an auxiliary input to the degradation perception model to improve the estimation accuracy of the degradation description vector.

[0032] In this embodiment, the specific process of obtaining the candidate adjacent edge set for constructing the region degradation constraint graph in S2) is as follows: S201): Divide the image to be restored into multiple initial image units, and calculate the structural consistency score between adjacent initial image units based on the edge response, texture complexity, brightness distribution, color distribution and gradient direction of each initial image unit. Then, perform weighted fusion of the multiple sub-consistency scores to obtain the structural consistency score between adjacent initial image units. When the structural consistency score is greater than or equal to a preset structural consistency threshold, merge the corresponding adjacent initial image units into the same image region to obtain multiple structurally consistent image regions. The preset structural consistency threshold ranges from 0.5 to 0.9. Specifically, the initial image unit can be a regular rectangular image block or an irregular image region obtained based on the superpixel method. When using a regular rectangular image block, the image to be restored can be divided according to a preset block size and a preset step size; when using an irregular image region, the initial superpixel region can be generated based on pixel color, spatial distance, edge response, and gradient changes. For any two spatially adjacent initial image units, a structural consistency score is calculated between them. The structural consistency score is determined by edge consistency, texture consistency, brightness distribution consistency, color distribution consistency, and gradient direction consistency. If the structural consistency score between two adjacent initial image units is greater than or equal to a preset structural consistency threshold, they are merged; if the structural consistency score is less than the preset structural consistency threshold, they are retained as different image regions. One specific implementation method is as follows: Let the image to be recovered be... Classified as Given a set of image regions, the region set is obtained: in, Indicates the first Image regions, This indicates the number of image regions. Each region can be a regular rectangular image block or an irregularly structured region. S202): Record the spatial location information of each image region in the image to be recovered, wherein the spatial location information includes at least one of spatial coordinates, region size, region center position, region mask, and adjacent region index; Specifically, the first The spatial location information of an image region can be represented as: in, and Indicates the first The x and y coordinates of the top-left corner of the bounding rectangle of the image region in the image to be recovered. and Let represent the width and height of the circumscribed rectangle, respectively. Indicates the first The center point coordinates of each image region Indicates the first The region mask corresponding to each image region; for a regular rectangular region... It can be represented as For regions with consistent irregular structures, It can be represented as the average of all pixel coordinates within the region; region mask. For a binary mask corresponding to the spatial size of the image to be restored, when the pixel position belongs to the first... When there is an image region, the corresponding position is set to 1; otherwise, it is set to 0. The above spatial coordinates, region size and center point coordinates can be normalized to the range of [0,1] according to the width and height of the image to be restored. Furthermore, based on the boundary contact relationship and center point distance between image regions, the adjacent region index of each image region is recorded. If two image regions share a common boundary in space, or if their normalized center point distance is less than or equal to a preset adjacent distance threshold, then the two image regions are recorded as adjacent regions. For the ... The image region and the first Each image region, the distance of the normalized center point It can be represented as: in, and They represent the first The image region and the first The center point coordinates of each image region and These represent the height and width of the image to be recovered, respectively. The preset adjacency distance threshold ranges from 0.02 to 0.20, preferably from 0.05 to 0.10. S203): Extract local structural information and semantic information of each image region, wherein the local structural information includes at least one of edge intensity (edge ​​response), texture complexity, brightness distribution, color distribution and gradient direction; Specifically, edge intensity can be extracted using the Canny edge detection operator, texture statistical features can be extracted using the gray-level co-occurrence matrix, and color distribution features can be extracted using color histograms or higher-order color statistics. For the th The local structural features of an image region can be represented as follows: in, Indicates edge strength. Indicates texture complexity. Indicates the characteristics of brightness distribution. Indicates color distribution characteristics, This represents the gradient direction feature. To avoid aligning with the region center... Confusion, color distribution features are used express.

[0033] Among them, edge strength can be determined by the mean edge response within the region, texture complexity can be determined by the texture energy or texture variance within the region, brightness distribution characteristics can be determined by the mean brightness, brightness variance or brightness histogram, color distribution characteristics can be determined by the mean color or color histogram, and gradient direction characteristics can be determined by the gradient direction histogram or principal gradient direction.

[0034] Furthermore, semantic information for each image region can be extracted using semantic segmentation models or visual semantic feature extraction models. The semantic information of an image region can be represented as: in, This represents the semantic feature extraction function. Indicates the first Semantic features of each image region; S204): Based on the spatial adjacency relationship, center point distance, structural similarity and semantic consistency between image regions, establish candidate adjacency relationships between image regions to obtain a set of candidate adjacency edges for constructing the region degradation constraint graph; Specifically, for any two image regions and If two entities are spatially adjacent, or the distance between their center points is less than a preset distance threshold, or their structural feature similarity is greater than a preset structural similarity threshold, or their semantic feature similarity is greater than a preset semantic similarity threshold, then they are considered candidate associated regions, and a candidate adjacency relationship is established. This candidate adjacency relationship is used to subsequently construct candidate edges in the region degradation constraint graph.

[0035] In one specific implementation, the candidate adjacent edge set can be represented as: in, Represents the set of candidate adjacent edges. Indicates the i-th image region and the i-th image region. Does a candidate adjacency relationship exist between image regions? When =1, it indicates that there is a candidate adjacency relationship between two image regions; when When =0, it means that there is no candidate adjacency relationship between the two image regions; Through the above steps, multiple image regions with spatial location, structural features and semantic information can be obtained, which provides a foundation for subsequent region degradation description vector extraction, region degradation constraint map construction and region-level adaptive restoration. It is important to note that the region segmentation does not rely solely on fixed blocks or ordinary superpixel results, but rather on merging adjacent image units based on their consistency in local structure and appearance distribution, thereby obtaining image regions that are more suitable for subsequent degradation estimation and region constraint modeling. Preferably, the region division does not treat the entire image to be restored as a uniform degradation object. Instead, it divides the image to be restored into multiple image regions with relatively consistent structures based on edge response, texture complexity, brightness distribution, color distribution, and gradient direction. In this way, subsequent degradation perception, region degradation constraint map construction, and restoration route generation can all be performed at the region level, thereby adapting to the problem of uneven degradation distribution in real-world scene images. It should be noted that the recorded region mask, region center coordinates, and adjacent region index are the core data carriers for determining spatial adjacency. Based on the stored spatial location information, the spatial adjacency relationship between image regions is determined using two types of criteria: Geometric position determination: Compare the region masks and coordinate boundaries of the two regions to determine whether the regions are physically adjacent on the image plane (boundaries overlap / stick together); Center point distance filtering: Calculate the Euclidean distance between the center positions of two regions. If the distance is less than the preset spatial distance threshold, it is determined that the two regions are spatially adjacent. Regions that meet the above geometric adjacency or center distance criteria are considered to have a spatial adjacency relationship and are included in the candidate adjacency edge set for subsequent construction of the region degradation constraint graph. In one specific implementation, the degradation-aware model is used to extract degradation-related features from each image region and output a region degradation description vector corresponding to that image region. The degradation-aware model includes a shared visual encoder and multiple degradation prediction heads. The shared visual encoder is a feature extraction network used by all image regions to map input region image patches to local degradation features; the multiple degradation prediction heads are multiple parameter prediction branches connected after the shared visual encoder, each prediction branch outputting a degradation parameter or degradation-related parameter. Specifically, for the first Each image region is first cropped or resampled into uniformly sized image patches, and then pixel normalization is performed. These image patches are then input into a shared visual encoder to obtain local degradation features. The shared visual encoder can employ a convolutional neural network, a visual Transformer network, or a combination of both, such as a visual Transformer structure consisting of an image patch embedding layer, multiple self-attention encoding layers, and a feature convergence layer. Multiple degradation prediction heads all use these local degradation features as input and can consist of fully connected layers, non-linear activation layers, and an output layer.

[0036] The multiple degradation prediction heads include a noise intensity prediction head, a fuzzy scale prediction head, a fuzzy direction prediction head, a compression artifact intensity prediction head, a low-light intensity prediction head, a fog intensity prediction head, a rain / snow occlusion probability prediction head, and a degradation uncertainty prediction head. The outputs of the noise intensity prediction head, fuzzy scale prediction head, compression artifact intensity prediction head, low-light intensity prediction head, fog intensity prediction head, and rain / snow occlusion probability prediction head are set with a Sigmoid function to normalize the corresponding outputs to the range of 0 to 1; a larger value indicates a stronger corresponding degradation factor. The fuzzy scale prediction head is an independent prediction branch among the above multiple degradation prediction heads, used to output a normalized fuzzy scale, which characterizes the size of the fuzzy kernel scale, fuzzy standard deviation, or motion fuzz length. The fuzzy direction prediction head is used to output fuzzy direction parameters, which can output a two-dimensional direction vector, and determine the fuzzy direction angle based on this two-dimensional direction vector.

[0037] The degradation uncertainty prediction head is used to output the uncertainty of the degradation parameter prediction results. The degradation uncertainty is used to characterize the reliability of the degradation estimation results of the degradation perception model for the current image region. The degradation uncertainty can be expressed as the degradation parameter prediction variance, logarithmic variance, or inverse quantization value of the confidence score, and can be guaranteed to be non-negative by using the Softplus function or exponential function.

[0038] In one specific implementation, the first The region degradation description vector for each image region includes noise intensity, blur scale, blur direction, compression artifact intensity, low illumination level, fog level, probability of rain / snow occlusion, and degradation uncertainty. For degradation parameters with practical image processing implications, the prediction head can first output normalized values ​​in the range of 0 to 1, and then map them to the actual degradation parameters according to a preset parameter range. For example, normalized noise intensity can be mapped to noise standard deviation, normalized blur scale can be mapped to blur kernel size or motion blur length, and normalized compression artifact intensity can be mapped to the inverse quantization value of the compression quality parameter.

[0039] The degradation perception model can be trained by synthesizing degradation samples. Specifically, a clear image or a clear image region is selected as a reference sample, and degradations such as noise, blur, compression, low light, fog, and rain / snow occlusion are generated according to a preset parameter range. The degradation parameters used in the synthesis process are recorded. Since these parameters are preset and recorded by the system during the synthesis process, they can be used as known degradation parameter labels for supervised training of the corresponding degradation prediction head.

[0040] During training, the network parameters of the shared visual encoder and multiple degradation prediction heads are adjusted via backpropagation based on the differences between the outputs of each degradation prediction head and the known degradation parameter labels. For continuous parameters such as noise intensity, blur scale, compression artifact intensity, low illumination level, and fog level, L1 loss, L2 loss, or smoothed L1 loss can be used; for blur direction, direction vector angle loss or cosine similarity loss can be used; for rain and snow occlusion probability, cross-entropy loss or regression loss can be used; for degradation uncertainty, negative log-likelihood loss based on prediction variance can be used. During the inference phase, the degradation perception model does not require real degradation parameter labels or retraining of network parameters, but directly outputs the region degradation description vector; subsequent closed-loop re-degradation verification only updates the degradation description vector, region degradation constraint map, and recovery route weights of the current image.

[0041] The specific method for the region degradation description vector in S3) described in this embodiment is as follows: S301): Input each image region into the degradation perception model and extract the local degradation features corresponding to that image region; Specifically: for the first Image regions First, the image is cropped or resampled into uniformly sized region image blocks, and then pixel normalization is performed to obtain normalized region image blocks. Then, the normalized region image blocks are input into the shared visual encoder. The local degradation characteristics are obtained: in, This represents the shared visual encoder in the degradation perception model. Indicates the first Local degradation features are extracted from image regions using a shared visual encoder. These local degradation features are in vector or feature map form and are used to characterize degradation information in the image region related to noise distribution, blurred edges, compression block effect, low-light response, fogging scattering, and texture loss. S302): Local degradation features are input into multiple degradation prediction heads to obtain the region degradation description vector corresponding to each image region; Specifically, this refers to: localized degradation features Input the degradation description prediction layer to obtain the region degradation description vector: in, This indicates a degradation description prediction function. Indicates the first Region degradation description vectors corresponding to each image region; S303): The region degradation description vector includes at least one of noise intensity, blur scale, blur direction, compression artifact intensity, low illumination level, fog level, probability of rain and snow occlusion, and degradation uncertainty. The degradation uncertainty is output by the uncertainty prediction head in the degradation perception model and is used to characterize the reliability of the degradation parameter estimation results corresponding to the image region. The degradation uncertainty includes at least one of degradation parameter prediction variance, logarithmic variance, or confidence inverse quantization value. Specifically: the first The region degradation description vector for an image region can be represented as: in, Indicates noise intensity. Indicates fuzzy scale. This indicates fuzzy direction encoding. Indicates the intensity of compression artifacts. Indicates the degree of low light intensity. Indicates the degree of atomization. Indicates the probability of being blocked by rain or snow. This indicates the uncertainty in the degradation estimate; Furthermore, noise intensity, blur scale, compression artifact intensity, low light level, and fog level are output by the corresponding prediction heads via the Sigmoid function, with values ​​ranging from 0 to 1. Higher values ​​indicate stronger degradation factors. The probability of rain / snow occlusion is output by the rain / snow occlusion probability prediction head via the Sigmoid function, with values ​​ranging from 0 to 1, representing the probability of rain / snow occlusion in the image region.

[0042] The fuzzy direction encoding does not directly use angle values, but is represented by a two-dimensional direction vector output by the fuzzy direction prediction head, or obtained by converting fuzzy direction angles. For example, when the fuzzy direction angle is... When the fuzzy direction coding is used, it can be represented as follows: It can also be further mapped to By using two-dimensional directional encoding, the distance calculation deviation caused by the periodicity of angles when using angle values ​​directly can be avoided. The degradation estimation uncertainty is output by the degradation uncertainty prediction head and converted to a non-negative value through the Softplus function or exponential function. Before participating in the construction of the region degradation description vector or the degradation similarity calculation, it can be mapped to the range of 0 to 1 according to a preset upper limit or statistical normalization method. The larger the degradation estimation uncertainty value, the less reliable the model's estimation result of the degradation parameters of the image region; the smaller the value, the more reliable the estimation result. Therefore, the region degradation description vector is composed of a normalized degradation intensity parameter, a two-dimensional fuzzy direction code, and a normalized degradation uncertainty parameter. Among them, the fuzzy direction component participates in the subsequent similarity calculation according to the two-dimensional direction code, and the remaining degradation intensity component and degradation uncertainty component participate in the subsequent similarity calculation according to the normalized value in the range of 0 to 1, thereby ensuring that the degradation similarity calculation between different image regions has a consistent numerical scale. S304): The region degradation description vector is used to distinguish the degradation differences of different regions in the image to be restored, so that the subsequent restoration process can adopt different restoration strategies for different regions; Specifically, noise intensity, blur scale, compression artifact intensity, low light level, fog level, and rain / snow occlusion probability are all normalized parameters ranging from 0 to 1. For each of these degradation parameters, a corresponding degradation judgment threshold is set, with the threshold value ranging from 0.5 to 0.8, preferably 0.6. When the noise intensity, blur scale, compression artifact intensity, low light level, fog level, or rain / snow occlusion probability of a certain image region is greater than or equal to the corresponding degradation judgment threshold, the image region is determined as a significantly degraded region, and the routing score of the corresponding restoration expert branch is increased during the restoration route generation process. Specifically, noise intensity corresponds to the denoising restoration expert branch, blur scale to the deblurring restoration expert branch, compression artifact intensity to the compression artifact removal restoration expert branch, low light level to the low light enhancement expert branch, fog level to the defogging restoration expert branch, and rain / snow occlusion probability to the rain / snow occlusion removal restoration expert branch. The adjusted routing scores of each expert branch are normalized using Softmax to obtain the restoration route weight of each restoration expert branch. Furthermore, the degradation uncertainty is used to adjust the degree of conservatism in the restoration of image regions. A preset uncertainty threshold is set for the degradation uncertainty, with a value ranging from 0.5 to 0.85, preferably 0.65. When the degradation uncertainty of a certain image region is greater than or equal to the preset uncertainty threshold, the restoration intensity coefficient of that image region is reduced, so that the final region restoration result retains more of the original observation information in the image to be restored; when the degradation uncertainty is less than the preset uncertainty threshold, the restoration intensity coefficient of that image region is maintained or increased, so that the fusion results of the restoration expert branches participate more in the final restoration. Thus, the region degradation description vector is not only used to determine the degradation type and degradation intensity of different regions, but also to control the participation weight of different restoration expert branches and the restoration intensity of the corresponding regions.

[0043] In S4), the spatial adjacency relationship between image regions is determined based on spatial location, structural similarity is calculated based on local structure, semantic consistency is calculated based on semantic information, and degradation similarity is calculated based on the region degradation description vector. Based on the calculated degradation similarity, spatial adjacency relationship, structural similarity, and semantic consistency, a region degradation constraint graph is constructed to describe the degradation association relationship and recovery constraint relationship between different image regions.

[0044] In this invention, the region degradation constraint graph is a graph structure with image regions as nodes and the relationships between image regions as edges. Each node corresponds to an image region, and the node features include the region degradation description vector, spatial location information, local structural information, and semantic information of that image region. The edge between any two image regions represents the degree of association between them in terms of degradation state, spatial location, structural features, or semantic features. This degree of association is jointly determined by degradation similarity, spatial correlation, structural similarity, and semantic consistency. A larger edge weight indicates a stronger consistency constraint that the two image regions need to maintain during subsequent degradation estimation, restoration route generation, and restoration result fusion.

[0045] Specifically, degradation similarity measures whether the degradation types and intensities of two image regions are similar; spatial correlation measures whether two image regions are spatially adjacent or close in distance; structural similarity measures whether the edge, texture, brightness, and color distributions of two image regions are similar; and semantic consistency measures whether the semantic categories or semantic features of two image regions are similar. The region degradation constraint graph is used to guide graph information propagation, recovery route weight generation, and region restoration result fusion, but not to represent causal relationships or degradation propagation directions between image regions.

[0046] In one specific implementation, reference is made to... Figure 2 The process of constructing the region degradation constraint graph includes processing input information, building graph nodes, establishing candidate edges, calculating edge weights, filtering valid connections and updating the graph structure, and generating the region degradation constraint graph. The region degradation constraint graph does not merely indicate whether image regions are adjacent, but further represents the correlation between different regions in terms of degradation factors and the constraints for restoring consistency.

[0047] The specific process for constructing a region degradation constraint graph to characterize region degradation associations and recovery constraints is as follows: S401): Each image region is treated as a node in the region degradation constraint graph, and the corresponding region degradation description vector, spatial location, local structural information and semantic information of the image region are used as node features; In one specific implementation, the region degradation constraint map is represented as follows: in, This represents a set of nodes, where each node corresponds to an image region. This represents a set of edges used to connect image regions that exhibit degenerate associations; Represents the adjacency weight matrix; No. The node characteristics of a node can be represented as: in, This represents a vector describing the region degradation. Indicates spatial location information, Represents local structural information. S402): Representing semantic information, which is represented by high-level semantic features of the image region; S403): Establishing candidate edges based on the spatial adjacency relationships between different image regions; Specifically, if two image regions are spatially adjacent, or the distance between their center points is less than a preset distance threshold, or they are structurally continuous, then candidate edges are established between the corresponding nodes. The set of candidate edges can be represented as: or in, Denotes the set of candidate edges. Indicates whether two image regions are spatially adjacent. This represents the distance between the center points of two regions. Indicates the spatial distance threshold; S403): Based on the degradation description vector similarity, texture continuity, edge direction consistency, brightness distribution similarity, and semantic category consistency between different image regions, the degradation similarity, structural similarity, and semantic consistency are quantized respectively, and the edge weight of the candidate edge is calculated accordingly. In one specific implementation, the first The node and the first Edge weights between nodes Determined in the following ways: in, Represents the row normalization function. Representing the normalized degraded similarity between two image regions, based on the region degradation description vector. The cosine similarity between them is determined; They represent the first The image region and the first Region degradation description vectors for each image region; It represents the normalized spatial correlation between two image regions, which is determined by transforming the distance between the center points of the two image regions using an exponential decay function; They represent the first The image region and the first Spatial location information of each image region; The normalized structural similarity between two image regions is determined by normalization transformation based on the distance between edge intensity, texture complexity, brightness distribution and gradient direction in local structural features; They represent the first The image region and the first Local structural features of an image region; The normalized semantic consistency between two image regions is determined based on the category consistency score between semantic information or the feature inner product of semantic vectors. They represent the first The image region and the first Semantic information of each image region; Let represent the weight coefficients for degenerate similarity, spatial relevance, structural similarity, and semantic consistency, respectively, and satisfy the following: and The similarity function The output values ​​are normalized to the same range to avoid bias in edge weight calculation due to differences in units between different feature measures. It should be noted that degenerate similarity, spatial correlation, structural similarity, and semantic consistency are all converted into normalized similarity scores before participating in edge weight calculation; Specifically, degradation similarity is calculated based on the normalized region degradation parameter encoding. For the i-th and j-th image regions, noise intensity, blur scale, compression artifact intensity, low illumination level, fog level, and rain / snow occlusion probability are extracted from their region degradation description vectors to form a degradation intensity vector. The values ​​of each of these components range from 0 to 1. For blur direction, the angle value is not directly used in the similarity calculation, but the two-dimensional direction component in the blur direction encoding is used to calculate the direction consistency. For degradation uncertainty, it is not directly used as a degradation intensity component in the similarity calculation, but is used to adjust the reliability of degradation similarity. When the degradation uncertainty of two image regions is high, the impact of their degradation similarity on the edge weight calculation is reduced. The similarity between degradation intensity vectors can be determined by the exponential decay function of normalized Euclidean distance, cosine similarity, or a weighted sum of the two; the consistency of fuzzy direction can be determined by the dot product or cosine similarity between two two-dimensional directional codes; the final degradation similarity is determined by degradation intensity similarity, fuzzy direction consistency, and degradation uncertainty adjustment factor, and normalized to the range of 0 to 1. Spatial correlation can be obtained by transforming the distance between the center points of two image regions using an exponential decay function; structural similarity can be determined by the normalized similarity between edge strength, texture complexity, brightness distribution, color distribution, and gradient direction; semantic consistency can be determined by semantic category consistency score or semantic feature vector similarity. Through the above normalization processing, feature measures from different sources have a consistent numerical range, and then weighted fusion is performed through weight coefficients to obtain stable region degradation constraint edge weights. S404): Construct a region degradation constraint graph based on nodes, node features, candidate edges, and edge weights to characterize the local correlation and global consistency constraints of degradation factors between different regions in an image; Specifically, after calculating the candidate edge weights, valid connections are filtered based on a preset edge weight threshold. When the edge weight of a candidate edge is greater than or equal to the preset threshold, the candidate edge is retained; when the edge weight of a candidate edge is less than the preset threshold, the candidate edge is deleted. This method preserves degenerate associations between highly correlated regions and reduces interference between weakly correlated regions. In one specific implementation, the set of valid edges can be represented as: in, This represents the edge weight threshold. It is derived from the set of nodes. Set of effective edges and adjacency weight matrix Together, they form a region degradation constraint graph. In this graph, nodes represent image regions, edges represent degradation relationships between regions, and edge weights are determined by degradation similarity, structural similarity, semantic consistency, and spatial correlation.

[0048] In step S5), the node features of the region degradation constraint graph are propagated to obtain graph enhancement degradation features, and restoration route weights corresponding to each image region are generated based on the graph enhancement degradation features. At the same time, the image region content is input to multiple restoration expert branches to obtain multiple candidate restoration features. Subsequently, the multiple candidate restoration features are weighted and fused based on the restoration route weights to obtain a preliminary restored image.

[0049] Preferably, the restoration routing weights are used to control the degree of participation of different restoration expert branches in corresponding image regions, enabling different image regions to adopt different restoration strategies based on their degradation type, degradation intensity, and inter-region degradation constraints. Compared with using a single restoration network to process the entire image uniformly, this method can dynamically select and fuse the outputs of different restoration expert branches based on regional degradation differences, thereby improving the restoration effect of complex mixed degradation images.

[0050] The specific process for obtaining the preliminary restoration map is as follows: S501): Input the node features in the region degradation constraint graph into the graph information propagation module to obtain the graph enhancement degradation features of each image region; In one specific implementation, the graph information propagation module is implemented using a graph convolutional network or a graph attention network. The graph information propagation module is used to retain the degradation features of each image region while simultaneously weighting and fusing the degradation information, structural information, and spatial information of adjacent or related image regions according to the constraint edge weights in the region degradation constraint graph, thereby obtaining graph-enhanced degradation features that satisfy region consistency constraints. For the ... The process of information propagation in a layer diagram can be represented as follows: in, Indicates the first Layer node feature matrix Represents the adjacency weight matrix. Indicates the first Layer learnable parameter matrix, This represents a non-linear activation function. After several layers of propagation, the image enhancement degradation features of each image region are obtained. ; S502): Input the image enhancement degradation features into the recovery route generation module to obtain the recovery route weights corresponding to each image region; In one specific implementation, the first The recovery routing weights for each image region are represented as follows: in, Indicates the first Each image region corresponds to Weight distribution of each recovery expert branch, This represents the route generation function. Indicates the first The image enhancement degradation features of each image region, after being normalized by Softmax, restore the routing weights to satisfy the following: in, Indicates the first The image region for the first The participation weight of each restoration expert branch; the restoration routing weight is used to control the degree of participation of different restoration expert branches in the corresponding image region; S503: Input the image content of each image region in the image to be restored into the shared feature encoding module to obtain the corresponding region input features, and input the region input features into multiple restoration expert branches based on deep neural networks to obtain multiple candidate restoration features; In one specific implementation, the shared feature encoding module consists of convolutional layers, residual blocks, or Transformer encoding layers, used to encode image regions. Mapped to region input features The multiple recovery expert branches are all deep neural network modules and share the same input features. However, each branch of restoration experts focuses on dealing with different types of degradation; Specifically, the multiple restoration expert branches include a denoising restoration expert branch, a deblurring restoration expert branch, a compression artifact removal restoration expert branch, a dehazing restoration expert branch, a rain / snow occlusion removal restoration expert branch, and a low-light enhancement expert branch. Each restoration expert branch can be composed of a convolutional residual network, an attention network, a Transformer restoration module, or a hybrid structure of convolution and attention, and outputs candidate restoration features with the same spatial size as the region input features or a reconstructable spatial size.

[0051] For the The image region, the first The output of each recovery expert branch can be represented as: in, Indicates the first The image region passes through the first Candidate recovery features obtained from each recovery expert branch Indicates the first A recovery expert branch built on deep neural networks, Indicates the first Region input features for each image region; Specifically, each restoration expert branch is pre-trained using synthetic degradation images corresponding to a single degradation type as training samples, and then weighted and fused by restoration routing weights during the joint training phase. The output of each restoration expert branch includes candidate restoration features and degradation residual features, and the restoration routing weights are generated based on the corresponding degradation components in the region degradation description vector. S504: Based on the recovery route weights corresponding to each image region, multiple candidate recovery features are weighted and fused to obtain the region recovery features of the corresponding image region; In one specific implementation, the first The region restoration features of each image region are obtained through the following methods: in, Indicates the first Region restoration features of an image region Indicates the first The image region for the first Recovery route weights for each recovery expert branch. This indicates the number of recovery expert branches. Using this weighted fusion method, if a region primarily suffers from noise degradation, the routing weight corresponding to the denoising recovery expert branch is higher; if a region primarily suffers from fuzzy degradation, the routing weight corresponding to the defuzzifying recovery expert branch is higher; if a region suffers from multiple types of degradation simultaneously, multiple recovery expert branches participate in the recovery of that region.

[0052] S505: Input the region restoration features corresponding to each image region into the image reconstruction module, and fuse them according to the spatial coordinates or region mask of each image region to obtain a preliminary restored image; Specifically, the image reconstruction module is used to map the region restoration features to the restored image content of the corresponding region, and to fuse the restoration results of each region into a complete image. For regular image block division, the region restoration results can be placed back to the corresponding positions in the original image according to the spatial coordinates of each image region; for overlapping sliding window division, the overlapping regions can be fused by weighted average; for irregular region division, the restoration results of different regions can be fused into a complete image according to the region mask. In one specific implementation, the preliminary image recovery can be represented as: in, This indicates that the image has been initially restored. Represents the image reconstruction function. This represents the region restoration features corresponding to each image region. Indicates the number of image regions; Through the above steps, the method can obtain regional-level recovery route weights based on the regional degradation constraint map, and use multiple deep learning recovery expert branches to perform targeted processing on different degradation types, thereby achieving adaptive recovery of complex mixed degradation images. The recovery expert branch constructed based on deep neural networks includes at least one of the following: denoising recovery expert branch, deblurring recovery expert branch, compression artifact removal recovery expert branch, defogging recovery expert branch, rain and snow occlusion removal recovery expert branch, and low-light enhancement expert branch; S6): Generate re-degradation parameters for the corresponding image region, and perform re-degradation simulation on the preliminary restored image based on the re-degradation parameters, and fuse the degradation simulation results to obtain a simulated degradation image; perform consistency verification between the simulated degradation image and the image to be restored to obtain the consistency error; specifically as follows: S601): Generate re-degradation parameters corresponding to each image region according to the region degradation description vector; Specifically, the noise intensity in the region degradation description vector can be used to generate noise injection parameters, the blur scale and blur direction can be used to generate blur kernel parameters, the compression artifact intensity can be used to generate compression quality parameters, the low illumination level can be used to generate brightness mapping parameters, the fog level can be used to generate fog simulation parameters, and the rain and snow occlusion probability can be used to generate occlusion simulation parameters. S602): Based on the re-degradation parameters, perform degradation simulation operations on the initially restored image. The degradation simulation operations include one or more of the following: noise injection, blur convolution, compression simulation, brightness mapping, fogging simulation, and rain / snow occlusion simulation. In a specific implementation, the re-degradation simulation process can be represented as follows: in, This represents a simulated degraded image. This indicates that the image has been initially restored. This represents the set of vectors describing regional degradation. Represents a re-degenerate simulation function; Furthermore, the re-degradation simulation function can be formed by combining multiple degradation simulation sub-functions in a preset order. If the preset execution order is noise injection, fuzzy convolution, compression simulation, brightness mapping, fogging simulation, and rain / snow occlusion simulation, then it can be expressed as: in, Represents the noise injection function. Generated from the noise intensity of each image region; Represents a fuzzy convolution function. It is generated by the blur scale and blur direction of each image region; This represents a compressed simulation function. Generated from the intensity of compression artifacts in each image region; Represents the brightness mapping function. Generated based on the low-light levels of each image region; Represents the atomization simulation function. Generated by the degree of fogging in each image region; This represents a function that simulates rain and snow shading. Generated based on the probability of rain and snow occlusion in each image region; The above It can be represented as a set of regional-level re-degradation parameters, or as a parameter map obtained by fusing regional-level re-degradation parameters.

[0053] Among them, the noise injection function can generate Gaussian noise or Poisson noise based on the noise intensity; the fuzzy convolution function can generate Gaussian blur kernel, motion blur kernel, or defocus blur kernel based on the fuzz scale and fuzz direction; the compression simulation function can generate JPEG compression or differentiable approximate JPEG compression results based on the compression artifact intensity; the brightness mapping function can perform gamma mapping or brightness scaling mapping based on the low illumination level; the fogging simulation function can perform atmospheric scattering simulation based on the fogging level; and the rain and snow occlusion simulation function can generate particle occlusion masks based on the rain and snow occlusion probability. The above degradation simulation methods are used to illustrate the correspondence between each degradation parameter and the degradation simulation operation. In actual implementation, other degradation models that can generate corresponding degradation effects based on the corresponding degradation parameters can also be used. The aforementioned degradation simulation sub-functions can be executed at the region level according to the re-degradation parameters of the corresponding image region, and the degradation simulation results of each region are fused according to the region mask or spatial location to obtain the simulated degradation image. If a certain image region does not contain the corresponding degradation type, the corresponding degradation sub-function can be set to an identity mapping, or the corresponding degradation intensity parameter can be set to 0, in order to maintain consistency with the description of performing one or more degradation simulation operations; S603): The degradation simulation results of each image region are fused to obtain a simulated degradation image; Specifically, for each image region, a local re-degradation operation is performed based on its corresponding region degradation description vector, and the local simulated degradation results are fused into a complete simulated degradation image according to the region's spatial location. If there is overlap between image regions, fusion is performed using a weighted average method. S604): Calculate the consistency error between the simulated degraded image and the image to be restored, wherein the consistency error includes pixel consistency error, structural consistency error, edge consistency error, frequency domain consistency error and perceptual feature consistency error; In a specific implementation, the consistency error can be expressed as: in, Indicates consistency error, representing simulated degraded image. Indicates the image to be recovered; Specifically, pixel consistency error Used to measure the difference in pixel values ​​between a simulated degraded image and the image to be restored, it can be calculated using L1 distance, L2 distance, or smoothed L1 distance; structural consistency error. Used to measure the differences between the two in local brightness, contrast, and structure distribution, it can be determined by 1- Obtain; Edge consistency error To measure the difference between two edge maps in terms of edges and contours, edge maps can be extracted first using the Sobel operator, Canny operator, or gradient operator, and then the L1 distance between the two edge maps can be calculated. Frequency domain consistency error To measure the difference in frequency distribution between two images, a Fourier transform can be performed on the two images to calculate the difference between their amplitude spectra or high-frequency components; perceptual feature consistency error. To measure the difference in high-level visual features between two images, two images can be input into a pre-trained visual feature extraction network or an image restoration feature extraction network, and the L1 or L2 distance between the corresponding feature maps can be calculated. The consistency error is not used alone as a criterion for judging the final restoration quality, but is considered in conjunction with the region degradation estimation. Uncertainty, structural changes before and after restoration, and regional restoration credibility are used together to control closed-loop updates, in order to reduce the pseudo-optimal results of degradation preservation caused by only pursuing re-degradation consistency.

[0054] S7) Based on the consistency error, the region degradation description vector, region degradation constraint map, and recovery route weights are updated in a closed loop. The recovery result is then regenerated based on the updated recovery route weights, and the final recovered image and region recovery confidence map are output. The specific process is as follows: S701): Calculate the local consistency error corresponding to each image region based on the consistency error. When the local consistency error of the target image region is greater than the preset error threshold, correct the region degradation description vector corresponding to the target image region. Specifically, the differences between the simulated degraded image and the image to be restored are mapped back to each image region to obtain the local consistency error of each region: in, Indicates the first Local consistency error of each image region Represents the corresponding number in the simulated degraded image. Image content of each region, Indicates the corresponding number in the image to be restored Image content of each region, This represents the function for calculating local consistency error. S702): Recalculate the edge weights between the target image region and other image regions based on the corrected region degradation description vector, and update the region degradation constraint graph; In one specific implementation, if the first Local consistency error in each image region Greater than the preset error threshold If the value is negative, it indicates that the recovery strategy for that region may be flawed. In this case, the degradation description vector for that region should be corrected. In one specific implementation, a local error vector is constructed based on the pixel errors, structural errors, edge errors, frequency domain errors, and perceptual feature errors of the simulated degraded image and the image to be restored within the target image region. Then, the local error vector Compared with the current region degradation description vector Input the degradation correction mapping function to obtain the degradation correction amount: in, This represents the degradation correction mapping function, which can be implemented by a multilayer perceptron, a lightweight convolutional network, or a pre-defined error mapping rule. It is used to determine the correction direction and magnitude for each degradation component. When the high-frequency energy difference between the simulated degraded image and the image to be restored in the target region exceeds a threshold, the noise intensity or artifact intensity component is increased; when the edge diffusion error exceeds a threshold, the blur scale component is increased; when the brightness mean error exceeds a threshold, the low-light intensity component is corrected; when the dark channel or transmittance error exceeds a threshold, the fogging component is corrected; when the local occlusion residual exhibits sparse highlight or stripe distribution, the rain / snow occlusion probability component is corrected. Subsequently, the region degradation description vector is updated according to the degradation correction amount: in, Indicates the preset correction step size. Indicates the first During the second closed-loop update Region degradation description vectors for each image region, This represents the degradation correction amount obtained based on the local consistency error. This represents the updated region degradation description vector. After updating, a range constraint is applied to the region degradation description vector to keep each degradation component within a preset value range. This constraint is lifted when the local consistency error of all regions is less than a preset error threshold, or when the number of closed-loop updates reaches a preset maximum. Stop closed-loop updates when the time comes; S703): Regenerate the recovery route weights based on the updated region degradation constraint map and adjust the recovery intensity of the corresponding image regions; Specifically, after the region degradation description vector changes, the degradation similarity, structural similarity, semantic consistency, and spatial correlation between the target image region and its neighboring regions are recalculated, thereby updating the edge weights in the region degradation constraint graph. If the updated edge weight is lower than a preset threshold, the edge is deleted; if the updated edge weight is higher than the preset threshold, the edge is strengthened. S704): Based on the adjusted recovery route weights and recovery strength, the recovery results are regenerated to obtain the final recovered image; In one specific implementation, the updated region degradation constraint graph is input again into the graph information propagation module and the recovery route generation module to obtain the recovery route weights again: in, This indicates the updated recovery route weight. This represents the graph-enhanced degradation features obtained based on the updated region degradation constraint map; S705): Generate a region restoration confidence map based on local consistency error, degradation estimation uncertainty, and structural changes before and after restoration; Specifically, the outputs of multiple recovery expert branches are re-fused based on the updated recovery route weights to obtain the final recovery image: in, This indicates the final restored image. Indicates the first Region restoration features after updating each image region; S706): When the confidence level of a certain image region in the region restoration confidence map is lower than the preset confidence threshold, reduce the texture generation intensity, edge enhancement intensity and detail compensation intensity of the image region, and retain more original observation information from the image to be restored in order to reduce false textures and over-restoration. In one specific implementation, the first The confidence score for each image region can be determined in the following way: ; in, Indicates the first The reliability of image region recovery This indicates local consistency error. This indicates the uncertainty in the degradation estimate. This indicates the amount of structural change before and after restoration. Indicates the degree of structural preservation. This represents the weighting coefficients. Mapping the confidence scores of all image regions back to the original image space yields the region reconstruction confidence map. In a specific implementation, for image regions with a confidence level below a preset threshold, a conservative restoration strategy is adopted. This involves reducing the intensity of high-frequency texture compensation, minimizing generative detail enhancement, and increasing the proportion of the original observed image in the output for that region. This prevents the model from generating unreliable details in uncertain regions and improves the confidence level of the final restoration result in scenarios such as monitoring, remote sensing, and industrial inspection. In a specific implementation, the conservative recovery strategy controls the fusion ratio between the recovery results and the original observation information by adjusting the regional recovery reliability. For the first... The output of the image region is as follows: in, Indicates the first The output result after confidence adjustment of each image region This represents the restoration result of the i-th image region. Indicates the first The original observation content of each image region in the image to be restored. Indicates the first The reliability of region recovery for each image region. When the value is low, the proportion of the restored result in the region in the final output is reduced, and the proportion of the original observation content is increased, thereby suppressing false textures, excessive edge enhancement and unreliable detail compensation in low confidence regions.

[0055] The above processes—image acquisition to be restored, region segmentation, degradation description vector extraction, regional degradation constraint graph construction, restoration route weight generation, re-degradation simulation and consistency verification, closed-loop update, and credibility graph output—together constitute an image restoration method based on regional degradation constraint graphs and closed-loop re-degradation verification.

[0056] Example 2 This invention also includes a computer device, the terminal device comprising a processor and a memory, the memory for storing a computer program, the computer program comprising program instructions, and the processor for executing the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., serving as the computing and control core of the terminal, suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions to achieve a corresponding method flow or corresponding function; the processor described in this embodiment can be used in the operation of an image restoration method based on a region degradation constraint map and closed-loop re-degradation verification, including the following steps: S1: Obtain the image to be restored, wherein the image to be restored is an image containing noise, blur, compression artifacts, low light, fog, or rain / snow occlusion; S2: Divide the image to be restored into regions to obtain multiple image regions, and extract the spatial location, local structure and semantic information of each image region; S3: Based on the multiple image regions, a degradation feature extraction is performed on each image region using a degradation perception model to obtain a corresponding region degradation description vector. The region degradation description vector is used to characterize the degradation type, degradation intensity, and degradation uncertainty in the image region. S4: Based on the region degradation description vector and the degradation similarity, spatial adjacency, structural similarity and semantic consistency between image regions, construct a region degradation constraint graph. The region degradation constraint graph is used to describe the degradation association and recovery constraint relationship between different image regions. S5: Generate restoration route weights for each image region based on the region degradation constraint map, obtain the weights of each restoration expert branch based on the graph enhancement degradation features through Softmax, and perform weighted fusion of multiple candidate restoration features to obtain a preliminary restored image; S6: Perform a re-degradation simulation on the initially restored image based on the region degradation description vector to obtain a simulated degradation image, and perform a consistency check between the simulated degradation image and the image to be restored to obtain the consistency error; S7: Based on the consistency error, perform closed-loop updates on the region degradation description vector, region degradation constraint map, and recovery route weights, and output the final recovery image and region recovery confidence map based on the updated recovery results.

[0057] Example 3: This embodiment provides a storage medium, specifically a computer-readable storage medium (Memory), which is a memory device in a terminal device used to store programs and data. It is understood that the computer-readable storage medium here can include both the built-in storage medium in the terminal device and extended storage media supported by the terminal device. The computer-readable storage medium provides storage space that stores the terminal's operating system. Furthermore, this storage space also stores one or more instructions suitable for loading and execution by a processor. These instructions can be one or more computer programs (including program code). It should be noted that the computer-readable storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device.

[0058] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements can be made without departing from the principle of the present invention, and these improvements should also be considered within the scope of protection of the present invention.

Claims

1. An image restoration method based on region degradation constraint maps and closed-loop re-degradation verification, characterized in that: include: S1): Obtain the image to be restored, which contains at least one degradation factor; S2): Divide the image to be restored into multiple initial image units, merge homogeneous regions to obtain multiple image regions, extract the spatial location, local structure and semantic information of each image region, establish the spatial adjacency relationship between image regions, and obtain a set of candidate adjacency edges for constructing the region degradation constraint graph; S3): Extract local degradation features from the image region using a degradation perception model to obtain the corresponding region degradation description vector; S4): Combining the spatial adjacency relationships, structural similarities, semantic consistency, and degradation similarities of each image region, a region degradation constraint map is constructed to characterize the region degradation association and recovery constraints; S5): Obtain graph augmentation degradation features and generate recovery route weights for each image region based on the graph augmentation degradation features; The image region content is then input into multiple restoration expert branches to obtain multiple candidate restoration features; Subsequently, multiple candidate recovery features are weighted and fused based on the recovery route weights to obtain a preliminary recovered image; S6): Generate the re-degradation parameters for the corresponding image region, and perform re-degradation simulation on the preliminary restored image based on the re-degradation parameters. Then, fuse the degradation simulation results to obtain the simulated degradation image. Perform consistency verification between the simulated degradation image and the image to be restored to obtain the consistency error. S7): Based on the consistency error, perform closed-loop updates on the region degradation description vector, region degradation constraint map, and recovery route weights, and regenerate the recovery results based on the updated recovery route weights, outputting the final recovery image and region recovery confidence map.

2. The image restoration method based on region degradation constraint map and closed-loop re-degradation verification according to claim 1, characterized in that: The specific process of obtaining the candidate adjacent edge set for constructing the region degradation constraint graph in S2) is as follows: S201): Calculate the structural consistency score between adjacent initial image units based on the features of the divided initial image units. When the structural consistency score is greater than or equal to the preset structural consistency threshold, merge the corresponding adjacent initial image units into the same image region to obtain multiple structurally consistent image regions. S202): Record the spatial location information of each image region in the image to be restored; S203): Extract the local structural and semantic information of each image region. S204): Based on the spatial adjacency relationship, center point distance, structural similarity and semantic consistency between image regions, establish candidate adjacency relationships between image regions to obtain a set of candidate adjacency edges for constructing a region degradation constraint graph.

3. The image restoration method based on region degradation constraint map and closed-loop re-degradation verification according to claim 1, characterized in that: The specific method for the region degradation description vector in S3) is as follows: S301): Input each image region into the degradation perception model and extract the local degradation features corresponding to that image region; S302): Local degradation features are input into multiple degradation prediction heads to obtain the region degradation description vector corresponding to each image region; S303): The region degradation description vector includes at least one of noise intensity, fuzz scale, fuzz direction, compression artifact intensity, low illumination level, fog level, probability of rain and snow occlusion, and degradation uncertainty, wherein the degradation uncertainty is output by the uncertainty prediction head in the degradation perception model, and the degradation uncertainty includes at least one of degradation parameter prediction variance, log variance, or confidence inverse quantization value.

4. The image restoration method based on region degradation constraint map and closed-loop re-degradation verification according to claim 2, characterized in that: The specific process for constructing a region degradation constraint graph to characterize region degradation associations and recovery constraints is as follows: S401): Each image region is treated as a node in the region degradation constraint graph, and the corresponding region degradation description vector, spatial location, local structural information and semantic information of the image region are used as node features; S402): Establish candidate edges based on the spatial adjacency relationships between different image regions; S403): Based on the degradation description vector similarity, texture continuity, edge direction consistency, brightness distribution similarity, and semantic category consistency between different image regions, the degradation similarity, structural similarity, and semantic consistency are quantized respectively, and the edge weight of the candidate edge is calculated accordingly. S404): Construct a region degradation constraint graph based on nodes, node features, candidate edges, and edge weights.

5. The image restoration method based on region degradation constraint map and closed-loop re-degradation verification according to claim 4, characterized in that: The region degradation constraint map is represented as follows: in, This represents a set of nodes, where each node corresponds to an image region. This represents a set of edges used to connect image regions that exhibit degenerate associations; Represents the adjacency weight matrix; No. The node and the first Edge weights between nodes Determined in the following ways: in, Represents the row normalization function. Representing the normalized degraded similarity between two image regions, based on the region degradation description vector. The cosine similarity between them is determined; They represent the first The image region and the first Region degradation description vectors for each image region; This represents the normalized spatial correlation between two image regions. They represent the first The image region and the first Spatial location information of each image region; This represents the normalized structural similarity between two image regions; They represent the first The image region and the first Local structural features of an image region; This indicates the normalized semantic consistency between two image regions. They represent the first The image region and the first Semantic information of each image region; Let represent the weight coefficients for degenerate similarity, spatial relevance, structural similarity, and semantic consistency, respectively, and satisfy the following: and The similarity function The output values ​​are normalized to the same range.

6. The image restoration method based on region degradation constraint map and closed-loop re-degradation verification according to claim 1, characterized in that: The specific process for obtaining the preliminary restoration map is as follows: S501): Input the node features in the region degradation constraint graph into the graph information propagation module to obtain the graph enhancement degradation features of each image region; S502): Input the image enhancement degradation features into the recovery route generation module to obtain the recovery route weights corresponding to each image region; S503: Input the image content of each image region in the image to be restored into the shared feature encoding module to obtain the corresponding region input features, and input the region input features into multiple restoration expert branches based on deep neural networks to obtain multiple candidate restoration features; S504: Based on the recovery route weights corresponding to each image region, multiple candidate recovery features are weighted and fused to obtain the region recovery features of the corresponding image region; S505: Input the region restoration features corresponding to each image region into the image reconstruction module, and fuse them according to the spatial coordinates or region mask of each image region to obtain a preliminary restored image.

7. The image restoration method based on region degradation constraint map and closed-loop re-degradation verification according to claim 6, characterized in that: The recovery expert branch constructed based on deep neural networks includes at least one of the following: denoising recovery expert branch, deblurring recovery expert branch, compression artifact removal recovery expert branch, defogging recovery expert branch, rain and snow occlusion removal recovery expert branch, and low-light enhancement expert branch; The recovery route weights are used to control the degree of participation of different recovery expert branches in the corresponding image regions, and the first... The recovery routing weights corresponding to each image region satisfy the following: in, Indicates the first The image region for the first Recovery route weights for each recovery expert branch. Indicates the number of expert branches for recovery; the first The region restoration features of each image region are obtained through the following methods: in, express Region restoration features of the image region Indicates the first Input features of each image region Indicates the first A recovery expert branch built on a deep neural network.

8. The image restoration method based on region degradation constraint map and closed-loop re-degradation verification according to claim 1, characterized in that: S7) Based on the consistency error, the region degradation description vector, region degradation constraint map, and recovery route weights are updated in a closed loop. The recovery result is then regenerated based on the updated recovery route weights, and the final recovered image and region recovery confidence map are output. The specific process is as follows: S701): Calculate the local consistency error corresponding to each image region based on the consistency error. When the local consistency error of the target image region is greater than the preset error threshold, correct the region degradation description vector corresponding to the target image region. S702): Recalculate the edge weights between the target image region and other image regions based on the corrected region degradation description vector, and update the region degradation constraint graph; S703): Regenerate the recovery route weights based on the updated region degradation constraint map and adjust the recovery intensity of the corresponding image regions; S704): Based on the adjusted recovery route weights and recovery strength, the recovery results are regenerated to obtain the final recovered image; S705): Generate a region restoration confidence map based on local consistency error, degradation estimation uncertainty, and structural changes before and after restoration; S706): When the confidence level of a certain image region in the region restoration confidence map is lower than the preset confidence threshold, reduce the texture generation intensity, edge enhancement intensity and detail compensation intensity of the image region, and retain more original observation information from the image to be restored, so as to reduce false textures and over-restoration.

9. A computer device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the image restoration method based on region degradation constraint map and closed-loop re-degradation verification as described in any one of claims 1 to 9.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements an image restoration method based on a region degradation constraint map and closed-loop re-degradation verification as described in any one of claims 1 to 9.

Images