Seamless image restoration method based on region-based parallel adaptive pixel consistency generation

Abstract

The present invention discloses a seamless image restoration method based on regional parallel adaptive pixel consistency generation, comprising: receiving an original image and an occlusion mask image, obtaining multi-scale image features through Gaussian pyramid decomposition and edge detector, and realizing adaptive region division and priority sorting in combination with a watershed algorithm; extracting structural features using a feature pyramid network and a non-local self-attention mechanism, and establishing a regional association map through multi-scale attention calculation; generating a scheduling table based on regional priorities and association relationships, and performing parallel region generation; calculating pixel continuity and structural consistency between adjacent regions, realizing multi-scale fusion, and generating a final restoration image. The present invention can effectively ensure the visual continuity and structural integrity of the restoration result, while improving processing efficiency.

Description

Technical Field

[0001] The invention belongs to the field of image restoration, and in particular to a seamless image restoration method based on region-based parallel adaptive pixel consistency generation. Background Art

[0002] Image restoration technology has broad application prospects and important research value in the fields of cultural relics restoration, film and television production, photo retouching, etc. With the rapid development of digital and intelligent technologies, the requirements for image restoration quality are also increasing, especially when dealing with scenes such as large-scale occlusion, complex textures and regular structures, it is necessary to take into account both restoration quality and processing efficiency. High-quality image restoration not only needs to maintain the visual continuity and structural integrity of the image, but also ensure the natural transition between the generated content and the surrounding environment, which puts higher requirements on the performance of the restoration algorithm.

[0003] Traditional image restoration methods mainly include diffusion-based methods, sample-based methods, and deep learning-based methods. Diffusion-based methods propagate boundary information by solving partial differential equations, but they often have difficulty in dealing with large occluded areas; sample-based methods perform restoration by searching and copying similar tiles, but they are not effective when dealing with non-repeating textures; deep learning-based methods use pre-trained models to generate missing content. Although this improves the restoration quality to a certain extent, it often has the problem of insufficient global consistency. In practical applications, these methods usually use fixed-size tile divisions or a single processing flow, which makes it difficult to adapt to the restoration needs of different types of images.

[0004] The current image restoration technology still faces some specific technical difficulties: First, in the region division stage, the existing methods often use fixed-size tiles or simple rectangular segmentation, which is difficult to adaptively divide according to the image content characteristics, resulting in obvious splicing marks at the restoration boundary; second, in the feature extraction process, most methods only focus on local features or single-scale feature representations, and cannot effectively capture the multi-scale structural information and long-distance semantic associations of the image; third, in terms of parallel processing, the existing methods lack effective regional dependency modeling and task scheduling mechanisms, making it difficult to balance processing efficiency and restoration quality; finally, in the result fusion stage, the traditional simple weighted average or alpha blending method is difficult to ensure pixel-level continuity and structural consistency, especially when processing images with complex textures and clear boundaries, which are prone to blur or distortion. These technical difficulties have seriously restricted the practical application of image restoration technology. Summary of the invention

[0005] The purpose of the invention is to provide a seamless image restoration method based on regional parallel adaptive pixel consistency generation, in order to solve at least one technical problem existing in the prior art.

[0006] The technical solution is a seamless image restoration method based on region-by-region parallel adaptive pixel consistency generation, comprising the following steps:

[0007] S1, receiving the original image and occlusion mask and performing Gaussian pyramid decomposition to obtain a multi-scale image set and a multi-scale mask set; processing the multi-scale image set by edge detection, calculating the boundary strength, and evaluating the regional importance in combination with the multi-scale mask set; based on the regional importance, using the watershed algorithm to generate an adaptive regional partitioning map, and calculating the regional priority sequence based on the boundary complexity;

[0008] S2. Based on the adaptive region partitioning map, the feature pyramid network is used to process the original image, extract multi-scale features, calculate feature correlation, obtain structural features through the non-local self-attention mechanism, and obtain the structural feature map;

[0009] S3, calculate the attention weight based on the multi-scale features and structural feature map, analyze the regional association strength, and generate the regional association map;

[0010] S4, formulate a region generation schedule according to the region priority sequence and the region association map, execute parallel region generation, and obtain a region generation result set;

[0011] S5. Calculate the pixel continuity and structural consistency between regions based on the region generation result set, adaptively fuse the generation results in the region generation result set, and output the final repaired image.

[0012] Beneficial effects: The present invention can accurately grasp the structural features of the image, ensure the reasonable decomposition of the restoration task, provide powerful feature expression capabilities, and realize the precise modeling of inter-regional dependencies; ensure efficient content generation, guarantee the restoration quality and processing efficiency, and effectively solve the problems of structural incoherence and texture incoordination faced by traditional methods when processing complex images, thereby realizing high-quality seamless image restoration. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 It is a flow chart of the present invention.

[0014] Figure 2 This is a flow chart of step S1 of the present invention.

[0015] Figure 3 This is a flow chart of step S2 of the present invention.

[0016] Figure 4 This is a flow chart of step S3 of the present invention.

[0017] Figure 5 This is a flow chart of step S4 of the present invention.

[0018] Figure 6This is a flow chart of step S5 of the present invention. DETAILED DESCRIPTION

[0019] like Figure 1 As shown, the present application proposes a seamless image restoration method based on region-by-region parallel adaptive pixel consistency generation, comprising the following steps:

[0020] S1. Receive the original image and the occlusion mask image, decompose them using Gaussian pyramid, and obtain a multi-scale image set and a multi-scale mask set; process the multi-scale image set using an edge detector, calculate the boundary strength, and obtain a multi-scale boundary atlas; calculate the region importance score based on the multi-scale boundary atlas and the multi-scale mask set; based on the region importance score, use the watershed algorithm to generate an adaptive region partition map; based on the adaptive region partition map and the multi-scale boundary atlas, calculate the region complexity score and obtain the region priority sequence;

[0021] S2. Based on the adaptive region partitioning map, the feature pyramid network is used to process the original image to obtain a multi-scale feature atlas; feature transformation and correlation calculation are performed on the multi-scale feature atlas to obtain a feature correlation matrix; based on the feature correlation matrix, the non-local self-attention mechanism is used to obtain a structural feature map;

[0022] S3. Based on the multi-scale feature atlas and the structural feature map, attention calculation is performed on the feature map of each scale to obtain a multi-scale attention weight set; based on the multi-scale attention weight set, the correlation strength between regions is calculated to generate a regional correlation map;

[0023] S4. Based on the regional priority sequence and the regional association graph, extract the regional priorities and association relationships, and generate a regional generation scheduling table; according to the regional generation scheduling table, perform parallel regional generation, and output a regional generation result set;

[0024] S5. Based on the region generation result set, the pixel continuity and structural consistency between adjacent regions are calculated to obtain a consistency score matrix; based on the consistency score matrix, the generation results in the region generation result set are adaptively fused to obtain the final repaired image.

[0025] like Figure 2 As shown, according to one aspect of the present application, step S1 is further:

[0026] S11, receiving an original image and an occlusion mask image, performing L-layer decomposition on the original image and the occlusion mask image respectively through Gaussian downsampling, and generating a multi-scale image set including L resolution levels and a corresponding multi-scale mask set; wherein L is a natural number greater than 1;

[0027] S12, based on the multi-scale image set, calculating the horizontal and vertical gradient values ​​of each scale image to obtain a horizontal gradient value and a vertical gradient value; substituting the horizontal gradient value and the vertical gradient value into a preset boundary strength calculation formula to obtain a boundary strength map of each scale; combining the boundary strength maps of all scales to form a multi-scale boundary atlas;

[0028] S13, based on the multi-scale boundary atlas and the multi-scale mask set, calculating the boundary strength value and the distance value to the nearest unobstructed area at each image position; weighted summing the boundary strength value and the distance value according to preset weights to obtain a region importance scoring matrix; inputting the region importance scoring matrix into the improved watershed algorithm to generate an adaptive region partition map;

[0029] S14. Based on the adaptive region partition map and the multi-scale boundary atlas, the boundary complexity value, area ratio value and content complexity value of each region are calculated; the boundary complexity value, area ratio value and content complexity value are weighted and summed according to preset weights to obtain the complexity score of each region; all regions are sorted according to the complexity score to generate a region priority sequence.

[0030] In one embodiment of the present application, the regional importance score calculation method is: regional importance score R(x, y) = α·B(x, y) + β·D(x, y) + γ·C(x, y); where B(x, y) = sqrt(Gx 2 (x,y) + Gy 2 (x, y)) is the boundary intensity value, Gx(x, y) and Gy(x, y) are the horizontal and vertical gradients respectively; D(x, y) = exp(-d(x, y) / σ) is the distance attenuation term, d(x, y) is the Euclidean distance to the nearest unoccluded area, and σ is the attenuation coefficient; C(x, y) = ∑w∈W max(0, B(w) - B(x, y)) is the local contrast, W is the local window centered at (x, y); α, β, γ are adaptive weight coefficients, and α+β+γ=1; x, y are image coordinate positions.

[0031] Improved watershed algorithm: region growing function W(p, q) = λ1·G(p, q) + λ2·E(p, q) + λ3·T(p, q); where G(p, q) = |I(p) - I(q)| is the grayscale gradient, I(p) and I(q) are the grayscale values ​​of pixels p and q; E(p, q) = exp(-||F(p) - F(q)|| 2 / σe) is the boundary response term, F(p) and F(q) are the corresponding boundary feature vectors, and σe is the scale parameter; T(p, q) = cos(θp - θq) is the texture consistency term, θp and θq are local directions; λ1, λ2, λ3 are adaptive weights and λ1 + λ2 + λ3 = 1; when W(p, q) is less than the dynamic threshold τ(t) = τ0·exp(-t / T), the region is merged, τ0 is the initial threshold, t is the number of iterations, and T is the temperature parameter.

[0032] This embodiment realizes intelligent segmentation and priority sorting of image restoration areas by introducing multi-scale analysis and adaptive region division methods. Specifically, the Gaussian pyramid decomposition is used to provide image information at different resolutions, so that the system can capture large-scale structures and detail features at the same time; by considering gradient strength and direction information at the same time, a more accurate boundary positioning capability is provided; the watershed algorithm combined with regional importance scoring realizes adaptive regional division, avoiding the limitations of the traditional fixed-size block method. This embodiment enables the system to more accurately identify and process regions with different complexities, and provides a scientific priority basis for subsequent parallel processing through the complexity scoring mechanism. Since multiple factors such as boundary strength, regional distance and local features are fully considered in the division process, the system can adaptively process different types of image content, and can provide reasonable processing strategies for images with complex textures, strong boundaries or large uniform areas. This embodiment improves the naturalness and continuity of the restoration results, and provides a reliable task allocation basis for subsequent parallel processing.

[0033] According to one aspect of the present application, step S11 is further:

[0034] S111, reading the original image and the occlusion mask image, performing brightness normalization processing on the original image to obtain a normalized image; performing binarization processing on the occlusion mask image to obtain a binary mask image; performing boundary filling on the normalized image and the binary mask image respectively to obtain a filled image and a filled mask;

[0035] S112, based on the image size of the original image and the preset minimum resolution, calculate the number of pyramid layers L, and construct a (2L-1)×(2L-1) Gaussian kernel matrix; normalize the Gaussian kernel matrix to obtain a standardized Gaussian kernel; multiply the standardized Gaussian kernel with different scale factors to obtain a multi-scale Gaussian kernel set;

[0036] S113, using the i-th layer kernel matrix in the multi-scale Gaussian kernel set to perform a convolution operation on the filled image to obtain a smoothed image; performing maximum pooling on the smoothed image according to a preset window to obtain a downsampled image; combining the downsampled images of all levels to form a multi-scale image set; wherein i is a preset value;

[0037] S114, using mean filtering to smooth the filling mask to obtain a smoothed mask; performing threshold segmentation on the smoothed mask to obtain a binary mask; performing maximum pooling on the binary mask according to a preset window to obtain downsampling masks of each level; combining the downsampling masks of all levels to form a multi-scale mask set.

[0038] In one embodiment of the present application, the multi-scale Gaussian kernel calculates H(x, y, σi) = K·exp(-(x 2 + y 2 ) / (2σi 2 ))·M(i); where σi = σ0·ri is the scale parameter of the i-th layer, σ0 is the basic scale, and r is the scale ratio; M(i)= 1 + α·log(L / i) is the scale adaptive modulation factor, L is the total number of pyramid layers, and α is the adjustment coefficient of the scale adaptive modulation factor; K is the normalization coefficient so that ∫∫H(x, y, σi)dxdy = 1; the number of layers L = └log2(min(W, H) / Rmin)┘, W and H are the image width and height, Rmin is the minimum resolution threshold, and └ ┘ is the rounding function.

[0039] This embodiment provides a rich image representation for subsequent processing by implementing multi-scale image decomposition. First, by preprocessing the original image and mask, including brightness normalization and boundary filling, the standardization and integrity of the data are ensured; the appropriate number of pyramid layers is determined based on the adaptive calculation of the image size and minimum resolution to avoid excessive loss of information. The design of the standardized Gaussian kernel and the construction of the multi-scale kernel set ensure the smoothness and stability of the downsampling process. Through the combination of Gaussian convolution and maximum pooling, the system achieves effective downsampling while maintaining important features; the smoothing of the mask and the threshold segmentation ensure the accurate transmission of the occluded area information. This embodiment is particularly suitable for processing images with different scale features, because it can maintain the key information of the image at different resolutions, and through scientific downsampling strategies and mask processing, it provides a reliable data basis for subsequent feature extraction and region division, and improves the robustness and adaptability of the entire restoration process.

[0040] According to one aspect of the present application, step S12 is further:

[0041] S121, calculating the horizontal direction gradient for each scale image in the multi-scale image set to obtain a horizontal gradient map; calculating the vertical direction gradient for each scale image in the multi-scale image set to obtain a vertical gradient map; combining the horizontal gradient map and the vertical gradient map to obtain a directional gradient set;

[0042] S122, based on the directional gradient set, calculating the square sum of the horizontal gradient map and the vertical gradient map to obtain a gradient amplitude map; based on the directional gradient set, calculating the inverse tangent of the horizontal gradient map and the vertical gradient map to obtain a gradient directional map; combining the gradient amplitude map and the gradient directional map to obtain a boundary feature set;

[0043] S123, processing the gradient amplitude map and the gradient direction map in the boundary feature set according to a preset boundary strength calculation formula to obtain a boundary strength map of each scale; performing non-maximum suppression on the boundary strength map to obtain an optimized boundary map;

[0044] S124. Based on the optimized boundary maps of all scales, normalize the boundary strength of each scale to obtain a normalized boundary atlas; integrate the normalized boundary atlases of all scales to obtain a multi-scale boundary atlas.

[0045] In one embodiment of the present application, the boundary strength calculation formula is: boundary response strength B(x, y) = β1·E(x, y) + β2·D(x, y) + β3·O(x, y); where E(x, y) = sqrt(Gx 2 + Gy 2 )·(1 + λ·|cos(2θ)|) is the enhanced edge response, θ is the gradient direction; D(x, y) = |Gxx·Gyy - Gxy 2 | is the second-order derivative response; O(x, y) = exp(-|θ - θref| 2 / σo 2 ) is the directional consistency term, θref is the reference direction; β1, β2, β3 are adaptive weights and ∑βi = 1; Gxx, Gyy, Gxy are second-order partial derivatives, Gx is the gradient of the image in the horizontal direction, Gy is the gradient of the image in the vertical direction, λ is the adjustment coefficient in the enhanced edge response, and σo is the standard deviation in the directional consistency term.

[0046] This embodiment achieves an accurate description of the image structure through a combination of boundary detection and multi-scale boundary feature extraction. The calculation of the directional gradient adopts horizontal and vertical bidirectional analysis, and provides a complete boundary feature description through the combined representation of gradient amplitude and direction; in the process of constructing the boundary feature set, the intensity and direction information of the boundary are captured through the collaborative analysis of the gradient amplitude map and the gradient direction map. The application of non-maximum suppression effectively suppresses redundant boundary responses and improves the accuracy of boundary positioning; in the process of generating the multi-scale boundary atlas, the consistency of boundary features of different scales is ensured through normalization processing. This embodiment is particularly suitable for processing images with complex boundary structures because it can accurately capture boundary features of different intensities and directions, and through multi-scale feature representation and optimization processing, it provides reliable boundary information for subsequent region division and improves the structural retention ability of the repair result.

[0047] According to one aspect of the present application, step S13 is further:

[0048] S131, read the multi-scale boundary atlas, calculate the gradient amplitude of each layer of the boundary map to obtain a gradient amplitude matrix, calculate the boundary continuity based on the gradient amplitude matrix to obtain a continuity matrix, perform weighted combination of the gradient amplitude matrix and the continuity matrix to obtain a boundary strength value matrix, and combine the boundary strength value matrices of all levels to form a multi-scale boundary strength set.

[0049] S132, reading a multi-scale mask set, using a fast marching algorithm to calculate the Euclidean distance field for each layer of mask image to obtain a distance field, performing anisotropic diffusion on the distance field to obtain a smoothed distance field, calculating a distance value to the nearest unobstructed area based on the smoothed distance field to obtain an unobstructed distance matrix, and combining the unobstructed distance matrices of all levels to form a multi-scale distance value set.

[0050] S133. Read the multi-scale boundary strength set and the multi-scale distance value set, construct an adaptive weight function to dynamically calculate the boundary strength weight and the distance value weight to obtain a weight matrix, perform weighted summation of the boundary strength value matrix and the unobstructed distance matrix according to the corresponding weights to obtain a regional importance matrix, and obtain a regional importance score matrix by cross-scale fusion of regional importance matrices of all levels.

[0051] S134, reading the regional importance score matrix, performing local extreme value detection on the regional importance score matrix to obtain a candidate seed point set, using an improved watershed algorithm to segment the regional importance score matrix to obtain an initial segmentation map, performing post-processing optimization on the initial segmentation map based on the regional area and boundary regularity, and generating an adaptive regional partition map.

[0052] In one embodiment of the present application, the fast marching algorithm is used to calculate the Euclidean distance field as follows: distance update D(p) = min(D(p), min(q∈N(p)){D(q) + w(p, q)}); where w(p, q) = sqrt((xp-xq) 2 + (yp-yq) 2 )·(1 + λ·|▽I(p) - ▽I(q)|) is the weighted distance; N(p) is the neighborhood of point p; ▽I is the image gradient; λ is the gradient weight coefficient; D(p) is the distance value of point p; xp is the horizontal coordinate of point p in the image, and yp is the vertical coordinate of point p in the image; the update sequence is performed according to the heap sort priority.

[0053] The specific anisotropic diffusion is: the diffusion equation ΞI / Ξt = div(c(||▽I||)·▽I) + S(I); where c(x) = exp(-x 2 / κ 2 ) is the diffusion coefficient, κ is the gradient threshold; S(I) = α·(I0 - I) is the source term, I0 is the original image; div is the divergence operator; diffusion time t∈[0, T], T is the maximum number of iterations, Ξ is the partial derivative, α is the source term adjustment coefficient in the anisotropic diffusion equation, and I is the pixel value of the current image.

[0054] This embodiment realizes intelligent segmentation of image restoration areas through regional importance scoring and adaptive regional division. The calculation of multi-scale boundary strength adopts a comprehensive analysis of gradient amplitude and boundary continuity to ensure the complete expression of boundary features; the application of the fast marching algorithm enables the system to efficiently calculate the distance field to the unobstructed area, and the introduction of anisotropic diffusion ensures the smoothness of the distance field. The design of the adaptive weight function realizes the dynamic balance between boundary strength and distance value. The regional importance scoring matrix obtained by cross-scale fusion fully reflects the restoration priority of each region; the improved watershed algorithm fully considers the continuity of local features when dividing the region, and ensures the regularity of the segmentation results through post-processing optimization. This embodiment is particularly suitable for processing images with complex structures and diverse contents, because it can adaptively adjust the segmentation strategy according to the local features of the image, and ensure the reasonable decomposition of the restoration task through multi-dimensional feature analysis and intelligent regional division, laying the foundation for subsequent parallel processing.

[0055] According to one aspect of the present application, step S14 is further:

[0056] S141, read the adaptive region partition map and the multi-scale boundary atlas, extract the boundary pixel sequence for each region to obtain the region boundary set, calculate the direction change rate of the boundary pixels to obtain the boundary direction change matrix, calculate the boundary complexity value of each region based on the boundary direction change matrix, and generate a boundary complexity table.

[0057] S142, read the adaptive region division map, count the total number of pixels in each region to obtain a region area table, calculate the ratio of the area of ​​each region to the area of ​​the entire occluded region to obtain an area ratio table, combine the region area table and the area ratio table to calculate the area ratio value of each region, and generate an area score table.

[0058] S143. Read the adaptive region division map and the multi-scale boundary atlas, extract the boundary features in each region to obtain a region feature table, analyze the texture complexity based on the region feature table to obtain a texture complexity value, calculate the degree of structural change to obtain a structural complexity value, combine the texture complexity value and the structural complexity value to obtain a content complexity value for each region, and generate a content complexity table.

[0059] S144. Read the boundary complexity table, area score table and content complexity table, normalize the three types of complexity values ​​to obtain a normalized complexity set, use preset weights to perform weighted summation on the values ​​in the normalized complexity set to obtain a regional complexity score table, sort the regions in descending order according to the complexity scores, and generate a regional priority sequence.

[0060] In one embodiment of the present application, the complexity score is: C(R) = ω1·Tc(R) + ω2·Sc(R) + ω3·Pc(R); where Tc(R) = ∑p∈R|▽ 2 I(p)| / |R| is the texture complexity, ▽ 2 is the Laplace operator, I(p) is the image pixel value at point p; Sc(R) = var({θi}) is the structural variation, θi is the set of main directions in region R; Pc(R) = 1 - exp(-α·Per(R) / sqrt(A(R))) is the shape complexity, Per(R) is the perimeter, A(R) is the area, α is the adjustment coefficient in the shape complexity term; ω1, ω2, ω3 are weights and ∑ωi = 1.

[0061] This embodiment achieves scientific determination of the region processing order through multi-dimensional complexity evaluation and priority sorting. The calculation of boundary complexity accurately reflects the complexity of the region outline by analyzing the directional change characteristics of boundary pixels; the introduction of area ratio value takes into account the impact of region size on the difficulty of restoration, ensuring the rational allocation of processing resources. The evaluation of texture and structural complexity adopts a comprehensive analysis of multi-scale boundary features to fully reflect the complexity of regional content; through the weighted combination of preset weights, the system achieves a balance among the three dimensions of boundary, area and content. This embodiment is particularly suitable for processing image restoration tasks with diverse content, because it can comprehensively consider multiple influencing factors, provide priority guidance for parallel processing through a scientific scoring mechanism, and improve the efficiency and quality of the restoration process.

[0062] like Figure 3 As shown, according to one aspect of the present application, step S2 is further:

[0063] S21. Based on the adaptive region partitioning map, the original image is input into the feature pyramid network, and features are extracted through convolution operation at each network layer; the extracted features are normalized to obtain a normalized feature map; the normalized feature maps of all levels are combined to form a multi-scale feature map set;

[0064] S22. Transform the feature vectors in the multi-scale feature map set to obtain a query feature vector group and a key-value feature vector group; calculate the dot product between the query feature vector group and the key-value feature vector group to obtain a feature correlation matrix; input the feature correlation matrix into a preconfigured non-local self-attention module for processing to generate a structural feature map.

[0065] This embodiment realizes multi-level feature extraction and global structural understanding of image content through feature pyramid network and non-local self-attention mechanism. The feature pyramid network captures the semantic information of the image at different scales through layer-by-layer convolution and feature normalization, forming a rich feature representation; this multi-scale feature extraction method enables the system to focus on the local details and global structure of the image at the same time. By querying the feature vector and transforming the key value, a correlation matrix between features is established, so that the system can capture the long-distance semantic association in the image. The introduction of the non-local self-attention mechanism further enhances the expressive power of the feature. By adaptively aggregating global information, the system can better understand and maintain the overall structure of the image. This embodiment is particularly suitable for processing images with complex textures and regular structures because it can effectively capture and maintain periodic patterns and geometric relationships in the image. Through the generation of structural feature maps, a reliable feature basis is provided for subsequent regional association analysis and content generation, which improves the structural consistency and semantic rationality of the restoration results.

[0066] According to one aspect of the present application, step S22 is further:

[0067] S221, based on the multi-scale feature atlas, applying linear mapping to the feature maps of each scale to obtain an initial transformation feature set; performing channel attention weighting on the initial transformation feature set to obtain a channel weighted feature set; performing spatial adaptive normalization on the channel weighted feature set to obtain a normalized feature set;

[0068] S222, based on the normalized feature set, respectively generate a query feature vector group, a key-value feature vector group, and a value feature vector group through at least three independent linear transformation layers; scale the query feature vector group and the key-value feature vector group to obtain a scaled query vector group and a scaled key-value vector group;

[0069] S223, calculating the dot product between the scaled query vector group and the scaled key value vector group to obtain a feature correlation matrix; applying an adaptive temperature coefficient to scale the feature correlation matrix to obtain a scaled correlation matrix; performing softmax normalization on the scaled correlation matrix to obtain a normalized feature correlation matrix;

[0070] S224. Perform matrix multiplication on the normalized feature correlation matrix and the value feature vector group to obtain a weighted feature matrix; apply the residual connection mechanism to fuse the weighted feature matrix with the normalized feature set to obtain a fused feature map; process the fused feature map through a non-local self-attention module to generate a structural feature map.

[0071] In one embodiment of the present application, the feature correlation matrix is: A(i, j) = softmax(Q(i)·K(j) T / sqrt(d) + P(i, j))·V(j); where Q(i), K(j), V(j) are query, key, and value feature vectors; P(i, j) =γ·exp(-||pi - pj|| 2 / σp 2 ) is the position coding term, pi, pj are feature positions; d is the feature dimension; γ is the position coding strength; σp is the position sensitivity parameter, T is transposed.

[0072] The adaptive temperature coefficient algorithm is specifically as follows: temperature coefficient T(s) = T0·exp(-k·H(s))·(1 + δ·V(s)); where H(s) = -∑pi·log(pi) is the characteristic entropy, pi is the normalized characteristic response; V(s) = var({ri}) / mean({ri}) is the characteristic variation coefficient, ri is the characteristic response intensity; T0 is the base temperature; k is the entropy sensitivity; δ is the variation parameter.

[0073] The non-local self-attention module is specifically: optimize the attention A'(i, j) = A(i, j)·(1 + λ·S(i, j))·M(i, j); where A(i, j) is the original attention score; S(i, j) = cos(f(i), f(j)) is the feature cosine similarity, f is the feature vector; M(i, j) = exp(-d(i, j) 2 / (2σm 2 )) is the spatial modulation term, d is the position distance; λ is the similarity weight; σm is the spatial scale parameter.

[0074] This embodiment realizes global correlation modeling of image features through feature transformation and non-local self-attention. The application of linear mapping and channel attention enhances the expressiveness of features, and spatial adaptive normalization ensures the stability of features; the design of query-key-value triples realizes effective correlation between features, and improves the accuracy of correlation calculation through scale scaling. The introduction of adaptive temperature coefficient optimizes the distribution of feature correlation, and the residual connection mechanism maintains the original feature information. This embodiment is particularly suitable for processing images with complex semantic associations because it can capture long-range feature dependencies and provide rich structural feature guidance through adaptive aggregation of global information, laying the foundation for generating high-quality restoration content.

[0075] like Figure 4 As shown, according to one aspect of the present application, step S3 is further:

[0076] S31. Based on the multi-scale feature graph set and the structural feature graph, the feature graph of each scale is converted into a query vector, a key-value vector and a value vector respectively; the normalized dot product of the query vector and the key-value vector is calculated to obtain an attention score; a softmax operation is performed on the attention score to obtain an attention weight; the attention weights of each scale are combined to form a multi-scale attention weight set;

[0077] S32. Based on the multi-scale attention weight set, the attention weights of each scale are weighted summed to obtain a regional association score; based on the regional association score, a connection relationship between regions is constructed to generate a regional association map.

[0078] This embodiment achieves accurate modeling and quantification of inter-regional dependencies through a multi-scale attention mechanism and regional association analysis. The calculation of multi-scale attention weights adopts a query-key mechanism, which accurately captures the similarity and degree of association between different regions by calculating the normalized dot product between feature vectors. The introduction of softmax normalization of attention weights enables the system to better handle long-distance dependencies. In the process of establishing the regional association map, the accuracy and reliability of the association strength are ensured by adaptively weighting the attention weights of each scale. This embodiment is particularly suitable for processing images with complex spatial dependencies because it can accurately quantify the degree of influence between different regions and provide a scientific basis for subsequent parallel scheduling. With the guidance of the regional association map, the system can better coordinate the generation process of different regions and ensure the semantic consistency and structural coherence of the generated content.

[0079] According to one aspect of the present application, step S31 is further:

[0080] S311, based on the multi-scale feature graph set, applying linear transformation to the feature graph of each scale to generate initial query features; performing linear transformation on the structural feature graph to obtain initial key-value features; reorganizing the initial query features and the initial key-value features by dimension to obtain a reorganized feature set;

[0081] S312, normalizing the features in the reorganized feature set by a scaling factor to obtain a normalized query vector and a normalized key value vector; calculating a matrix dot product of the normalized query vector and the normalized key value vector to obtain an attention score;

[0082] S313, performing softmax operation on the attention scores of each scale respectively to obtain the scale attention weight; performing spatial position encoding on the scale attention weight to obtain position weighted attention; combining the position weighted attention of each scale to form a multi-scale attention weight;

[0083] S314. Based on the multi-scale attention weights, the attention weights of each scale are adaptively weighted to obtain a weighted attention set; cross-scale consistency correction is performed on the weighted attention set to obtain a corrected attention weight; and the corrected attention weights of all scales are combined to form a multi-scale attention weight set.

[0084] In one embodiment of the present application, the multi-scale attention weight algorithm is: attention weight W(l,i,j) = softmax(S(l,i,j))·exp(-D(l,i,j)); where S(l,i,j) = (Q(l,i)·K(l,j) T) / sqrt(dl) +M(l,i,j) is the similarity score, dl is the feature dimension of the lth layer; D(l,i,j) = ||F(l,i) - F(l,j)|| 2 / σl 2 is the feature distance term; M(l,i,j) = μl·cos(θl,i - θl,j) is the direction modulation term; F(l,i) is the feature vector; θl,i is the feature direction; μl and σl are the hierarchical adaptation parameters, Q(l,i) is the query feature vector at the i-th position in the l-th layer; K(l,j) is the key feature vector at the j-th position in the l-th layer.

[0085] Cross-scale consistency correction algorithm: Correction weight C(l, i) = ∑m∈M ξ(l, m)·W(m, i)·U(l, m, i); where ξ(l, m) = softmax(-|lm| / τ) is the hierarchical correlation; W(m, i) is the original attention weight; U(l, m, i) = exp(-||F(l, i) - P(F(m, i))|| 2 / σu 2 ) is the feature consistency term; P is the feature projection function; τ is the temperature parameter; σu is the consistency scale, M is the set of levels in the cross-scale consistency correction algorithm, and F(m, i) is the feature vector of the i-th position in the m-th layer.

[0086] This embodiment achieves accurate quantification of feature associations through multi-scale attention calculations. The design of feature reorganization and dimensional normalization ensures the numerical stability of attention calculations, and the introduction of spatial position encoding enhances position perception capabilities; the adaptive weighting and cross-scale correction mechanism achieves effective fusion of attention information at different scales, ensuring the consistency of attention weights. This embodiment is particularly suitable for processing images with multi-level semantic associations because it can accurately capture feature dependencies at different scales, and through adaptive weight allocation, it provides a reliable quantitative basis for regional association analysis, thereby improving the semantic consistency of the restoration results.

[0087] like Figure 5 As shown, according to one aspect of the present application, step S4 is further:

[0088] S41, based on the regional priority sequence and the regional association map, analyzing the dependency relationship between regions and calculating the correlation between regions; combining regions with correlations lower than a preset threshold into parallel generation groups; sorting the parallel generation groups according to the priorities in the regional priority sequence, and generating a regional generation schedule;

[0089] S42. Generate a scheduling table based on regions, extract the features of the unoccluded regions around the regions of each parallel generation group as context information; input the context information into a pre-configured enhanced CAA module for content generation to obtain generated content; smooth the boundaries of the generated content to form a region generation result set.

[0090] In an embodiment of the present application, a region is associated with a scheduling mapping, specifically: Region association quantization: RelationScore(i, j) = α * AttentionWeight(i, j) + β * BorderContact(i, j) + γ * FeatureSimilarity(i, j); where AttentionWeight(i, j) is the attention weight, BorderContact(i, j) is the region boundary contact metric, and FeatureSimilarity(i, j) is the feature similarity; α, β, and γ are weight coefficients, and α + β + γ = 1.

[0091] Construct a dependency relationship: When RelationScore(i, j) > ThresholdHigh, regions i and j have a strong dependency relationship, and regions with a strong dependency relationship must be processed sequentially; when ThresholdLow ≤ RelationScore(i, j) ≤ ThresholdHigh, it is a weak dependency relationship, and regions with a weak dependency relationship can be processed in parallel, but additional consistency constraints are required; when RelationScore(i, j) < ThresholdLow, regions i and j can be processed completely in parallel.

[0092] The parallel group division algorithm is specifically as follows: Initialize the set of regions to be processed according to the region priority sequence, and create an empty list of parallel groups; The group formation is specifically as follows:

[0093] For each region i in priority_sequence:

[0094] if no_strong_dependency(i, current_group):

[0095] add_to_current_group(i);

[0096] else:

[0097] create_new_group();

[0098] add_to_new_group(i).

[0099] Process each region in turn according to the priority sequence. For the region currently being processed, first check whether there is a strong dependency between it and the existing regions in the current parallel group. If the region does not have a strong dependency with any region in the current parallel group, then add this region to the current parallel group. However, if it is found that the region has a strong dependency with any region in the current parallel group, then it is necessary to open a new parallel group and make this region the first member of the new parallel group. For a specific example: suppose there are regions A, B, C, and D, where A and B do not have a strong dependency; B and C have a strong dependency; C and D do not have a strong dependency. Then the processing process will be: create the first parallel group and add A; check that there is no strong dependency with A, add B to the current group; check that there is a strong dependency with B, create a new parallel group, add C to the new group; check that there is no strong dependency with C, add D to the group where C is. The resulting parallel groups will be: first group: {A, B}; second group: {C, D}. This grouping method ensures that regions with strong dependencies are not arranged in the same parallel group, thus avoiding problems that may be caused by data dependency. When the group size is less than the threshold and the dependency between groups is low, the small groups are merged; when the number of regions in a group exceeds the number of processor cores, the large group is split.

[0100] The parallel groups are sorted based on the weighted average of the priorities, a directed acyclic graph is constructed considering the dependencies between the groups, and the processing order of the groups is determined using topological sorting. For strong dependencies, they are processed strictly in priority order; for weak dependencies, limited out-of-order execution is allowed; for no dependencies, they are processed completely in parallel. The generation quality is monitored, the threshold is adjusted dynamically, and the system is rescheduled when an exception occurs; the load of each processing unit is monitored, and the size of the parallel group is adjusted dynamically.

[0101] This embodiment achieves efficient parallel content generation and boundary smoothing through the regional generation scheduling strategy and the enhanced Correspondence-Aware Attention (CAA) module. Based on the scheduling mechanism of regional priorities and association relationships, the system can intelligently organize parallel generation tasks to improve processing efficiency while ensuring generation quality; by analyzing the dependencies between regions, the sequential processing of strongly correlated regions and the parallel generation of weakly correlated regions are ensured. The introduction of the enhanced CAA module enables the system to better utilize contextual information for content generation and achieve more accurate content generation. Smoothing the boundaries of the generated content effectively solves the discontinuity problem at the junction of regions. This embodiment is particularly suitable for processing images with large occlusions, because it can improve processing efficiency while ensuring generation quality, and ensure the natural transition and visual continuity of the generated content through reasonable task scheduling and boundary processing.

[0102] According to one aspect of the present application, step S42 is further:

[0103] S421, read the region generation schedule table, for each parallel generation group, extract the range of K pixels extending outward from the boundary of each region to obtain an extended boundary map, extract the pixel values ​​of the unobstructed region on the basis of the extended boundary map to obtain a boundary pixel set, perform multi-scale feature extraction on the boundary pixel set to obtain context information, where K is a natural number greater than 1.

[0104] S422, read the context information, calculate the similarity between each feature point and its neighborhood features to obtain a feature similarity matrix, construct a feature propagation graph based on the feature similarity matrix to obtain feature propagation weights, and perform weighted aggregation on the context information according to the feature propagation weights to obtain enhanced context features.

[0105] S423, read the enhanced context features, input them into the enhanced CAA module, obtain an aligned feature map through adaptive feature alignment, generate the content of each region based on the aligned feature map and the original context information to obtain initial generated content, and apply structure-guided detail enhancement to the initial generated content to obtain enhanced generated content.

[0106] S424, read the enhanced generated content and the original context information, extract the boundary pixels between the generated area and the original area to obtain the boundary transition zone, apply adaptive weighted gradient domain fusion to the boundary transition zone to obtain a smooth boundary area, and combine all processed area contents to form an area generation result set.

[0107] In one embodiment of the present application, the enhanced CAA module algorithm: feature alignment function A(p, q) = ∑w∈Ωα(p, w)·F(q+τ(w))·G(w); where α(p, w) = softmax(φ(F(p)) T ψ(F(w))) is the attention weight, φ and ψ are feature transformation functions; τ(w) = argmax(S(w,q)) is the offset field, S is the similarity map; F(·) is the feature map; G(w) = exp(-||wp|| 2 / σg 2 ) is the spatial weight; Ω is the search window; p is the target position, q is the source position, w is a position in the search window Ω, F(p) is the feature vector at the target position p, and σg is the standard deviation in the spatial weight G(w).

[0108] Gradient domain fusion algorithm: boundary transition function T(x, y) = ρ1·P(x, y) + ρ2·G(x, y) + ρ3·V(x, y); where P(x, y) = exp(-d(x, y) / σp) is the position weight and d is the distance to the boundary; G(x, y) = cos(θg(x, y) - θr(x, y)) is the gradient direction consistency, θg and θr are the generated and reference gradient directions, respectively; V(x, y) = min(||▽Ig||, ||▽Ir||) / max(||▽Ig||, ||▽Ir||) is the gradient amplitude ratio; ρ1, ρ2, ρ3 are adaptive weights and ∑ρi = 1, σp is the standard deviation in the position weight P(x, y), ▽Ig is the gradient of the generated image, and ▽Ir is the gradient of the reference image.

[0109] This embodiment achieves high-quality regional content generation through context guidance and boundary smoothing. Extended boundary extraction and multi-scale feature analysis ensure the integrity of context information, feature similarity calculation and the introduction of propagation weights optimize the feature aggregation process; the enhanced CAA module improves the accuracy of generated content through adaptive feature alignment, and structure-guided detail enhancement ensures the realism of the generated results. This embodiment is particularly suitable for processing image restoration tasks that require precise boundary transitions, because it can achieve natural content transitions, ensure smooth connection of boundaries through gradient domain fusion, and effectively avoid visible splicing marks.

[0110] like Figure 6 As shown, according to one aspect of the present application, step S5 is further:

[0111] S51, based on the region generation result set, calculating the gradient continuity value and texture coherence value of the boundary of adjacent regions to obtain a pixel consistency score; based on the region generation result set, evaluating the degree of geometric constraint satisfaction and perspective relationship preservation between regions to obtain a structural consistency score; combining the pixel consistency score and the structural consistency score to form a consistency scoring matrix;

[0112] S52. Calculate the fusion weights according to the consistency score matrix and construct a Laplacian pyramid. Based on the Laplacian pyramid, perform multi-scale fusion on each region in the region generation result set to obtain a fusion result. Perform local detail enhancement processing on the fusion result to generate a final repaired image.

[0113] This embodiment achieves seamless fusion and detail enhancement of generated results through pixel consistency evaluation and multi-scale fusion. The system constructs a complete consistency scoring matrix by calculating the gradient continuity and texture coherence between adjacent regions; the evaluation based on geometric constraints and perspective relationships ensures the overall coordination of the structure. In the multi-scale fusion process of the Laplacian pyramid, the smooth transition of features of different scales is achieved through adaptive adjustment of fusion weights; the fusion of each level maintains the continuity of the boundaries and avoids visible stitching marks. This embodiment is particularly suitable for processing images with complex textures and fine structures, because it can well preserve and enhance local details while maintaining overall consistency, and ensures the high quality and visual naturalness of the final restoration results through multi-scale adaptive fusion and detail enhancement.

[0114] According to one aspect of the present application, step S52 is further:

[0115] S521, read the consistency score matrix and the region generation result set, calculate the fusion weight of each region based on the consistency score matrix, adaptively adjust the fusion weight based on the region boundary gradient information to obtain the adjusted fusion weight, input the region generation result set and the adjusted fusion weight into the Laplacian pyramid decomposer to generate a multi-scale decomposition result.

[0116] S522, reading the multi-scale decomposition results and adjusting the fusion weights, applying the adjusted fusion weights to each scale level to perform feature weighting to obtain a weighted feature map, and organizing the weighted feature maps of each level into a pyramid structure to obtain a weighted feature pyramid.

[0117] S523, read the weighted feature pyramid, start from the coarsest scale, perform upsampling and feature fusion layer by layer to obtain a fused feature map, maintain boundary continuity in each layer of fusion process to obtain a boundary continuous feature map, and combine the boundary continuous feature maps of each layer through residual connection to obtain a multi-scale fusion result.

[0118] S524, reading the multi-scale fusion result, extracting local structural features to obtain a structural feature map, constructing a detail enhancement filter based on the structural feature map to obtain an enhancement filter kernel, applying the enhancement filter kernel to the multi-scale fusion result to perform adaptive detail enhancement, and generating a final repaired image.

[0119] In one embodiment of the present application, the fusion weight is: W(x, y) = η1·C(x, y) + η2·E(x, y) +η3·T(x, y); wherein C(x, y) = exp(-∑l|Ll(x, y)| / σl) is inter-layer consistency, Ll is the Laplacian pyramid coefficient of the lth layer, and σl is the standard deviation in the inter-layer consistency; E(x, y) = 1 - |▽I(x, y)| / max(|▽I|) is edge smoothness, ▽I is the gradient of the image; T(x, y) = cos(∠(▽I(x, y), ▽R(x, y))) is gradient consistency, R is the reference image, ∠ is the angle between the two vectors, and ▽ is the gradient; η1, η2, η3 are adaptive weights and ∑ηi =1.

[0120] The Laplacian pyramid decomposer is specifically: optimize the objective function L(x) = ||G(x) - y|| 2 + μ·R(x); where G(x) = ∑i wiPi(x) is the pyramid reconstruction term, Pi is the i-th layer pyramid operator, and wi is the layer weight; R(x) = ∑j βj·||▽jx|| is the regularization term, and ▽j is the j-order difference operator; y is the target image; x is the variable in the optimization objective function; μ is the regularization parameter; weight update wi = wi·exp(τ·(E(i-1) - Ei)), Ei is the i-th layer reconstruction error, and τ is the adjustment parameter in the weight update; βj = exp(-j 2 / σβ 2 ) is the order attenuation coefficient, and σβ is the standard deviation of the order attenuation coefficient.

[0121] The detail enhancement filter is: H(x, y) = B(x, y) + λ·G(x, y)·D(x, y); where B(x, y) is the base frequency band; G(x, y) = 1 + γ·|L(x, y)| / max(|L|) is the gain function, L is the Laplacian operator response; D(x, y) = σn 2 / (σn 2 + σl 2 (x, y)) is the detail modulation term, σn 2 is the noise variance, σl 2 is the local variance; λ is the enhancement strength, and γ is the contrast parameter.

[0122] This embodiment achieves seamless integration of restoration results through multi-scale feature fusion and detail enhancement. The calculation of adaptive fusion weights takes into account the regional boundary gradient information to ensure the smoothness of the fusion process; the multi-level fusion of the Laplacian pyramid realizes the effective combination of features of different scales, and the residual connection mechanism maintains the integrity of the detail information. The extraction of local structural features and the application of enhancement filters further improve the restoration quality. This embodiment is particularly suitable for processing images that require high-quality details, because it can well preserve and enhance local details while maintaining overall consistency, and through multi-scale adaptive fusion, it ensures the visual naturalness and detail realism of the final restoration result.

[0123] In another embodiment of the present application, assume that there is a picture of length x, in which half of the right area (i.e., x / 2 to x) is masked, and half of the left area (i.e., 0 to x / 2) is known image content. In the image restoration process, the CAA (Correspondence-Aware Attention) module is first used to generate the middle area, i.e., the part from x / 4 to 3x / 4. Since the area from x / 4 to x / 2 is already known image information, the CAA module can effectively use the pixel consistency of the known area to generate the area from x / 2 to 3x / 4, ensuring that the generated area from x / 2 to 3x / 4 is highly consistent with the area from x / 4 to x / 2 on the left at the pixel level, thereby generating a natural connection and good quality image content. When the middle area (x / 4 to 3x / 4) is generated (the part from x / 4 to x / 2 is used as a known area, and the part from x / 2 to 3x / 4 is used as a high-quality generated area). At this time, the known area is extended to x / 4 to 3x / 4, and the CAA module is continued to be used to generate the remaining right area (x / 2 to x). In this process, the CAA module uses the already generated high-quality area from x / 2 to 3x / 4 to generate the area from 3x / 4 to x, and finally ensures that the generated area from x / 2 to x is consistent with it at the pixel level, achieving seamless connection of the entire image. Through this phased generation strategy, the CAA module ensures that the generated area is aligned with the pixels of the surrounding known or generated areas at each step, so that all areas in the entire image restoration process can be naturally connected, avoiding problems such as discontinuous boundaries or inconsistent quality.

[0124] In order to effectively deal with the repair of large occluded areas, this embodiment adopts a phased parallel generation strategy. First, the middle area (x / 4 to 3x / 4) is generated to ensure the pixel consistency of this part with the known area. Then, after the generation of the middle area is completed, continue to generate the x / 2 to x area on the right. This phased parallel generation method reduces the computational overhead and boundary inconsistency problems caused by one-time generation in traditional large-area repair. When each area is generated, the information of the known area is obtained through the CAA module to ensure the seamless connection of the generation results. The parallel generation strategy effectively improves the generation efficiency while ensuring pixel consistency. By repairing multiple sub-areas at the same time, the repair of large occluded areas can be completed quickly. In addition, the CAA module plays a key role in each generation stage. It ensures that the generated area can be perfectly integrated with the known area in terms of visual effects through the sharing and alignment of pixel information.

[0125] This embodiment applies the CAA module to the image inpainting task, which solves the problem of pixel inconsistency in traditional methods when dealing with large-area occlusion repair. Through the introduction of the CAA module, the generated area can be aligned with the surrounding known areas at the pixel level, so that the generated image can maintain a high degree of coherence and consistency in both local details and overall transitions. The CAA module achieves adaptive pixel consistency in the repair process, which has a significant effect on solving problems such as blurred boundaries and unnatural transitions. The CAA module not only improves the visual quality of image repair, but also enables each generated area to maintain seamless connection when interacting with the surrounding areas. This embodiment shows great application potential in the fields of large-area image repair, multi-view generation, and panoramic image generation.

[0126] The present invention realizes high-quality seamless image restoration by generating pixel consistency in parallel in different regions. The five key steps of region division, feature extraction, correlation analysis, parallel generation and result fusion are organically combined to form a complete technical system. Through multi-scale analysis of Gaussian pyramid and edge detection, the system can accurately grasp the structural characteristics of the image; the adaptive region division based on the watershed algorithm ensures the reasonable decomposition of the restoration task. The combination of feature pyramid network and non-local self-attention mechanism provides powerful feature expression ability, and the calculation of multi-scale attention weights realizes the accurate modeling of inter-regional dependencies. The parallel scheduling strategy and enhanced CAA module ensure efficient content generation, while the multi-scale fusion based on Laplacian pyramid ensures the seamless connection of the results. The present invention is particularly suitable for processing image restoration tasks of large-scale and complex scenes, because it can simultaneously ensure the restoration quality and processing efficiency. Through multi-level feature analysis and adaptive processing strategies, it effectively solves the problems of structural incoherence and texture incoordination faced by traditional methods when processing complex images.

[0127] The present invention can be widely used in image editing software, especially when repairing damaged or missing images, to help repair defects in the image; the regional parallel generation technology can handle large-area image damage and ensure that the repaired area is seamlessly connected with the original image. In film and television post-production and virtual reality content generation, it is often necessary to fill blank areas in images or videos. Through adaptive pixel consistency repair, a complete image can be generated in a virtual scene to make the visual experience smoother. For example, when shooting a scene from multiple perspectives, this technology can effectively repair or expand the edge area of ​​the picture. Unmanned driving systems rely on cameras and sensors to obtain environmental information, but due to various reasons (such as occlusion, shadows, etc.), the image may be partially missing. Through image repair technology, these missing areas can be completed in the perception system, so as to understand the environment more comprehensively and improve the safety of autonomous driving. The present invention can also be understood as an extension of the image translation perspective, that is, by generating partial images to fill in information outside the perspective. This is very important for generating continuous scenes from different perspectives in virtual scenes. For example, in VR and 3D modeling, image repair can be used to generate content from other perspectives from a known perspective, expanding the visual range of the scene.

[0128] The preferred embodiments of the present invention are described in detail above; however, the present invention is not limited to the specific details in the above embodiments. Within the technical concept of the present invention, various equivalent transformations can be made to the technical solutions of the present invention, and these equivalent transformations all belong to the protection scope of the present invention.

Claims

1. A seamless image restoration method based on region-by-region parallel adaptive pixel consistency generation, characterized in that: The steps include: S1, receiving the original image and the occlusion mask and performing Gaussian pyramid decomposition to obtain a multi-scale image set and a multi-scale mask set; processing the multi-scale image set by edge detection, calculating the boundary strength, and evaluating the region importance in combination with the multi-scale mask set; Based on the regional importance, the watershed algorithm is used to generate an adaptive regional partition map, and the regional priority sequence is calculated based on the boundary complexity; S2. Based on the adaptive region partitioning map, the feature pyramid network is used to process the original image, extract multi-scale features, calculate feature correlation, obtain structural features through the non-local self-attention mechanism, and obtain the structural feature map; S3, calculate the attention weight based on the multi-scale features and structural feature map, analyze the regional association strength, and generate the regional association map; S4, formulate a region generation schedule according to the region priority sequence and the region association map, execute parallel region generation, and obtain a region generation result set; S5, calculating the pixel continuity and structural consistency between regions based on the region generation result set, adaptively fusing the generation results in the region generation result set, and outputting the final repaired image; Step S1 is further as follows: S11, receiving an original image and an occlusion mask image, performing L-layer decomposition on the original image and the occlusion mask image respectively through Gaussian downsampling, and generating a multi-scale image set including L resolution levels and a corresponding multi-scale mask set; wherein L is a natural number greater than 1; S12, based on the multi-scale image set, calculating the horizontal and vertical gradient values ​​of each scale image to obtain a horizontal gradient value and a vertical gradient value; substituting the horizontal gradient value and the vertical gradient value into a preset boundary strength calculation formula to obtain a boundary strength map of each scale; combining the boundary strength maps of all scales to form a multi-scale boundary atlas; S13, based on the multi-scale boundary atlas and the multi-scale mask set, calculating the boundary strength value of each image position and the distance value to the nearest unoccluded area; The boundary strength value and the distance value are weighted and summed according to the preset weights to obtain the regional importance scoring matrix; The regional importance score matrix is ​​input into the improved watershed algorithm to generate an adaptive regional partition map; S14. Based on the adaptive region partition map and the multi-scale boundary atlas, the boundary complexity value, area ratio value and content complexity value of each region are calculated; the boundary complexity value, area ratio value and content complexity value are weighted and summed according to preset weights to obtain a complexity score for each region; all regions are sorted according to the complexity scores to generate a region priority sequence; Step S2 is further as follows: S21. Based on the adaptive region partitioning map, the original image is input into the feature pyramid network, and features are extracted through convolution operation at each network layer; the extracted features are normalized to obtain a normalized feature map; the normalized feature maps of all levels are combined to form a multi-scale feature map set; S22, transforming the feature vectors in the multi-scale feature graph to obtain a query feature vector group and a key-value feature vector group; Calculate the dot product between the query feature vector group and the key-value feature vector group to obtain a feature correlation matrix; The feature correlation matrix is ​​input into the pre-configured non-local self-attention module for processing to generate a structural feature map; Step S3 is further as follows: S31, based on the multi-scale feature atlas and the structural feature map, converting the feature map of each scale into a query vector, a key-value vector and a value vector respectively; Calculate the normalized dot product of the query vector and the key-value vector to get the attention score; Perform softmax operation on the attention scores to obtain the attention weights; combine the attention weights of each scale to form a multi-scale attention weight set; S32, based on the multi-scale attention weight set, weighted summing the attention weights of each scale to obtain a regional association score; Based on the regional association scores, the connection relationship between regions is constructed to generate a regional association graph; Step S4 is further as follows: S41, based on the regional priority sequence and the regional association map, analyzing the dependency relationship between regions and calculating the correlation between regions; combining regions with correlations lower than a preset threshold into parallel generation groups; sorting the parallel generation groups according to the priorities in the regional priority sequence, and generating a regional generation schedule; S42, based on the region generation schedule table, extracting features of surrounding unobstructed regions for each region of the parallel generation group as context information; Input the context information into the preconfigured enhanced CAA module to generate content, thereby obtaining generated content; smooth the boundaries of the generated content to form a region generation result set; Step S5 is further as follows: S51, based on the region generation result set, calculating the gradient continuity value and texture coherence value of the boundary of adjacent regions to obtain a pixel consistency score; based on the region generation result set, evaluating the degree of geometric constraint satisfaction and perspective relationship preservation between regions to obtain a structural consistency score; combining the pixel consistency score and the structural consistency score to form a consistency scoring matrix; S52. Calculate the fusion weights according to the consistency score matrix and construct a Laplacian pyramid. Based on the Laplacian pyramid, perform multi-scale fusion on each region in the region generation result set to obtain a fusion result. Perform local detail enhancement processing on the fusion result to generate a final repaired image.

2. The seamless image restoration method based on region-by-region parallel adaptive pixel consistency generation according to claim 1, characterized in that: Step S11 is further as follows: S111, reading the original image and the occlusion mask image, performing brightness normalization processing on the original image to obtain a normalized image; performing binarization processing on the occlusion mask image to obtain a binary mask image; performing boundary filling on the normalized image and the binary mask image respectively to obtain a filled image and a filled mask; S112, based on the image size of the original image and the preset minimum resolution, calculate the number of pyramid layers L, and construct a (2L-1)×(2L-1) Gaussian kernel matrix; normalize the Gaussian kernel matrix to obtain a standardized Gaussian kernel; multiply the standardized Gaussian kernel with different scale factors to obtain a multi-scale Gaussian kernel set; S113, using the i-th layer kernel matrix in the multi-scale Gaussian kernel set to perform a convolution operation on the filled image to obtain a smoothed image; performing maximum pooling on the smoothed image according to a preset window to obtain a downsampled image; combining the downsampled images of all levels to form a multi-scale image set; Where i is the preset value; S114, smoothing the filling mask by using mean filtering to obtain a smoothed mask; performing threshold segmentation on the smoothed mask to obtain a binary mask; Perform maximum pooling on the binary mask according to the preset window to obtain the downsampling masks at each level; The downsampling masks of all levels are combined to form a multi-scale mask set.

3. The seamless image restoration method based on region-by-region parallel adaptive pixel consistency generation according to claim 1, characterized in that: Step S12 is further as follows: S121, calculating the horizontal direction gradient for each scale image in the multi-scale image set to obtain a horizontal gradient map; calculating the vertical direction gradient for each scale image in the multi-scale image set to obtain a vertical gradient map; combining the horizontal gradient map and the vertical gradient map to obtain a directional gradient set; S122, based on the directional gradient set, calculating the square sum of the horizontal gradient map and the vertical gradient map to obtain a gradient amplitude map; based on the directional gradient set, calculating the inverse tangent of the horizontal gradient map and the vertical gradient map to obtain a gradient directional map; combining the gradient amplitude map and the gradient directional map to obtain a boundary feature set; S123, processing the gradient amplitude map and the gradient direction map in the boundary feature set according to a preset boundary strength calculation formula to obtain a boundary strength map of each scale; performing non-maximum suppression on the boundary strength map to obtain an optimized boundary map; S124. Based on the optimized boundary maps of all scales, normalize the boundary strength of each scale to obtain a normalized boundary atlas; integrate the normalized boundary atlases of all scales to obtain a multi-scale boundary atlas.

4. The seamless image restoration method based on region-by-region parallel adaptive pixel consistency generation according to claim 1, characterized in that: Step S22 is further as follows: S221, based on the multi-scale feature atlas, applying linear mapping to the feature maps of each scale to obtain an initial transformation feature set; performing channel attention weighting on the initial transformation feature set to obtain a channel weighted feature set; Perform spatial adaptive normalization on the channel weighted feature set to obtain a normalized feature set; S222, based on the normalized feature set, respectively generate a query feature vector group, a key-value feature vector group, and a value feature vector group through at least three independent linear transformation layers; scale the query feature vector group and the key-value feature vector group to obtain a scaled query vector group and a scaled key-value vector group; S223, calculating the dot product between the scaled query vector group and the scaled key value vector group to obtain a feature correlation matrix; Applying adaptive temperature coefficients to scale the feature correlation matrix to obtain a scaled correlation matrix; Perform softmax normalization on the scaled correlation matrix to obtain a normalized feature correlation matrix; S224, performing matrix multiplication on the normalized feature correlation matrix and the value feature vector group to obtain a weighted feature matrix; The weighted feature matrix is ​​fused with the normalized feature set using the residual connection mechanism to obtain a fused feature map. The fused feature map is processed through a non-local self-attention module to generate a structural feature map.

5. The seamless image restoration method based on region-by-region parallel adaptive pixel consistency generation according to claim 1, characterized in that: Step S31 is further as follows: S311, based on the multi-scale feature atlas, applying a linear transformation to the feature map of each scale to generate an initial query feature; Performing linear transformation on the structural feature graph to obtain initial key-value features; reorganizing the initial query features and the initial key-value features by dimension to obtain a reorganized feature set; S312, normalizing the features in the reorganized feature set by a scaling factor to obtain a normalized query vector and a normalized key value vector; calculating a matrix dot product of the normalized query vector and the normalized key value vector to obtain an attention score; S313, performing softmax operation on the attention scores of each scale respectively to obtain the scale attention weight; Encode the spatial position of the scale attention weights to obtain position-weighted attention; combine the position-weighted attention of each scale to form multi-scale attention weights; S314. Based on the multi-scale attention weights, the attention weights of each scale are adaptively weighted to obtain a weighted attention set; cross-scale consistency correction is performed on the weighted attention set to obtain a corrected attention weight; and the corrected attention weights of all scales are combined to form a multi-scale attention weight set.

Images