Integrated image restoration method and system based on image mask modeling

Abstract

The invention relates to the technical field of image processing, in particular to an integrated image restoration method and system based on image mask modeling, and the method comprises the following steps: collecting the boundary data of the whole image, separating the boundary of a mask region from the boundary of a non-mask region, and analyzing the gradient change characteristics of the boundary of the mask region. And extracting local depth information of the mask region from the gradient change features, and constructing a mask region boundary residual distribution model. According to the method, the mask region and the non-mask region are separated, the boundary gradient features are extracted, the residual distribution model is established on the basis of gradient change, the subtle change of the boundary of the mask region is accurately captured, and the accuracy of the boundary features is improved. By constructing a depth information data set and combining local depth information with gradient features, adjustment of layered dynamic weight distribution of a mask area basic structure is realized, effective extension of area features to the inside is ensured, and the possibility of boundary information loss is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an integrated image restoration method and system based on image mask modeling. Background Art

[0002] The field of image processing technology mainly studies the acquisition, analysis, processing and improvement methods of digital images, including image denoising, enhancement, compression, segmentation and feature extraction. Its goal is to process images through algorithms and computer technology to meet specific application requirements or provide support for subsequent data analysis and pattern recognition.

[0003] Image restoration methods are an important branch of image processing, which aims to repair damaged or missing parts of an image and restore it to a state as close to the original image as possible. This technology is mainly used to eliminate noise, scratches or missing areas in an image. Common uses include digital restoration of old photos, film and television post-production, art restoration, ancient painting restoration, and image restoration in medical imaging and forensic analysis.

[0004] The existing technology in image restoration has a relatively simple treatment of the boundary between the masked area and the non-masked area. It usually relies on the global restoration strategy to directly act on the missing area, which easily leads to the loss of boundary details. The restored boundary part often cannot achieve a natural transition with the non-masked area. In the global restoration process, the existing technology lacks specificity in the extraction and application of contextual features. It only relies on simple feature statistics or fixed algorithm inference, which makes it difficult to capture the precise dependency relationship between the masked area and the context, and easily leads to context mismatch problems. In addition, in the restoration of the internal masked area, the existing technology lacks a dynamic adjustment mechanism for the internal pixel features, which makes the restoration effect prone to unevenness or abrupt transitions. For example, the restoration method based on traditional algorithms often shows blurred boundaries or discontinuous internal textures on complex texture images. Summary of the invention

[0005] The purpose of the present invention is to solve the shortcomings of the prior art and to propose an integrated image restoration method and system based on image mask modeling.

[0006] In order to achieve the above object, the present invention adopts the following technical solution: an integrated image restoration method based on image mask modeling, comprising the following steps: S1: Collect the boundary data of the entire image, separate the boundary between the masked area and the non-masked area, analyze the gradient change characteristics of the boundary of the masked area, extract the local depth information of the masked area from the gradient change characteristics, and construct a residual distribution model of the boundary of the masked area; S2: Based on the data in the residual distribution model of the mask region boundary, the basic features of the mask region are analyzed according to the gradient direction and depth information of the mask region boundary, and the basic feature distribution is applied to the inside of the mask region to generate a basic structural feature map of the mask region; S3: Based on the data in the basic structural feature map of the masked area, context features of the masked area and the non-masked area of ​​the whole image are obtained, the global dependency relationship between the masked area and the surrounding context features is analyzed, and the preliminary restoration features of the masked area are adjusted according to the global dependency relationship to generate a global distribution restoration result; S4: Based on the global distribution restoration result, extract the pixel distribution information inside the mask area, perform local balanced adjustment on the pixel distribution inside the mask area, obtain the coordinates of the fusion transition zone at the boundary of the mask area, perform interpolation adjustment on the pixel values ​​of the fusion transition zone, and generate the mask area fusion optimization result.

[0007] As a further solution of the present invention, the step of analyzing the gradient change characteristics of the mask region boundary is specifically: S111: Collect the boundary data of the entire image, separate the boundary between the masked area and the non-masked area, and use the formula: ; Calculate the gradient magnitude of the boundary pixels of the mask area ; in, is the grayscale change rate of the pixel in the horizontal x direction, is the grayscale change rate of the pixel in the vertical y direction, is the gray value matrix of the input image; S112: According to the gradient amplitude of the pixel points at the boundary of the mask area, the formula is adopted: ; Calculate boundary pixels The gradient residual value of ; in, is the boundary pixel of the mask area The gradient amplitude of is the boundary pixel The average value of the gradient magnitude of adjacent non-masked area pixels; S113: Based on the gradient residual values, the gradient residual values ​​of the mask boundary points are sorted according to coordinates, and a gradient residual distribution feature recognition result is generated by statistically analyzing the central tendency and discrete tendency of the residual values ​​of all boundary points.

[0008] As a further solution of the present invention, the step of acquiring the residual distribution model of the mask region boundary is specifically as follows: S121: Based on the gradient residual distribution feature recognition result, the residual value is matched with the gradient direction, and the corresponding relationship between the gradient change amplitude and the preset depth information is combined to obtain the local depth corresponding to each boundary point to obtain a depth information data set; S122: Based on the depth information data set, local depth information and gradient change features are integrated to construct a mask region boundary residual distribution model.

[0009] As a further solution of the present invention, the step of acquiring the basic structural feature map of the mask area is specifically as follows: S211: Based on the data in the residual distribution model of the mask region boundary, the gradient value and texture characteristics in the neighborhood range of the mask region boundary are analyzed, the weights are set in layers according to the gradient size and the weights are adjusted in combination with the texture characteristics to generate a dynamic weight distribution list of the neighborhood pixels; S212: Based on the dynamic weight distribution list of the neighborhood pixels, the basic features of the boundary pixels of the mask area are analyzed, the gradient direction and depth value are gradually expanded into the mask area, and the feature distribution of the internal pixels is adjusted according to the neighborhood distance to generate a basic structural feature map of the mask area.

[0010] As a further solution of the present invention, the step of obtaining the global distribution repair result is specifically: S311: Based on the data in the basic structural feature map of the mask area, the gradient direction, depth value and texture characteristics of the boundary of the mask area are extracted, and the context features of the non-masked area of ​​the whole image are partitioned and counted, and the gradient direction, depth value and texture characteristic distribution range are recorded, and the context features are classified to generate feature data of the mask area and the context area; S312: Based on the feature data of the mask area and the context area, the formula is used: ; Calculate in context features Mask region features under given conditions The conditional probability , get the global dependency information; in, Indicates the features in the mask area In the presence of contextual features The probability of occurrence, Represents the mask region features The prior probability of Representing contextual features The total probability of S313: According to the global dependency information, the preliminary restoration features of the mask area are adjusted, and a global distribution restoration result is generated by gradually optimizing the gradient direction, depth value and texture characteristics.

[0011] As a further solution of the present invention, the step of obtaining the coordinates of the mask region boundary fusion transition zone is specifically as follows: S411: Based on the global distribution restoration result, extract the pixel point data inside the mask area, including color distribution, grayscale value and texture characteristics, balance the distribution of grayscale value concentrated or discrete areas, smooth the texture change mutation points, and obtain the adjustment result of the pixel distribution inside the mask area; S412: Based on the adjustment result of the pixel distribution inside the mask area, the formula is used: ; Calculate the coordinates of the transition zone at the boundary of the mask area : in, Indicates The weight value of the neighboring pixels, Indicates The coordinate values ​​of the neighboring pixels, Represents the total number of pixels in the neighborhood that participate in weighting.

[0012] As a further solution of the present invention, the steps of obtaining the mask region fusion optimization result are specifically as follows: S421: based on the coordinate neighborhood pixel color values ​​and texture gradients of the mask area boundary fusion transition zone, sequentially collect the neighborhood color values ​​at the junction, adjust the weights and perform interpolation processing on the pixels at the junction to generate a smooth transition result of the boundary area; S422: Based on the smooth transition result of the boundary area, the smoothed boundary pixel data is recursively extended layer by layer to the interior of the mask area, and the internal pixel values ​​are gradually adjusted according to the neighborhood color value and texture distribution to generate a mask area fusion optimization result.

[0013] An integrated image restoration system based on image mask modeling, the integrated image restoration system based on image mask modeling is used to execute the above-mentioned integrated image restoration method based on image mask modeling, the system comprises: The boundary data analysis module separates the boundary between the mask area and the non-mask area based on the boundary data of the entire image, analyzes the gradient change characteristics of the boundary of the mask area, and extracts the gradient change characteristics of the boundary of the mask area; The boundary residual modeling module extracts the local depth information of the mask area from the gradient change characteristics based on the boundary gradient change characteristics of the mask area, and constructs a mask area boundary residual distribution model; The basic feature generation module analyzes the basic features of the mask region based on the data in the residual distribution model of the mask region boundary according to the gradient direction and depth information of the mask region boundary, applies the basic feature distribution to the inside of the mask region, and generates a basic structural feature map of the mask region; The global feature adjustment module obtains context features of the mask area and the non-mask area of ​​the whole image based on the data in the basic structure feature map of the mask area, analyzes the global dependency between the mask area and the surrounding context features, adjusts the preliminary repair features of the mask area according to the global dependency, and generates a global distribution repair result; The fusion optimization module extracts the pixel distribution information inside the mask area based on the global distribution restoration result, performs local balanced adjustment on the pixel distribution inside the mask area, obtains the coordinates of the fusion transition zone at the boundary of the mask area, performs interpolation adjustment on the pixel values ​​in the fusion transition zone, and generates the mask area fusion optimization result.

[0014] Compared with the prior art, the advantages and positive effects of the present invention are: In the present invention, by separating the masked area and the non-masked area and extracting the boundary gradient features, a residual distribution model is established based on the gradient change, and the subtle changes in the boundary of the masked area are accurately captured, thereby improving the accuracy of the boundary features. By constructing a depth information data set, the local depth information is combined with the gradient features, and the hierarchical dynamic weight distribution of the basic structure of the masked area is adjusted, ensuring the effective extension of the regional features to the inside and reducing the possibility of boundary information loss. In the global repair process, the conditional probability is calculated by comparing and associating the global context features of the masked area and the non-masked area, the global dependency between the masked area and the context is clarified, and the adjustment path of the preliminary repair features is optimized. In subsequent operations, the pixel interpolation adjustment and layer-by-layer expansion strategy are used to smooth the fusion transition between the masked area boundary and the internal pixels, and gradually optimize the overall repair effect. The overall processing logic is continuous and targeted in terms of partition step-by-step analysis, feature extraction, and repair path optimization. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a schematic diagram of the workflow of the present invention; Figure 2 A flow chart of analyzing the gradient change characteristics of the mask area boundary of the present invention; Figure 3 A flow chart of obtaining a residual distribution model of a mask region boundary in the present invention; Figure 4 A flowchart of obtaining a basic structural feature map of a mask region according to the present invention; Figure 5 A flowchart of obtaining a global distribution repair result for the present invention; Figure 6 A flow chart of obtaining the coordinates of the mask region boundary fusion transition zone of the present invention; Figure 7 The present invention is a flowchart for obtaining mask region fusion optimization results. DETAILED DESCRIPTION

[0016] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0017] In the description of the present invention, it should be understood that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, in the description of the present invention, "multiple" means two or more, unless otherwise clearly and specifically defined.

[0018] See also Figure 1 The present invention provides a technical solution: an integrated image restoration method based on image mask modeling, comprising the following steps: S1: Collect the boundary data of the entire image, separate the boundary of the masked area and the non-masked area, obtain the gradient amplitude of the pixel points at the boundary of the masked area and perform residual calculation, analyze the gradient change characteristics of the boundary of the masked area according to the calculation results, extract the local depth information of the masked area from the gradient change characteristics, integrate the local depth information and the gradient change characteristics, and construct a residual distribution model of the boundary of the masked area; S2: Based on the data in the residual distribution model of the mask region boundary, the gradient value and texture characteristics in the neighborhood of the mask region boundary are dynamically weighted, and the basic features of the mask region are analyzed according to the gradient direction and depth information of the mask region boundary. The basic feature distribution is applied to the inside of the mask region, and the pixels at the mask region boundary are dynamically fitted to generate the basic structural feature map of the mask region; S3: Based on the data in the basic structural feature map of the masked area, the context features of the masked area and the non-masked area of ​​the whole image are obtained, the global dependency between the masked area and the surrounding context features is analyzed, the preliminary repair features of the masked area are adjusted according to the global dependency, and the global distribution repair results are generated by iteratively optimizing the preliminary repair features; S4: Based on the global distribution restoration result, extract the pixel distribution information inside the mask area, perform local equalization adjustment on the pixel distribution inside the mask area, extract the color and texture gradient of the boundary area according to the boundary position between the mask area and the non-masked area of ​​the whole image, combine the pixel distribution and color gradient at the boundary position, obtain the coordinates of the fusion transition area of ​​the mask area boundary, perform interpolation adjustment on the pixel values ​​of the fusion transition area, smooth the junction of the mask area boundary and the non-masked area through the boundary interpolation result, and extend the boundary interpolated data to the inside of the mask area, so as to realize the layer-by-layer fusion optimization of the boundary and internal pixels, and generate the mask area fusion optimization result; The residual distribution model of the mask area boundary includes boundary pixel gradient data, neighborhood texture change distribution, and boundary gradient residual characteristics. The basic structure feature map of the mask area includes boundary pixel fitting features, basic feature distribution data, and gradient direction mapping. The global distribution repair result specifically includes preliminary repair feature distribution, context dependency mapping, and global pixel distribution optimization data. The mask area fusion optimization result specifically refers to pixel balanced distribution data, boundary interpolation adjustment data, and internal layer-by-layer fusion results.

[0019] See also Figure 2 ,The steps of analyzing the gradient change characteristics of the mask area boundary are as follows: S111: Collect the boundary data of the entire image, separate the boundary between the masked area and the non-masked area, and use the formula: ; Calculate the gradient magnitude of the boundary pixels of the mask area ; in, It is the grayscale change rate of the pixel in the horizontal x direction, which is obtained by convolution on the image using a filter (such as the Sobel filter). It is the grayscale change rate of the pixel in the vertical y direction, which is also obtained by convolution calculation of the filter in the vertical direction. It is the grayscale value matrix of the input image, which comes from the image data and is obtained through image preprocessing (such as grayscale conversion).

[0020] Take the neighborhood of a point in the gray value matrix as: ; ; ; Substitute into the formula: ; The result shows that the gradient amplitude of the pixel is 89.44.

[0021] The specific process of collecting the boundary data of the entire image and separating the boundary of the masked area and the non-masked area is as follows: first, all the boundary information of the image is extracted through image segmentation technology, and the edge detection algorithm such as the Canny algorithm or the Sobel operator is used to obtain the preliminary pixel coordinate set of the overall boundary of the image. The image is further binarized based on the definition of the masked area to distinguish the masked area from the non-masked area, and the boundary pixels of the masked area are screened twice. The masked boundary pixel set and the non-masked boundary pixel set are established through coordinate mapping; after the boundary screening is completed, the gradient of the boundary pixels of the masked area is calculated, and the calculated gradient value is used to reflect the intensity of the grayscale change of the boundary pixel point. A larger gradient value indicates that the point may be a significant boundary or edge of the image. Combined with the texture change analysis of adjacent pixels, the gradient value can help determine the significance of the boundary pixel points of the masked area and its grayscale relationship with the neighborhood. For example, the size of the gradient value depends on the grayscale change range of the image. For example, in an image with a grayscale value range of 0-255, the common interval distribution of gradient values ​​is: low gradient value: generally in the range of 0-20, indicating that the grayscale change of the pixel is not obvious, usually appearing in flat areas or areas with low noise; medium gradient value: in the range of 20-100, indicating that there is a certain grayscale change of the pixel, which may appear in texture or weak boundary areas; high gradient value: greater than 100, indicating that the grayscale change of the pixel is significant, usually corresponding to clear edges or significant object boundaries.

[0022] S112: According to the gradient amplitude of the pixel points at the boundary of the mask area, the formula is used: ; Calculate boundary pixels The gradient residual value of ; in, is the boundary pixel of the mask area The gradient amplitude is calculated by the formula Calculate the corresponding boundary pixels respectively Obtained, specifically reflects the grayscale change intensity of the image pixel at the boundary, is the boundary pixel The average value of the gradient amplitude of adjacent non-masked area pixels is calculated as: , is the boundary pixel The number of adjacent pixels, is the gradient value of the adjacent pixel, and the gradient value is obtained by statistical calculation of the grayscale gradient of the neighboring pixels. Represents the two-dimensional coordinates of the boundary pixels in the image, is the row number (vertical index), is the column number (horizontal index), represented by the two-dimensional matrix of the image, pixel point The position of is determined by its row and column index in the matrix, is with The coordinates of adjacent pixels, where is the index number of the pixel in the neighborhood.

[0023] Suppose the coordinates of a boundary pixel in the image are , whose gradient value is The gradient value of adjacent pixels is , the corresponding average neighborhood gradient is: ; Substituting these values ​​into the formula to calculate the residual value: ; The results show that: At the boundary pixel point, the gradient residual value is 24.44.

[0024] S113: Based on the gradient residual values, the gradient residual values ​​of the mask boundary points are sorted according to the coordinates, and the central tendency and discrete tendency of the residual values ​​of all boundary points are statistically analyzed to generate a gradient residual distribution feature recognition result; First, we need to statistically organize the gradient residual values ​​of all boundary points, and then calculate the gradient residual values ​​of the mask boundary points by coordinates. After sorting, a gradient residual distribution table is constructed, and the residual distribution is visualized using a histogram. The central tendency and discrete tendency of the residuals are analyzed using statistical methods. The central position of the overall gradient residual is obtained using the average value method. For example, each value in the calculated gradient residual set is accumulated and divided by the total number of boundary points to obtain the average residual value. At the same time, the extreme value difference in the neighborhood data is used to calculate the difference range of the gradient change inside and outside the boundary. Specifically, the gradient residual value set of multiple boundary points can be collected to obtain the residual mean and the degree of fluctuation of adjacent values, and its change characteristics can be analyzed in combination with the residual distribution diagram of the overall boundary. In addition, by detecting the relative difference in gradient change values ​​inside and outside the mask boundary, intuitive diagrams (such as box plots) are used to further compare the local gradient distribution and extract the boundary change characteristics. For example, in a certain area, the gradient residual values ​​are concentrated in the range of 20 to 30, indicating that the boundary point features are more obvious, and when the gradient residual value is lower than 15, the significance of the boundary point gradually decreases; at the same time, the gradient changes of specific boundary points are locally verified during the calculation process. For example, by further sampling areas with larger residual values ​​(such as the part where the boundary point residual is 30.55), the gradient value fluctuations of its upper and lower neighbors are statistically analyzed, the change amplitude between the masked and non-masked areas is recorded, and the boundary area is described in detail. In this way, the significant changes of the boundary can be clarified and gradient features with depth information can be extracted.

[0025] See also Figure 3 , the specific steps for obtaining the residual distribution model of the mask region boundary are: S121: Based on the identification result of the gradient residual distribution feature, the residual value is matched with the gradient direction, and the corresponding relationship between the gradient change amplitude and the preset depth information is combined to obtain the local depth corresponding to each boundary point, so as to obtain a depth information data set; A correspondence between the gradient change amplitude and the depth information is established using a preset depth imaging reference. For example, in experimental data, points with gradient residual values ​​in the range of 20-30 usually correspond to depth information in the range of 1.4-1.6 meters. The local depth value of each boundary point is calculated using this reference relationship. For example, the gradient residual values ​​of boundary pixels are extracted from multiple reference images. At the same time, the three-dimensional model real depths corresponding to these points in the image are combined, and a data fitting method (such as linear regression or polynomial fitting) is used to construct a mapping relationship between gradient residual and depth. In an application scenario, the gradient residual value of each boundary point in the target image is queried using the mapping table established above to directly obtain the corresponding local depth value, including a point with a residual of 24.44 mapped to a depth value of 1.5 meters. Then, these depth values ​​are uniformly recorded with the gradient residual in the coordinate order of the boundary pixels to generate a depth information data set indexed by the boundary point coordinates.

[0026] S122: Based on the depth information dataset, the local depth information and gradient change features are integrated to construct a residual distribution model of the mask region boundary; First, the local depth information and the corresponding gradient residual value are matched and integrated in the form of coordinate index. For example, the gradient residual value 24.44 and the depth information 1.5 meters are integrated into the boundary point A three-dimensional feature table containing coordinates, gradient residuals and depth information is generated from a dataset; then, the gradient residuals and depth information are normalized separately to unify the range to eliminate data scale differences, such as mapping the range of gradient residuals to the interval of 0-1, and the depth information is also standardized in the same way; on this basis, the distribution characteristics of the overall data are statistically analyzed through data analysis tools (such as Python's Pandas or NumPy), such as calculating the correlation between gradient residuals and depth information, and further analyzing the local pattern of depth distribution and gradient changes; finally, MATLAB or Python's Matplotlib is used to generate visualization results of the residual distribution model, such as drawing a two-dimensional scatter plot or contour map of gradient residuals and depth information to show the relationship between the gradient change of boundary points and the distribution of depth information, and the results are stored as data files for subsequent analysis.

[0027] See also Figure 4 , the steps for obtaining the basic structural feature map of the mask area are as follows: S211: Based on the data in the residual distribution model of the mask region boundary, the gradient value and texture characteristics in the neighborhood range of the mask region boundary are analyzed, the weights are set hierarchically according to the gradient size and the weights are adjusted in combination with the texture characteristics to generate a dynamic weight distribution list of the neighborhood pixels; First, the distribution range of gradient values ​​and texture characteristics is analyzed. For example, the range of gradient values ​​in the neighborhood range obtained from the residual distribution model is between 10-50, and the range of texture change characteristic values ​​is between 0.2-0.8. The experimental data shows that the gradient value has a more significant impact on the boundary pixels, so the weight of the gradient value is initially set to be higher; secondly, in the weight adjustment process, the gradient value of the pixels in the neighborhood is layered according to the size of the gradient value. For example, for pixels with a gradient value greater than 40, a higher weight is assigned, and for pixels with a gradient value less than 20, a lower weight is assigned. At the same time, the final weight is fine-tuned in combination with the texture characteristic value. For example, when the gradient value is 4 5 and the texture change characteristic is 0.7, the weight of this point is too high; in the operation implementation, each pixel point in the neighborhood is processed one by one, its gradient value and texture characteristic value are read, and the weights of the two are adjusted according to the set rules. For example, points with gradient values ​​above 40 are adjusted to the high priority weight range (for example, 0.8-1.0), and pixels with texture change characteristics below 0.5 have their weights reduced to the medium and low priority range (for example, 0.4-0.6); finally, after completing the weight adjustment, the dynamic weight results are recorded in the corresponding pixel point coordinate table to ensure that the dynamic weight of each pixel in the neighborhood can be further used.

[0028] S212: Based on the dynamic weight distribution list of the neighborhood pixels, the basic features of the boundary pixels of the mask area are analyzed, the gradient direction and the depth value are gradually extended to the inside of the mask area, and the feature distribution of the internal pixels is adjusted according to the neighborhood distance to generate a basic structural feature map of the mask area; First, the gradient direction data of the mask boundary is analyzed, and the distribution range of the gradient directions of different boundary points is counted. For example, the main gradient direction of a certain boundary area is concentrated in 60-80 degrees. Then, the depth characteristics at the boundary are analyzed in combination with the depth information. For example, the depth value gradually decreases within the boundary point range, changing from 1.6 meters to 1.2 meters. These data are used to determine the changing trend of the basic features. In the fitting process, the gradient direction and depth information of the boundary are gradually extended to the inside of the mask area, and the feature distribution of the internal pixels is dynamically adjusted by the neighborhood distance. For example, when the distance from the boundary point to the internal pixel is 5 pixels, its depth feature value is adjusted in a linear decreasing manner, and the gradient direction gradually tends to the main direction of the boundary. For the pixel points close to the center of the mask area, the multi-point fitting method is preferentially used, and the gradient directions and depth values ​​of multiple neighborhood boundary points are referenced for weighted fitting to smooth the transition characteristics between the boundary and the central area. Finally, the basic structure feature map is generated, and the fitting data is verified in combination with the comparison results of the boundary and internal features. For example, by verifying the smoothness and consistency of the internal features of the mask area, the accuracy of the fitting process is confirmed, and the basic feature fitting results of the mask area are completed.

[0029] See also Figure 5 , the specific steps for obtaining the global distribution repair results are: S311: Based on the data in the basic structural feature map of the mask area, the gradient direction, depth value and texture characteristics of the boundary of the mask area are extracted, and the context features of the non-masked area of ​​the whole image are partitioned and counted, and the distribution range of the gradient direction, depth value and texture characteristics is recorded, and the context features are classified to generate feature data of the mask area and the context area; First, the basic structural features of the masked area boundary are analyzed, such as extracting gradient direction, depth value and texture characteristics, and storing these feature data according to the coordinates of the boundary points; then, the features of the non-masked area of ​​the entire image are partitioned and counted, and the non-masked area adjacent to the masked area is screened out through the neighborhood search method, and the distribution range of the gradient direction and depth value is recorded. For example, the points with a gradient direction of 45 degrees on the masked area boundary are compared with the gradient direction data of the adjacent non-masked area to determine the main distribution of similar gradient directions; then, the contextual relationship between the masked area and the non-masked area is verified through experimental methods, such as counting the consistency ratio of the gradient direction of the masked boundary points and the gradient direction of the adjacent non-masked area, and at the same time analyzing the transition law of the depth value to obtain the deviation range of the depth characteristic between the masked area boundary and the context area.

[0030] S312: Based on the feature data of the mask area and the context area, the formula is used: ; Calculate in context features Mask region features under given conditions The conditional probability , get the global dependency information; in, Represents the characteristics of the mask area, which can include gradient direction, depth value, texture characteristics, Represents the contextual features of the non-masked area in the whole image, including gradient direction, depth value, and texture characteristics. These features are obtained by statistical analysis of the neighborhood of the masked area and the global non-masked area. Indicates the features in the mask area In the presence of contextual features The probability of occurrence is calculated by counting the number of occurrences of context features in the neighborhood of the mask boundary. For example, the matching frequency of the context feature values ​​in the corresponding neighborhood at the boundary point of the mask area is counted. Features in the mask area The number of occurrences in the field and the characteristics of the mask area Compared with the total number, Represents the mask region features The prior probability is obtained from the basic structural feature map of the mask area, and is calculated by counting the distribution of the internal and boundary features of the mask area. For example, the gradient direction data in the mask area is counted, the proportion of each gradient direction is calculated, and the distribution of the depth value is statistically analyzed. For example, after dividing into depth value intervals, the probability of occurrence of each interval is calculated. Representing contextual features The total probability is calculated through the statistical data of the non-masked area of ​​the whole image, for example: statistically analyzing the frequency distribution of gradient directions in the whole image, calculating the overall distribution ratio of depth values ​​in the non-masked area, and analyzing the frequency of occurrence of texture characteristics in the whole image. Represents the mask region features and contextual features The joint probability of simultaneous occurrence is obtained by multiplying the frequency of occurrence of the context feature in the neighborhood of the mask region feature and the prior probability of the mask region feature. Representing contextual features The probability of occurrence in the entire image is a normalization factor, which is calculated by summing the distribution of context features in the entire image.

[0031] Let the feature of the mask area be the gradient direction The context feature is the gradient direction Spend: :By counting the distribution of context features at the mask boundary, calculate the frequency of occurrence of context features with a gradient direction of 50 degrees in the neighborhood of mask feature points with a gradient direction of 45 degrees. For example, among 100 mask boundary points, 80 neighborhood context features have a gradient direction of 50 degrees, then .

[0032] : Count the features with a gradient direction of 45 degrees from the mask area feature map. For example, among 1000 mask feature points, 600 have a gradient direction of 45 degrees. .

[0033] : Count the proportion of context features with a gradient direction of 50 degrees from the entire image. For example, the entire image contains 2000 feature points in the non-masked area, of which 1400 have a gradient direction of 50 degrees. .

[0034] Substitute the above data into the formula: ; The results show that: the mask region features The gradient direction of the degree is in the context feature The probability of occurrence under the condition of degree is 0.686.

[0035] S313: adjusting the preliminary restoration features of the mask area according to the global dependency information, and generating a global distribution restoration result by gradually optimizing the gradient direction, depth value and texture characteristics; First, the calculated mask region features The gradient direction of the degree is in the context feature The probability value under the condition of Into the optimization analysis. According to the experimental statistical rules, if the conditional probability value is If the conditional probability value is within the range of , it means that the mask area feature has a high correlation with the context feature and does not require large-scale adjustment, but only needs to refine the local feature distribution. , it means that the correlation is medium, and it is necessary to adjust the features in the mask area as a whole; , it means that the mask area has a low correlation with the context feature, and the mask area feature needs to be corrected first. Falling into the medium correlation range, according to the global dependency characteristics of the gradient direction, the preliminary restoration features of the mask area are adjusted as a whole. The specific operations are as follows: first, the distribution of the gradient direction in the mask area is re-optimized. For example, for areas with large gradient direction deviations, a direction fitting method based on contextual features is used to adjust the gradient direction in the mask area point by point to the main direction close to the neighborhood context feature (such as 50 degrees); then, the area after the gradient direction adjustment is smoothed to solve the problem of sudden changes in direction during the adjustment process, and the adjusted gradient direction distribution is smoothly transitioned to the interior of the mask area through the local mean method; at the same time, the depth value of the mask area is assisted in optimization. By counting the depth distribution range of the context area (such as 1.2 meters to 1.6 meters), the depth value of the mask area is limited to the context range, and the correlation between the depth value and the gradient direction is adjusted synchronously. After each round of optimization, the conditional probability of the mask area features and the context features is recalculated, and the results are checked step by step. Whether the value is raised to a high correlation range (such as ), if the high correlation standard is reached, the optimization is stopped; in the final generated global distribution restoration result, the gradient direction and depth value distribution of the mask area are consistent with the context features, and the correlation is significantly improved, thereby ensuring that the features of the mask area are consistent and reasonable in the global context.

[0036] See also Figure 6 , the steps for obtaining the coordinates of the transition zone of the mask region boundary fusion are as follows: S411: Based on the global distribution restoration result, extract the pixel point data inside the mask area, including color distribution, grayscale value and texture characteristics, balance the distribution of grayscale value concentrated or discrete areas, smooth the texture change mutation points, and obtain the adjustment result of the pixel distribution inside the mask area; First, use Python's OpenCV library to analyze the grayscale image of the mask area, extract the grayscale value and color distribution information of each pixel in the mask area, and generate a grayscale value histogram of the mask area through the cv2.calcHist function to count the concentration range and distribution trend of the grayscale value; then, use texture analysis tools such as Gabor filter to perform local frequency analysis on the texture characteristics of the mask area, extract the texture change characteristics of the high-frequency area and the low-frequency area, and use the Sobel operator to obtain the gradient direction data in the mask area and generate a gradient direction distribution map; perform balanced analysis on the extracted grayscale value histogram and use cv2.eq The ualizeHist function evenly adjusts the distribution of grayscale value concentrated or discrete areas to make the distribution of pixel grayscale values ​​inside the mask area tend to be consistent; for texture feature data, the texture feature mutation points are smoothed by calculating the local mean of the high-frequency mutation area (using NumPy's mean function np.mean); finally, according to the boundary position of the mask area and the non-masked area of ​​the whole image, the color and texture gradient data of the boundary area are extracted, and the color gradient change value and texture characteristics of the boundary pixels are analyzed by the Sobel operator and Gabor filter. The color and texture characteristic data of each boundary pixel are recorded to form complete mask area boundary data.

[0037] S412: Based on the adjustment result of the pixel distribution inside the mask area, the formula is used: ; Calculate the coordinates of the transition zone at the boundary of the mask area : in, Indicates The weight value of the neighboring pixels is calculated based on the distance from the neighboring pixels to the center point and the color gradient change: it is obtained by calculating the Euclidean distance between the neighboring pixel point and the center point of the mask area boundary. The closer the distance, the greater the weight. For example ,in , is the coordinate of the center point of the mask area boundary, color gradient weight: obtained by calculating the color gradient change of the neighboring pixels. The smoother the gradient change, the higher the weight, for example , Indicates The color gradient change value of each pixel, and the final weight value is obtained by weighted combination of distance weight and color gradient weight, for example , In order to balance the parameters, experimental data were set to record the restoration effects under different parameter combinations (such as color smoothness index, gradient continuity index, etc.), and the parameter combination with the highest color smoothness and the best gradient continuity was determined as the final value, Indicates The coordinate values ​​of the neighboring pixels are directly obtained through the image coordinate system, for example, by using image processing tools (such as OpenCV's numpy.nonzero) to extract the coordinates of all pixels on the boundary of the mask area. Indicates the total number of pixels in the neighborhood that participate in the weighted calculation, which depends on the size of the neighborhood selected during the calculation. For example, if the neighborhood is hour, .

[0038] Suppose 4 neighboring pixels are selected in the boundary area, and their coordinates are , , , ; Color gradient weight ; Distance weight By combining the weight formula , calculate the final weight: ; Substituting into the formula: ; Calculate the weighted sum: ; calculate : ; calculate : ; Calculation results: .

[0039] The results show that the center coordinates of the transition zone of the mask region boundary fusion are , which means that by weighted averaging of neighboring pixels, the position of the fusion transition area can smoothly transition into the mask area, providing an accurate reference point for further color and texture adjustment.

[0040] See also Figure 7 , the specific steps for obtaining the mask region fusion optimization results are: S421: based on the coordinate neighborhood pixel color values ​​and texture gradients of the transition zone at the mask region boundary fusion, the neighborhood color values ​​at the junction are collected in sequence, the weights are adjusted and the pixels at the junction are interpolated to generate a smooth transition result of the boundary region; First, the neighboring pixels at the junction of the masked area boundary and the non-masked area are selected as the focus of the interpolation processing. For example, when repairing a landscape picture with a part of the area missing due to damage, the masked area may be located at the junction of the mountains and the sky, and the color change of the boundary area needs to maintain a natural transition. At this time, the interpolation processing for the boundary needs to first collect the color values ​​of the neighborhood on both sides of the boundary (the junction of the non-masked area and the masked area). Assuming that the mask boundary color gradually changes from dark blue to light blue, and the area adjacent to the sky is light blue, the distance weight and the color gradient change weight are combined when calculating the neighborhood weight. The neighboring pixels such as the light blue area have a higher weight, and the dark blue area has a lower weight. Then, the boundary pixel color is interpolated and adjusted based on the weighted calculation. For example, the darker blue boundary value will gradually be replaced by the neighboring light blue value in a smooth transition, thereby generating a smooth transition boundary. The whole process ensures the texture continuity of the color value and the boundary neighborhood, thereby realizing the interpolation adjustment and smooth optimization of the mask boundary pixels.

[0041] S422: Based on the smooth transition result of the boundary area, the smoothed boundary pixel data is recursively extended layer by layer to the inside of the mask area, and the internal pixel values ​​are gradually adjusted according to the neighborhood color value and texture distribution to generate the mask area fusion optimization result; The smoothed boundary data is extended layer by layer to the interior of the mask area. Taking the integrated restoration of image mask modeling as an example, in the same landscape image, the missing part of the interior of the mask area may be a mixed area of ​​mountains and sky. After the boundary interpolation is completed, the next step is to recursively interpolate the interior layer by layer. The expansion process starts from the boundary layer and gradually expands toward the center of the mask. Each expansion uses the interpolation result of the previous layer as a reference, and combines the color value and texture distribution of the neighboring pixels for multi-directional interpolation. For example, when generating a color transition inside the mask, the interpolation value of the light blue area of ​​the boundary layer will gradually expand to the interior, while considering the texture changes of the mountain color to increase the color gradient complexity. After each layer of expansion is completed, the color gradient and texture distribution in the result are checked. For example, in the landscape image, ensure that the color change of the sky blends naturally with the texture change of the mountain to avoid abrupt boundaries. Through this layer-by-layer interpolation and expansion method, the fusion optimization of the mask area is finally achieved, and the generation is consistent with the whole Figure 1 The natural transition effect is achieved, completing the goal of integrated image restoration.

[0042] An integrated image restoration system based on image mask modeling, the integrated image restoration system based on image mask modeling is used to execute the above-mentioned integrated image restoration method based on image mask modeling, the system comprises: The boundary data analysis module separates the boundary between the mask area and the non-mask area based on the boundary data of the entire image, analyzes the gradient change characteristics of the boundary of the mask area, and extracts the gradient change characteristics of the boundary of the mask area; The boundary residual modeling module extracts the local depth information of the mask area from the gradient change features based on the boundary gradient change features of the mask area, and constructs the boundary residual distribution model of the mask area. The basic feature generation module analyzes the basic features of the mask region based on the data in the residual distribution model of the mask region boundary according to the gradient direction and depth information of the mask region boundary, applies the basic feature distribution to the inside of the mask region, and generates a basic structural feature map of the mask region; The global feature adjustment module obtains the context features of the masked area and the non-masked area of ​​the whole image based on the data in the basic structure feature map of the masked area, analyzes the global dependency between the masked area and the surrounding context features, adjusts the preliminary repair features of the masked area according to the global dependency, and generates a global distribution repair result; Based on the global distribution restoration results, the fusion optimization module extracts the pixel distribution information inside the mask area, performs local balanced adjustment on the pixel distribution inside the mask area, obtains the coordinates of the fusion transition zone at the boundary of the mask area, performs interpolation adjustment on the pixel values ​​in the fusion transition zone, and generates the mask area fusion optimization result.

[0043] The above are only preferred embodiments of the present invention and are not intended to limit the present invention in other forms. Any technician familiar with the profession may use the technical contents disclosed above to change or modify them into equivalent embodiments with equivalent changes and apply them to other fields. However, any simple modification, equivalent change and modification made to the above embodiments based on the technical essence of the present invention without departing from the technical solution of the present invention still falls within the protection scope of the technical solution of the present invention.

Claims

1. An integrated image restoration method based on image mask modeling, characterized in that: The following steps are involved: S1: Collect the boundary data of the entire image, separate the boundary between the masked area and the non-masked area, analyze the gradient change characteristics of the boundary of the masked area, extract the local depth information of the masked area from the gradient change characteristics, and construct a residual distribution model of the boundary of the masked area; S2: Based on the data in the residual distribution model of the mask region boundary, the basic features of the mask region are analyzed according to the gradient direction and depth information of the mask region boundary, and the basic feature distribution is applied to the inside of the mask region to generate a basic structural feature map of the mask region; S3: Based on the data in the basic structural feature map of the masked area, context features of the masked area and the non-masked area of ​​the whole image are obtained, the global dependency relationship between the masked area and the surrounding context features is analyzed, and the preliminary restoration features of the masked area are adjusted according to the global dependency relationship to generate a global distribution restoration result; S4: Based on the global distribution restoration result, extract the pixel distribution information inside the mask area, perform local balanced adjustment on the pixel distribution inside the mask area, obtain the coordinates of the fusion transition zone at the boundary of the mask area, perform interpolation adjustment on the pixel values ​​of the fusion transition zone, and generate the mask area fusion optimization result.

2. The integrated image restoration method based on image mask modeling according to claim 1, characterized in that: The step of analyzing the gradient change characteristics of the mask region boundary is specifically as follows: S111: Collect the boundary data of the entire image, separate the boundary between the masked area and the non-masked area, and use the formula: ； Calculate the gradient magnitude of the boundary pixels of the mask area ; in, is the grayscale change rate of the pixel in the horizontal x direction, is the grayscale change rate of the pixel in the vertical y direction, is the gray value matrix of the input image; S112: According to the gradient amplitude of the pixel points at the boundary of the mask area, the formula is adopted: ； Calculate boundary pixels The gradient residual value of ; in, is the boundary pixel of the mask area The gradient amplitude of is the boundary pixel The average value of the gradient magnitude of adjacent non-masked area pixels; S113: Based on the gradient residual values, the gradient residual values ​​of the mask boundary points are sorted according to coordinates, and a gradient residual distribution feature recognition result is generated by statistically analyzing the central tendency and discrete tendency of the residual values ​​of all boundary points.

3. The integrated image restoration method based on image mask modeling according to claim 2 is characterized in that: The steps for obtaining the residual distribution model of the mask region boundary are specifically as follows: S121: Based on the gradient residual distribution feature recognition result, the residual value is matched with the gradient direction, and the corresponding relationship between the gradient change amplitude and the preset depth information is combined to obtain the local depth corresponding to each boundary point to obtain a depth information data set; S122: Based on the depth information data set, local depth information and gradient change features are integrated to construct a mask region boundary residual distribution model.

4. The integrated image restoration method based on image mask modeling according to claim 3 is characterized in that: The steps for obtaining the basic structural feature map of the mask area are specifically as follows: S211: Based on the data in the residual distribution model of the mask region boundary, the gradient value and texture characteristics in the neighborhood range of the mask region boundary are analyzed, the weights are set in layers according to the gradient size and the weights are adjusted in combination with the texture characteristics to generate a dynamic weight distribution list of the neighborhood pixels; S212: Based on the dynamic weight distribution list of the neighborhood pixels, the basic features of the boundary pixels of the mask area are analyzed, the gradient direction and depth value are gradually expanded into the mask area, and the feature distribution of the internal pixels is adjusted according to the neighborhood distance to generate a basic structural feature map of the mask area.

5. The integrated image restoration method based on image mask modeling according to claim 4 is characterized in that: The steps for obtaining the global distribution repair result are specifically as follows: S311: Based on the data in the basic structural feature map of the mask area, the gradient direction, depth value and texture characteristics of the boundary of the mask area are extracted, and the context features of the non-masked area of ​​the whole image are partitioned and counted, and the gradient direction, depth value and texture characteristic distribution range are recorded, and the context features are classified to generate feature data of the mask area and the context area; S312: Based on the feature data of the mask area and the context area, the formula is used: ； Calculate in-context features Mask region features under given conditions The conditional probability , get the global dependency information; in, Indicates the features in the mask area In the presence of contextual features The probability of occurrence, Represents the mask region features The prior probability of Representing contextual features The total probability of S313: According to the global dependency information, the preliminary restoration features of the mask area are adjusted, and a global distribution restoration result is generated by gradually optimizing the gradient direction, depth value and texture characteristics.

6. The integrated image restoration method based on image mask modeling according to claim 5, characterized in that: The steps for obtaining the coordinates of the mask region boundary fusion transition zone are specifically as follows: S411: Based on the global distribution restoration result, extract the pixel point data inside the mask area, including color distribution, grayscale value and texture characteristics, balance the distribution of grayscale value concentrated or discrete areas, smooth the texture change mutation points, and obtain the adjustment result of the pixel distribution inside the mask area; S412: Based on the adjustment result of the pixel distribution inside the mask area, the formula is used: ； Calculate the coordinates of the transition zone at the boundary of the mask area : in, Indicates The weight value of the neighboring pixels, Indicates The coordinate values ​​of the neighboring pixels, Represents the total number of pixels in the neighborhood that participate in weighting.

7. The integrated image restoration method based on image mask modeling according to claim 6 is characterized in that: The steps for obtaining the mask region fusion optimization result are specifically as follows: S421: based on the coordinate neighborhood pixel color values ​​and texture gradients of the mask area boundary fusion transition zone, sequentially collect the neighborhood color values ​​at the junction, adjust the weights and perform interpolation processing on the pixels at the junction to generate a smooth transition result of the boundary area; S422: Based on the smooth transition result of the boundary area, the smoothed boundary pixel data is recursively extended layer by layer to the interior of the mask area, and the internal pixel values ​​are gradually adjusted according to the neighborhood color value and texture distribution to generate a mask area fusion optimization result.

8. An integrated image restoration system based on image mask modeling, characterized in that: According to any one of claims 1 to 7, the integrated image restoration method based on image mask modeling comprises: The boundary data analysis module separates the boundary between the mask area and the non-mask area based on the boundary data of the entire image, analyzes the gradient change characteristics of the boundary of the mask area, and extracts the gradient change characteristics of the boundary of the mask area; The boundary residual modeling module extracts the local depth information of the mask area from the gradient change characteristics based on the boundary gradient change characteristics of the mask area, and constructs a mask area boundary residual distribution model; The basic feature generation module analyzes the basic features of the mask region based on the data in the residual distribution model of the mask region boundary according to the gradient direction and depth information of the mask region boundary, applies the basic feature distribution to the inside of the mask region, and generates a basic structural feature map of the mask region; The global feature adjustment module obtains context features of the mask area and the non-mask area of ​​the whole image based on the data in the basic structure feature map of the mask area, analyzes the global dependency between the mask area and the surrounding context features, adjusts the preliminary repair features of the mask area according to the global dependency, and generates a global distribution repair result; The fusion optimization module extracts the pixel distribution information inside the mask area based on the global distribution restoration result, performs local balanced adjustment on the pixel distribution inside the mask area, obtains the coordinates of the fusion transition zone at the boundary of the mask area, performs interpolation adjustment on the pixel values ​​in the fusion transition zone, and generates the mask area fusion optimization result.

Images