A stable field image reconstruction method based on principal texture direction and differential approximation

Abstract

The application discloses a stable field image reconstruction method based on a main texture direction and differential approximation, which combines a stable field with image texture to construct a stable field equation of local image texture; secondly, a direction interval of the main texture direction at the missing pixel point is obtained according to gradient information of each known point in the neighborhood of the missing pixel point, and the known points located in the interval are used as effective field sources participating in the final reconstruction, so that the reconstruction process is performed according to the texture and edge direction; finally, the differential approximation method is adopted for the effective field sources, a field source function between the effective field sources and the missing pixel point is constructed by using a low-order Taylor expansion, the known point information in the effective field source region is fully utilized, and a specific image stable field reconstruction equation is obtained to complete the reconstruction. Experimental verification shows that the method realizes efficient and accurate reconstruction of different types of missing images, and is especially excellent in the reconstruction of the edges and textures of the missing image region.

Description

Technical Field

[0001] This invention belongs to the technical field of damaged image reconstruction, specifically relating to a stable field image reconstruction method based on principal texture direction and differential approximation. Background Technology

[0002] Image reconstruction refers to the process of reconstructing areas of an image that are missing information using known information in the image. Its goal is to restore the image to its original state and meet the visual requirements of the human eye. Early research mainly focused on the removal or filling of specific targets, but later it expanded to the digital reconstruction of precious cultural relics, as well as various interferences (such as occlusion, light spots, etc.) that occur during the image acquisition process.

[0003] Traditional reconstruction algorithms are mainly divided into two categories: one is reconstruction based on geometric features (Inpainting), and its representative algorithms include BSCB algorithm (Bertalmio M, Sapiro G, Caseles V, et al. Image inpainting[C]. Proceedings of the 27th annual conference on Computer graphics and interactive techniques.2000:417-424), TV algorithm (Shen J, Chan TF. Mathematical models for local nontexture inpaintings[J]. SIAM Journal on Applied Mathematics,2002,62(3):1019-1043) and CDD algorithm (Chan TF, Shen J. Nontexture inpainting by curvature-driven dif-fusions[J]. Journal of visual communication and image representation,2001,12(4):436-449). This type of algorithm uses partial differential equations (PDEs) as its core, spreading information from the known region to the inside of the missing region through multiple iterations and calculations of higher-order partial derivatives to complete the reconstruction. Another type is texture-filling-based reconstruction (Completion), with the Criminisi algorithm being a representative example (Criminisi A, Pérez P, Toyama K. Region filling and object removal by exemplar-based image inpainting[J].IEEE Transactions on image processing,2004,13(9):1200-1212). This type of algorithm targets large texture regions in the image, first selecting the block with the highest priority to be repaired within the missing region, and then traversing the entire image to find the block most similar to the missing region, completing the reconstruction by block filling. Both of these traditional image reconstruction algorithms require a large number of iterations during the reconstruction process, and do not perform detailed reconstruction for every missing pixel in the missing region, resulting in low reconstruction efficiency and unsatisfactory reconstruction accuracy.

[0004] To meet the higher requirements of image reconstruction accuracy and efficiency, Ye Xueyi et al. proposed a stable field-based image reconstruction model (Ye Xueyi, Qi Zhenzhen, He Zhiwei, et al. Local Region Reconstruction Based on Directional Derivative of Image Field [J]. Journal of Image and Graphics, 2014, 19(7): 998-1005), and first proposed the concept of accurate reconstruction. Li Xiaofei et al. considered the energy transfer relationship between known points and missing points, and assumed that the energy transferred from the known points is first transferred to virtual points in the nearest neighbor region around the missing points, and then the reconstruction is completed by nearest neighbor interpolation, bilinear interpolation and cubic convolution interpolation respectively (Li Xiaofei, Ye Xueyi, Chen Huiyun, et al. Study on Transfer Function in Stable Field Image Reconstruction [J]. Journal of Image and Graphics, 2018, 23(3): 333-345). However, the image reconstruction based on the stable field model mentioned above uses all known points within a fixed region as the effective field source around the missing point, without making a more accurate selection of the effective field source based on texture information. In addition, when designing the field source function, only one energy transfer method is considered, without taking into account that not all known points around the missing point have second derivatives to satisfy the conditions of the second-order Taylor expansion. This means that the information of these known points is not fully utilized, resulting in the above method still exhibiting incorrect reconstruction of edges and textured regions when reconstructing images with complex textures, and the reconstruction efficiency remains low. Summary of the Invention

[0005] To address the issues of accuracy and efficiency in current image local texture stable field reconstruction algorithms, this invention provides a stable field image reconstruction method based on the main texture direction and differential approximation. This method focuses on the relationship between the field and the source in the stable field to accurately and efficiently reconstruct image defect areas.

[0006] A stable field image reconstruction method based on principal texture direction and differential approximation, comprising the following steps:

[0007] Step (1). Construct the stable field equation for the local texture of the image based on the stable field model;

[0008] In mathematical physics, a stable field refers to a physical quantity whose properties at any point within a certain region do not change or are approximately unchanged over time. The static nature, resolvability, and stability of local image textures make it possible to combine them with stable fields. Combining image processing knowledge, this paper uses stable fields to describe local image textures and establishes the stable field equations for local image textures. Then, using the Green's function method, it solves for the stable field reconstruction formula for a single missing pixel.

[0009] Step (2). Select an effective field source based on the main texture direction;

[0010] The effective field sources in the stable field reconstruction formula for a single defective pixel obtained in step (1) are screened. First, the distance between the known point and the defective point is considered, and a field source region of fixed size centered on the defective point is selected. Then, the main texture direction in the region is obtained according to the gradient magnitude and gradient direction of each known point in the fixed field source region. The corresponding effective field source template is designed according to the main texture direction for the final reconstruction, thereby reducing the impact of gradient direction dispersion in the neighborhood of the defective point on the accurate reconstruction of the edge.

[0011] Step (3). Construct the field source function based on the differential approximation;

[0012] The field source function between the effective field source obtained by filtering according to the main texture direction in step (2) and the corresponding missing point is solved. First, the differential approximation value is calculated according to the Taylor expansion of the effective field source at the missing point of the corresponding order. Then, the similarity factor is defined using this value. Second, the distance factor is designed. The field source function in the stable field reconstruction model is designed by combining the two influencing factors.

[0013] Step (4). Substitute the selected effective field source and the constructed field source function into the image local texture stabilization field function in step (1) to obtain the specific image stabilization field reconstruction formula, and use the formula to complete the reconstruction of each pixel in the local defect area.

[0014] Step (1). The specific method is as follows:

[0015] Step (1.1). Use the stable field model in mathematical physics to describe the local texture of the image and establish the local texture stable field equation of the image. The specific formula is expressed as follows:

[0016] L·I(r)=f,(r∈D) (1)

[0017] In the formula, L represents the linear differential operator, r represents the spatial coordinates of the missing texture in the image, i.e. the missing pixel, I(r) is a function describing the missing texture region D in the image (for grayscale images, the value of I(r) represents the grayscale value of the corresponding spatial point; for color images, such as RGB images, it represents the value of any color channel), and f represents the field source that provides energy to the missing pixel from the known pixel.

[0018] Step (1.2). Select the defect point r located at the boundary of the defect area and the pixel point r located in the known area. i Substituting into equation (1), the specific formula is expressed as follows:

[0019] LG(r,r i )=δ(rr i (2)

[0020] In the formula, δ(rr) i This indicates that when only known pixel points r with unit intensity are considered... i The magnitude of the field source acting solely at the defect point r, G(r,r) i ) represents a known pixel r i The source function between the defect point r and the field source function.

[0021] Equation (2) is solved using the superposition principle in the Green's function method, yielding the stable field reconstruction formula for a single defective pixel. The specific formula is expressed as follows:

[0022]

[0023] In the formula, n represents the total number of effective field sources around the defect point r, and I(r) i ) is r i The specific pixel value of a point.

[0024] Step (2). The specific method is as follows:

[0025] Step (2.1). Considering the distance between the known point and the missing point, select a point centered on the missing point with a radius of... A neighborhood of a unit length is used as a fixed field source region.

[0026] Step (2.2). Calculate the gradient magnitude m(u,v) and gradient direction θ(u,v) at each known point in the fixed source region. The specific formula is expressed as follows:

[0027]

[0028]

[0029] In the formula, g u and g v These represent the horizontal and vertical gradients at each known point, respectively.

[0030] Step (2.3). Map θ(u,v) to [0, 2π] with a value of π / 4 and divide it into four directional intervals α1 to α4 (opposite directions can be merged into the same direction). The specific formula is expressed as follows:

[0031]

[0032] In the formula, k represents the corresponding interval number.

[0033] Step (2.4). Accumulate the gradient magnitudes m(u,v) of all known points whose gradient directions θ(u,v) are within the same interval to obtain the sum of the gradient magnitudes H for each interval. kThen, the gradient direction interval with the largest sum of gradient magnitudes at known points within the fixed source region is taken as the region where the gradient direction of the pixel to be reconstructed is located, and the interval β corresponding to the main texture direction is selected according to the perpendicular relationship between the gradient and the texture. k′ The specific formula is expressed as follows:

[0034]

[0035]

[0036] In the formula, k′ represents the index of the gradient direction interval with the largest sum of gradient magnitudes at all known points within the fixed source region. Known points falling within this main texture direction interval are selected as the final effective source.

[0037] Step (3). The specific method is as follows:

[0038] Step (3.1). If a second-order partial derivative exists at a certain effective field source, that is, if the field source and its eight surrounding neighboring pixels are all known pixels, then the approximate differential value at the defect point can be obtained using the second-order Taylor formula with the pixel value of the effective field source. The specific formula is expressed as follows:

[0039]

[0040]

[0041]

[0042]

[0043]

[0044]

[0045] In the formula, (x,y) represents the coordinates of the missing point, (x0,y0) represents the coordinates of the effective field source, and p and q represent the normalized values ​​of the effective field source relative to the x-axis and y-axis coordinates of the missing point, respectively.

[0046] When there are missing pixels in the neighborhood of the effective source, if there is at least one known point in either the horizontal or vertical direction, then a first-order Taylor expansion is used to obtain the differential approximation of the missing pixel. The specific formula is expressed as follows:

[0047]

[0048]

[0049]

[0050] Step (3.2). The similarity factor sim(r,r) is designed using the difference ratio between the Taylor expansion estimate obtained in step (3.1) and the corresponding source pixel value. i The distance factor dst(r,r) is designed based on the fact that the effective field source attenuates relative to the square of the distance during the transmission of information to the defect point. i Combining the two influencing factors, a weighted average is used to obtain the final source function G(r,r). i The specific formula is expressed as follows:

[0051]

[0052]

[0053]

[0054] In the formula, g(r) i This can avoid including defect points in the calculation, when r i When it is a missing pixel, g(r) i When r = 0, i When the number of pixels is known, g(r) i ) = 1.

[0055] The beneficial effects of this invention are as follows:

[0056] This invention, based on a stable field reconstruction model for image texture, focuses on the relationship between the field and source within the stable field, and conducts in-depth analysis of the selection of effective field sources and the construction of the field source function. For each missing pixel, firstly, the direction interval of the main texture direction at the missing pixel is obtained based on the gradient information of known points in the neighborhood of the missing pixel. Known points located within this interval are used as effective field sources for the final reconstruction, ensuring that the reconstruction process follows the direction of texture and edges, thus overcoming the shortcoming of incorrect extension of reconstruction results in edge and texture regions. Secondly, a differential approximation method is used for the effective field sources, employing a low-order Taylor expansion to calculate the estimated value of the point at the missing point. A similarity factor between the field and the source is designed, and combined with the distance factor between the two points, the field source function is constructed to complete the reconstruction, fully utilizing the known point information in the effective field source region. Experimental results verify that this method achieves efficient and accurate reconstruction of images with different types of defects, especially showing excellent performance in the reconstruction of edges and textures in image defect areas. Attached Figure Description

[0057] Figure 1 This is a flowchart of the method of the present invention;

[0058] Figure 2 This is a diagram of the local texture stabilization field model of the image in the method of the present invention;

[0059] Figure 3 These are the four directional regions in the method of this invention, divided by a factor of π / 4.

[0060] Figure 4 These are the four effective field source templates divided based on the main texture direction in the method of this invention. Detailed Implementation

[0061] The invention will be further described below with reference to the accompanying drawings.

[0062] like Figure 1 As shown, the specific steps of the method of the present invention are as follows:

[0063] Step (1). Construct the stable field equation for the local texture of the image based on the stable field model;

[0064] Using the steady-field model from mathematical physics to describe the local texture in an image, we can obtain, for example... Figure 2 The image shows a stable field model of local texture, where Ω is the boundary (Ω∈D) surrounding the image defect region; Ω1 is a circle with center r and radius arbitrarily small distance Δl, where r... i Let be any point on Ω1.

[0065] Step (1.1). Establish the local texture stability field equation of the image, and the specific formula is expressed as follows:

[0066] L·I(r)=f,(r∈D) (1)

[0067] In the formula, L represents the linear differential operator, r represents the spatial coordinates of the missing texture in the image, i.e. the missing pixel, I(r) is a function describing the missing texture region D in the image (for grayscale images, the value of I(r) represents the grayscale value of the corresponding spatial point; for color images, such as RGB images, it represents the value of any color channel), and f represents the field source that provides energy to the missing pixel from the known pixel.

[0068] Step (1.2). Select the defect point r located at the boundary of the defect area and the pixel point r located in the known area. i Substituting into equation (1), the specific formula is expressed as follows:

[0069] LG(r,r i )=δ(rr i (2)

[0070] In the formula, δ(rr) i This indicates that when only known pixel points r with unit intensity are considered... i The magnitude of the field source acting solely at the defect point r, G(r,r) i ) represents a known pixel r i The source function between the defect point r and the field source function.

[0071] Equation (2) is solved using the superposition principle in the Green's function method, yielding the stable field reconstruction formula for a single defective pixel. The specific formula is expressed as follows:

[0072]

[0073] In the formula, n represents the total number of effective field sources around the defect point r, and I(r) i ) is r i The specific pixel value of a point.

[0074] Step (2). Select an effective field source based on the main texture direction;

[0075] The effective field sources in the stable field reconstruction formula for a single defective pixel obtained in step (1) are screened. First, the distance between the known point and the defective point is considered, and a field source region of fixed size centered on the defective point is selected. Then, the main texture direction in the region is obtained according to the gradient magnitude and gradient direction of each known point in the fixed field source region. The corresponding effective field source template is designed according to the main texture direction for the final reconstruction, thereby reducing the impact of gradient direction dispersion in the neighborhood of the defective point on the accurate reconstruction of the edge.

[0076] Step (2.1). Considering the distance between the known point and the missing point, select a point centered on the missing point with a radius of... A neighborhood of a unit length is used as a fixed field source region.

[0077] Step (2.2). Calculate the gradient magnitude m(u,v) and gradient direction θ(u,v) at each known point in the fixed source region. The specific formula is expressed as follows:

[0078]

[0079]

[0080] In the formula, g u and g v These represent the horizontal and vertical gradients at each known point, respectively.

[0081] Step (2.3). Figure 3 As shown, θ(u,v) is mapped to [0, 2π] with a magnitude of π / 4 and divided into four directional intervals α1 to α4 (opposite directions can be merged into the same direction). The specific formula is expressed as follows:

[0082]

[0083] In the formula, k represents the corresponding interval number.

[0084] Step (2.4). Accumulate the gradient magnitudes m(u,v) of all known points whose gradient directions θ(u,v) are within the same interval to obtain the sum of the gradient magnitudes H for each interval. k Then, the gradient direction interval with the largest sum of gradient magnitudes at known points within the fixed source region is taken as the region where the gradient direction of the pixel to be reconstructed is located, and the interval β corresponding to the main texture direction is selected according to the perpendicular relationship between the gradient and the texture. k′ , and obtained Figure 4 The diagram shows four effective field source templates divided based on the main texture direction. The specific formula is expressed as follows:

[0085]

[0086]

[0087] In the formula, k′ represents the number of the gradient direction interval with the largest sum of gradient magnitudes of known points within the fixed source region. The corresponding effective source template is selected for reconstruction to prevent interference from known points with little impact on missing points, thereby reducing the impact of gradient direction dispersion in the neighborhood on edge reconstruction.

[0088] Step (3). Construct the field source function based on the differential approximation.

[0089] Solve for the field source function between the effective field source obtained by filtering according to the main texture direction in step (2) and the corresponding defect point;

[0090] Step (3.1). If a second-order partial derivative exists at a certain effective field source, that is, if the field source and its eight surrounding neighboring pixels are all known pixels, then the approximate differential value at the defect point can be obtained using the second-order Taylor formula with the pixel value of the effective field source. The specific formula is expressed as follows:

[0091]

[0092]

[0093]

[0094]

[0095]

[0096]

[0097] In the formula, (x,y) represents the coordinates of the missing point, (x0,y0) represents the coordinates of the effective field source, and p and q represent the normalized values ​​of the effective field source relative to the x-axis and y-axis coordinates of the missing point, respectively.

[0098] When there are missing pixels in the neighborhood of the effective source, if there is at least one known point in either the horizontal or vertical direction, a first-order Taylor expansion is used to obtain the differential approximation of the missing pixel. The specific formula is expressed as follows:

[0099]

[0100]

[0101]

[0102] Step (3.2). The similarity factor sim(r,r) is designed using the difference ratio between the Taylor expansion estimate obtained in step (3.1) and the corresponding source pixel value. i The distance factor dst(r,r) is designed based on the fact that the effective field source attenuates relative to the square of the distance during the transmission of information to the defect point. i Combining the two influencing factors, a weighted average is used to obtain the final source function G(r,r). i The specific formula is expressed as follows:

[0103]

[0104]

[0105]

[0106] In the formula, g(r) i This can avoid including defect points in the calculation, when r i When it is a missing pixel, g(r) i When r = 0, i When the number of pixels is known, g(r) i ) = 1.

[0107] Step (4). Substitute the selected effective field source and the constructed field source function into the image local texture stabilization field function in step (1) to obtain the specific image stabilization field reconstruction formula, and use the formula to complete the reconstruction of each pixel in the local defect area.

[0108] It should be noted that the above embodiments can be freely combined as needed. The above description is only a detailed explanation of the preferred embodiments and principles of the present invention, but is not intended to limit the present invention. For those skilled in the art, there will be changes in the specific implementation methods based on the ideas provided by the present invention, and these changes should also be considered within the scope of protection of the present invention.

Claims

1. A stable field image reconstruction method based on principal texture direction and differential approximation, characterized in that, The steps are as follows: Step (1). Construct the stable field equation for the local texture of the image based on the stable field model; In the mathematical physics sense, a stable field refers to a physical quantity whose properties at any point within a certain region do not change or are approximately unchanged over time. The static nature, resolvability, and stability of local image textures make it possible to combine them with stable fields. Combining relevant knowledge of image processing, we use stable fields to describe the local texture of an image and establish the stable field equation for the local texture of the image. Then, we use the Green's function method to solve it and obtain the stable field reconstruction formula for a single missing pixel. Step (2). Select an effective field source based on the main texture direction; The effective field sources in the stable field reconstruction formula for a single defective pixel obtained in step (1) are screened. First, the distance between the known point and the defective point is considered, and a field source region of fixed size centered on the defective point is selected. Then, the main texture direction in the region is obtained according to the gradient magnitude and gradient direction of each known point in the fixed field source region. The corresponding effective field source template is designed according to the main texture direction for the final reconstruction, thereby reducing the impact of gradient direction dispersion in the neighborhood of the defective point on the accurate reconstruction of the edge. Step (3). Construct the field source function based on the differential approximation; The field source function between the effective field source obtained by filtering according to the main texture direction in step (2) and the corresponding missing point is solved. First, the differential approximation value is calculated according to the Taylor expansion of the effective field source at the missing point of the corresponding order. Then, the similarity factor is defined using the differential approximation value. Next, the distance factor is designed. The field source function in the stable field reconstruction model is designed by combining the two influencing factors. Step (4). Substitute the selected effective field source and the constructed field source function into the image local texture stabilization field function in step (1) to obtain the specific image stabilization field reconstruction formula, and use the formula to complete the reconstruction of each pixel in the local defect area.

2. The stable field image reconstruction method based on principal texture direction and differential approximation according to claim 1, characterized in that, Step (1). The specific method is as follows: Step (1.1). Use the stable field model in mathematical physics to describe the local texture of the image and establish the local texture stable field equation of the image. The specific formula is expressed as follows: L·I(r)=f,(r∈D) (1) In the formula, L represents the linear differential operator, r represents the spatial coordinates of the local missing texture of the image, that is, the missing pixel, I(r) is a function describing the local texture missing region D of the image. For grayscale images, the value of I(r) represents the gray value of the corresponding spatial point; for color images, such as RGB images, it represents the value of any color channel, and f represents the field source that provides energy to the missing pixel. Step (1.2). Select the defect point r located at the boundary of the defect area and the pixel point r located in the known area. i Substituting into equation (1), the specific formula is expressed as follows: LG(r,r i )=δ(r-r i ) (2) In the formula, δ(rr) i This indicates that when only known pixel points r with unit intensity are considered... i The magnitude of the field source acting solely at the defect point r, G(r,r) i ) represents a known pixel r i The source function between the defect point r and the field source function; Equation (2) is solved using the superposition principle in the Green's function method, yielding the stable field reconstruction formula for a single defective pixel; the specific formula is expressed as follows: In the formula, n represents the total number of effective field sources around the defect point r, and I(r) i ) is r i The specific pixel value of a point.

3. The stable field image reconstruction method based on principal texture direction and differential approximation according to claim 2, characterized in that, Step (2). The specific method is as follows: Step (2.1). Considering the distance between the known point and the missing point, select a point centered on the missing point with a radius of... A neighborhood of a unit length is used as a fixed field source region; Step (2.2). Calculate the gradient magnitude m(u,v) and gradient direction θ(u,v) at each known point in the fixed source region; the specific formula is expressed as follows: In the formula, g u and g v These represent the horizontal and vertical gradients at each known point, respectively. Step (2.3). Map θ(u,v) to [0,2π] with a value of π / 4 and divide it into 4 directional intervals α1~α4 (opposite directions can be merged into the same direction); the specific formula is expressed as follows: In the formula, k represents the corresponding interval number; Step (2.4). Accumulate the gradient magnitudes m(u,v) of all known points whose gradient directions θ(u,v) are within the same interval to obtain the sum of the gradient magnitudes H for each interval. k Then, the gradient direction interval with the largest sum of gradient magnitudes at known points within the fixed source region is taken as the region where the gradient direction of the pixel to be reconstructed is located, and the interval β corresponding to the main texture direction is selected according to the perpendicular relationship between the gradient and the texture. k′ The specific formula is expressed as follows: In the formula, k′ represents the number of the gradient direction interval with the largest sum of gradient magnitudes of all known points within the fixed source region; the known points falling within this main texture direction interval are selected as the final effective source.

4. The stable field image reconstruction method based on principal texture direction and differential approximation according to claim 3, characterized in that, Step (3). The specific method is as follows: Step (3.1). If a second-order partial derivative exists at a certain effective field source, that is, when the field source and its 8 surrounding neighboring pixels are all known pixels, then the second-order Taylor formula can be used to obtain the differential approximation at the defect point using the pixel value of the effective field source; the specific formula is expressed as follows: In the formula, (x,y) represents the coordinates of the missing point, (x0,y0) represents the coordinates of the effective field source, and p and q represent the normalized values ​​of the effective field source relative to the x-axis and y-axis coordinates of the missing point, respectively. When there are missing pixels in the neighborhood of the effective source, if there is at least one known point in either the horizontal or vertical direction, the differential approximation of the missing point is obtained using a first-order Taylor expansion; the specific formula is expressed as follows: Step (3.2). The similarity factor sim(r,r) is designed using the difference ratio between the Taylor expansion estimate obtained in step (3.1) and the corresponding source pixel value. i The distance factor dst(r,r) is designed based on the fact that the effective field source attenuates relative to the square of the distance during the transmission of information to the defect point. i Combining the two influencing factors, a weighted average is used to obtain the final source function G(r,r); i The specific formula is expressed as follows: In the formula, g(r) i This can avoid including defect points in the calculation, when r i When it is a missing pixel, g(r) i When r = 0, i When the number of pixels is known, g(r) i ) = 1.

Images