Image super-resolution reconstruction method based on ridgelet

Abstract

The invention relates to the technical field of image super-resolution reconstruction, and discloses a ridgelet-based image super-resolution reconstruction method, which comprises the following steps of: modeling an input low-resolution image into a continuous function, directly predicting a pixel value from a space coordinate through a multi-layer perceptron network, and obtaining continuous function representation of the whole image; periodic features of space coordinates are extracted, and the expression ability of a periodic structure is improved through a periodic coding enhancement network of Fourier transform; analyzing the fractal dimension of the ridgelet structure of the original image, and introducing a fractal dimension retention constraint according to the fractal dimension; constructing a high-order derivative regularization term near the ridgelet; network parameters are optimized through end-to-end training, and super-resolution images of any scale are generated; according to the invention, the image is represented as a continuous function, so that the image can be sampled under any resolution, and amplification of any scale is realized without retraining the model.

Description

Technical Field

[0001] The present invention relates to the technical field of image super-resolution reconstruction, and more particularly to an image super-resolution reconstruction method based on ridgelet. Background Art

[0002] Image super-resolution reconstruction technology aims to restore high-resolution images from low-resolution images. It is widely used in medical imaging, remote sensing monitoring, video surveillance, digital content creation and other fields. With the development of artificial intelligence technologies such as deep learning, neural network-based super-resolution methods have become mainstream. However, existing technologies still face many challenges in practical application, mainly reflected in the following aspects:

[0003] On the one hand, existing super-resolution methods mostly use discrete pixel representation and are usually trained for specific magnifications (such as 2x, 4x, 8x). They lack flexible support for arbitrary magnifications. When actual needs do not match the training magnification, it is often necessary to retrain the model or use interpolation and other methods, resulting in a decrease in reconstruction effect. In addition, at high magnifications, traditional methods are prone to problems such as grid artifacts and jagged edges, which affect the visual quality of the image. On the other hand, many natural images and professional images (such as medical images, satellite remote sensing images, etc.) generally have complex periodic texture structures and self-similar fractal characteristics. When reconstructing such structures, existing methods often find it difficult to accurately restore periodic patterns, resulting in texture distortion and blurring. Or discontinuity, which is mainly due to the lack of effective modeling and constraints of periodic features in traditional methods, and the failure to fully utilize the multi-scale self-similarity and fractal geometry characteristics of images, resulting in limited reconstruction quality when processing images with complex textures such as fabrics, building surfaces, natural scenes, clouds, leaves, mountains, etc.; In addition, the existing super-resolution technology has a trade-off between maintaining the global structure of the image and local details. At high magnification, it is often difficult to take into account both the accuracy of the global structure and the richness of local details, resulting in the inability to meet high-fidelity requirements in some application scenarios. These problems have seriously restricted the promotion and application of super-resolution technology in high-quality image demand scenarios such as medical diagnosis, environmental monitoring, security monitoring, and digital content production.

[0004] Therefore, there is an urgent need for a new image super-resolution reconstruction method that can break through the limitations of existing technologies and take into account periodic structures, fractal characteristics and multi-scale detail expression, so as to meet the higher requirements of image quality and detail restoration in various high-resolution image application scenarios. Summary of the Invention

[0005] The present invention provides an image super-resolution reconstruction method based on ridgelet, which solves the technical problems in related technologies such as insufficient flexibility in supporting arbitrary scale magnification, easy generation of artifacts during high-magnification reconstruction, poor reconstruction effect of complex periodic textures and fractal structures in images, and difficulty in simultaneously maintaining the global structure and local details of the image.

[0006] The present invention provides an image super-resolution reconstruction method based on ridgelet, comprising:

[0007] The input low-resolution image is modeled as a continuous function, and the pixel values ​​are directly predicted from the spatial coordinates through a multi-layer perceptron network to obtain a continuous function representation of the entire image;

[0008] Based on the continuous function representation, the periodic features of the spatial coordinates are extracted. The network is enhanced with the periodic encoding of the Fourier transform to improve the expression of periodic structures. The periodic feature output is further used to impose ridgelet constraints to ensure that the reconstructed image can maintain the ridgelet features of the original image.

[0009] Based on continuous function representation and ridgelet constraints, the fractal dimension of the ridgelet structure of the original image is analyzed, and fractal dimension preservation constraints are introduced accordingly to ensure that the reconstructed result has the multi-scale self-similar structure characteristics of the original image;

[0010] Based on continuous function representation, ridgelet constraint and fractal dimension constraint, a high-order derivative regularization term is constructed near the ridgelet.

[0011] By integrating continuous function representation, ridgelet constraints, fractal dimension constraints and high-order derivative regularization terms, the network parameters are optimized through end-to-end training to achieve super-resolution image generation of arbitrary scale.

[0012] Furthermore, the multilayer perceptron network includes an input layer, multiple hidden layers and an output layer. The input layer receives normalized spatial coordinates, and the output layer generates pixel values ​​corresponding to the coordinate points.

[0013] Furthermore, after the introduction of periodic coding, the input layer is changed to receive the encoded feature vector, and the input dimension is changed from 2 to 2m1L dim , m1 represents the total number of frequencies, L dim Indicates the coordinate dimension.

[0014] Furthermore, the ridgelet-constrained loss function ensures that the second-order derivative of the reconstruction function in the gradient direction is close to zero, that is, the rate of change of the function along the gradient direction reaches an extreme value at the extreme point.

[0015] Furthermore, the ridge wave characteristics maintained by the ridge wave constraint have periodic consistency, and a ridge wave periodic consistency constraint needs to be applied so that the ridge wave characteristics of the reconstruction function remain consistent after a periodic offset occurs. The periodicity is determined by Fourier analysis.

[0016] Furthermore, the fractal dimension is calculated by a box counting method, and a linear regression is performed on the number of boxes of different scales required to cover the ridge wave structure in a double logarithmic coordinate system, and the slope of the regression line is obtained as an estimated value of the fractal dimension.

[0017] Furthermore, skip connections are added between adjacent hidden layers of the multilayer perceptron network so that the output of each layer is not only passed to the next layer, but also fused with features of deeper layers to form a multi-scale feature representation.

[0018] Furthermore, the high-order derivative regularization term includes two parts: the first-order derivative and the second-order derivative. The first-order derivative regularization term uses the L1 norm to promote gradient sparsity, and the second-order derivative regularization term uses the L2 norm to control the curvature of the function.

[0019] Furthermore, the training process of the multi-layer perceptron network adopts a batch processing method, and uses an importance sampling strategy to increase the sampling probability of ridge wave areas and high-frequency detail areas, so that the network pays more attention to key structural features in the image.

[0020] The present invention provides a computer storage medium comprising a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement the above-mentioned ridgelet-based image super-resolution reconstruction method.

[0021] The beneficial effects of the present invention are as follows: by representing an image as a continuous function, the method can sample the image at any resolution and achieve any scale magnification without retraining the model;

[0022] By introducing periodic coding and ridgelet periodic consistency constraints, the reconstruction quality of complex periodic texture structures is improved. When processing images containing regular textures, the accuracy of periodic texture structures is improved, the texture continuity is enhanced, and the limitations of traditional methods in processing periodic structures are effectively overcome.

[0023] The multi-scale self-similar structural characteristics of the image are effectively maintained through fractal dimension constraints. When processing natural images with complex fractal features, the structural similarity index is improved, the visual realism score is enhanced, and the reconstructed image maintains structural complexity and texture richness similar to the original image at different scales. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 This is a flow chart of a ridgelet-based image super-resolution reconstruction method in the present invention;

[0025] Figure 2 is a histogram comparing the image reconstruction quality results of different methods in the present invention;

[0026] Figure 3 is a graph showing the relationship between magnification and reconstruction quality in the present invention;

[0027] Figure 4 is a graph showing the relationship between computing efficiency and storage requirements in the present invention;

[0028] Figure 5 It is a radar chart of the technical effect evaluation results of various application scenarios in the present invention;

[0029] Figure 6 It is a line graph showing the influence of different constraint item weights on the reconstruction quality in the present invention. DETAILED DESCRIPTION

[0030] The subject matter described herein will now be discussed with reference to example embodiments. It should be understood that these embodiments are discussed solely to enable those skilled in the art to better understand and implement the subject matter described herein, and that the functions and arrangements of the elements discussed may be varied without departing from the scope of this specification. Various examples may omit, substitute, or add various processes or components as needed. Furthermore, features described in some examples may be combined in other examples.

[0031] At least one embodiment of the present invention discloses a ridgelet-based image super-resolution reconstruction method, such as Figure 1 Shown, including:

[0032] Step 1: Model the input low-resolution image as a continuous function, and directly predict the pixel value from the spatial coordinates through a multi-layer perceptron network to obtain a continuous function representation of the entire image;

[0033] This step is the foundation for all subsequent steps. By representing the image as a continuous function, it provides the mathematical foundation for subsequent operations such as periodic encoding, ridgelet constraints, and fractal dimension preservation. The continuous function representation established in this step enables subsequent steps to sample and process the image at arbitrary resolution.

[0034] In this step, according to an embodiment of the present application, an implicit neural representation model is applied to model the image as a continuous function, and pixel values ​​are directly predicted from spatial coordinates.

[0035] The specific implementation is as follows:

[0036] Step 1.1, build a multi-layer perceptron network;

[0037] The Multi-Layer Perceptron (MLP) network is used to learn the mapping function from two-dimensional space coordinates (x, y) to pixel values:

[0038] F:R2 →R c ;

[0039] Where F represents the mapping function, R 2 represents the two-dimensional real space, R c Represents a c-dimensional real space (i.e., the output is the pixel value of c channels), where c is the number of channels of the image (usually 3 for color images, corresponding to the three RGB channels).

[0040] The multilayer perceptron network is a feedforward neural network consisting of multiple fully connected layers. In the embodiment of the present application, the detailed structure of the MLP network includes:

[0041] Input layer: receives normalized spatial coordinates (x, y), with 2 neurons, where x and y represent the normalized horizontal and vertical coordinates respectively;

[0042] Hidden layer 1: fully connected layer with 256 neurons and ReLU activation function;

[0043] Hidden layer 2: fully connected layer with 256 neurons and ReLU activation function;

[0044] Hidden layer 3: fully connected layer with 256 neurons and ReLU activation function;

[0045] Hidden layer 4: fully connected layer with 256 neurons and ReLU activation function;

[0046] The mathematical expression of the ReLU activation function is:

[0047] ReLU(x1)=max(0,x1);

[0048] Where ReLU(x1) represents the output value of the ReLU activation function, x1 is the input value, and max(0,x1) means taking the larger value between 0 and x1.

[0049] Output layer: fully connected layer, the number of neurons is c (3 for color images), generating pixel values ​​of corresponding coordinate points.

[0050] Each fully connected layer performs the following operations:

[0051]

[0052] in represents the output of layer i1, represents the output of the i1-1th layer, is the weight matrix connecting the i1-1th layer and the i1th layer, is the bias vector of the i1-1th layer, and σ is the activation function.

[0053] Step 1.2, extract training data;

[0054] For the input low-resolution image, first extract a set of coordinate pixel value pairs as training data:

[0055]

[0056] in Represents the i2th coordinate point in the image, Indicates that the low-resolution image is The actual pixel value at , N1 is the number of sampling points.

[0057] In medical imaging applications, such as low-resolution MRI scan image processing, the image coordinates are first normalized to (0, 1). 2 Interval, where (0, 1) 2 Representing a two-dimensional normalized coordinate space, we then sample coordinate points on a uniform grid, where each coordinate point corresponds to a pixel value. For a 256×256 MRI image, we can sample 64,000 points as training data.

[0058] Step 1.3, train the MLP network by minimizing the loss function:

[0059]

[0060] Among them L pixel represents the pixel-level loss function, N1 is the number of sampling points, represents the sum of i2 from 1 to N1, Represents network coordinates The predicted value of Indicates that the low-resolution image is The actual pixel value at Represents the squared error between the predicted value and the true pixel value.

[0061] Through the above steps, we can obtain an implicit neural representation model that represents the image as a continuous function. Unlike the traditional discrete pixel representation, this continuous function representation allows the image to be sampled at arbitrary resolution, laying the foundation for achieving super-resolution reconstruction at any scale.

[0062] Step 2: Based on the continuous function representation, the periodic features of the spatial coordinates are extracted. The network is enhanced with a periodic encoding of the Fourier transform to improve the expression of the periodic structure. The periodic feature output is further used to impose ridgelet constraints to ensure that the reconstructed image can maintain the ridgelet features of the original image.

[0063] This step builds on the continuous function representation established in Step 1. By introducing periodic encoding, this step enhances the network's ability to express periodic structures and constructs ridgelet constraints to ensure that the reconstructed image retains the ridgelet characteristics of the original image. The continuous function representation in Step 1 provides a differentiable mathematical foundation for the periodic encoding and ridgelet constraints in this step.

[0064] In this step, according to one embodiment of the present application, periodic coding is introduced to enhance the network's ability to express periodic structures, and ridgelet constraints are constructed to ensure that the reconstructed image maintains the ridgelet characteristics of the original image.

[0065] The specific implementation is as follows:

[0066] Step 2.1, construct a periodic encoding function based on Fourier transform;

[0067] The periodic encoding function based on Fourier transform is expressed as:

[0068]

[0069] Where γ represents the periodic encoding function, R 2 represents the two-dimensional real space, → represents the mapping relationship, Indicates 2ml dim dimensional real space (ie the output is 2m1L dim dimensional feature vector), m1 represents the number of frequencies, L dim Indicates the coordinate dimension.

[0070] Map the input coordinates (x, y) to a high-dimensional feature vector:

[0071]

[0072] Where γ(x, y) represents the eigenvector after receiving the encoding of the normalized spatial coordinates (x, y), sin represents the sine function, cos represents the cosine function, 2π represents the radian value of the complete cycle, f1, They represent the 1st frequency and the m1th frequency respectively, x and y represent the horizontal and vertical coordinates in the normalized spatial coordinates respectively, and m1 represents the total number of frequencies.

[0073] The l1th frequency value is expressed as:

[0074]

[0075] in Represents the l1th frequency value, f0 is the fundamental frequency, and m1 represents the total number of frequencies.

[0076] It should be noted that this encoding method can map low-dimensional coordinates to high-dimensional space, enhancing the network's ability to express complex periodic patterns.

[0077] In practical applications, m1 = 10 and f0 = 1 can be selected, generating a feature vector of length 2 × 10 × 2 = 40. This periodic encoding is particularly suitable for processing images containing repetitive textures, such as fabrics, architectural surfaces, or medical images with periodic structures (e.g., CT scans of muscle tissue). For example, traditional methods struggle to accurately reconstruct the repeating pattern in fabric texture images. However, this application enhances the reconstruction quality by using Fourier features.

[0078] Step 2.2, modify the network structure;

[0079] The input layer is changed to receive the encoded feature vector γ(x, y), and the input dimension is changed from 2 to 2m1L dim ,The rest of the network structure remains unchanged;

[0080] The modified multi-layer perceptron network structure is as follows:

[0081] Input layer: receives the encoded feature vector γ(x, y), the number of neurons is 2m1L dim (When m1=10,L dim =2, 40);

[0082] Hidden layer 1: fully connected layer with 256 neurons and ReLU activation function;

[0083] Hidden layer 2: fully connected layer with 256 neurons and ReLU activation function;

[0084] Hidden layer 3: fully connected layer with 256 neurons and ReLU activation function;

[0085] Hidden layer 4: fully connected layer with 256 neurons and ReLU activation function;

[0086] Output layer: fully connected layer with c neurons (3 for color images) and no activation function.

[0087] Step 2.3, construct the ridgelet constraint loss function;

[0088] The ridgelet constraint loss function ensures that the second-order derivative of the reconstruction function in the gradient direction is close to zero, that is, the rate of change of the function along the gradient direction reaches the extreme value at the extreme point, and the expression is:

[0089]

[0090] Among them L ridge represents the ridgelet constraint loss function, Represents the Hessian matrix (second-order derivative matrix) of the mapping function F at the normalized spatial coordinates (x, y), represents the gradient vector (first-order derivative vector) of the mapping function F at the normalized spatial coordinate (x, y), ||·|| 2 Represents the square of the L2 norm.

[0091] In the specific implementation, the finite difference method can be used to approximate the calculation of the gradient and Hessian matrix; for the gradient You can use the central difference formula:

[0092]

[0093] in represents the partial derivative of the function F with respect to the variable X, ≈ represents approximately equal, F(x+h1,y) represents the function value at the coordinate (x+h1,y), F(x-h1,y) represents the function value at the coordinate (x-h1,y), h1 represents the step size of the finite difference, and 2h1 represents the total difference interval;

[0094]

[0095] in represents the partial derivative of the function F with respect to the variable y, F(x, y+h1) represents the function value at the coordinates (x, y+h1), and F(x, y-h1) represents the function value at the coordinates (x, y-h1).

[0096] Step 2.4, detect texture periodicity in the original image and construct ridgelet periodicity consistency constraints;

[0097] This step ensures that the ridge wave characteristics of the reconstructed function remain consistent after the periodic shift. The ridge wave periodic consistency constraint is expressed as:

[0098]

[0099] Among them L period represents the ridgelet periodic consistency constraint loss function, and Represent the Hessian matrix of the mapping function F at the coordinates (x+p1, y) and (x, y+p1), represents the Hessian matrix of the mapping function F at the normalized spatial coordinate (x, y), p1 represents the detected texture period length, Represents the square of the L2 norm.

[0100] In satellite image processing applications, such as farmland monitoring, crops often exhibit a regularly arranged texture structure. When applying this method, the original image's spectrum is first analyzed through Fourier transform to find the spatial frequency corresponding to the energy peak and determine the texture period p1. Ridgelet periodicity consistency constraints are then applied to ensure that the reconstructed high-resolution satellite image retains the regularly arranged structure of the farmland, effectively improving the accuracy of object recognition and change detection.

[0101] It can be seen that by introducing periodic coding and ridgelet constraints, the network can more accurately express and reconstruct the periodic structure and ridgelet features in the image, thereby improving the quality and detail fidelity of the reconstructed image.

[0102] Step 3: Based on the continuous function representation and ridgelet constraints, the fractal dimension of the ridgelet structure of the original image is analyzed, and a fractal dimension preservation constraint is introduced accordingly to ensure that the reconstructed result has the multi-scale self-similar structure characteristics of the original image;

[0103] This step, based on the results of steps 1 and 2, introduces a fractal dimension-preserving constraint to ensure that the reconstructed image retains the multiscale self-similar structure of the original image. The continuous function representation in step 1 and the ridgelet constraint in step 2 provide the mathematical tools for calculating the fractal dimension in this step.

[0104] In this step, according to an embodiment of the present application, a fractal dimension preservation constraint is introduced to ensure that the reconstructed image maintains the multi-scale self-similar structural characteristics of the original image.

[0105] The specific implementation is as follows:

[0106] Step 3.1, calculate the fractal dimension of the ridgelet structure of the original image;

[0107] First, extract the ridgelet structure R of the original image HR , which can be obtained by calculating the local maximum point in the direction of the eigenvector of the second-order derivative matrix of the image;

[0108] Then, the box-counting method is used to calculate the fractal dimension of the ridge wave structure:

[0109]

[0110] Among them D f Represents the fractal dimension, lim represents the limit operation, ε1 represents the side length scale of the box, → represents tending to, log represents the logarithmic function (usually the natural logarithm), and N2(ε1) represents the number of boxes with a side length of ε1 required to cover the ridge wave structure.

[0111] In actual calculation, by Calculate Then and Perform linear regression and obtain the slope of the regression line as the estimated value of the fractal dimension.

[0112] In actual implementation, the box counting method is performed as follows:

[0113] Binarize the image ridge wave structure to obtain a binary image B;

[0114] Use grids of different sizes (such as pixels) covers the binary image;

[0115] For each grid size Count the number of boxes containing ridge wave structures

[0116] In the double logarithmic coordinate system, plot and The relationship between , linear regression;

[0117] The slope of the linear regression is the fractal dimension D f estimated value.

[0118] In natural landscape image processing, images such as mountains, rivers, and clouds exhibit rich fractal properties. For example, for an aerial image of a mountain range with a resolution of 512×512, the fractal dimension of its ridge structure calculated using the above method typically yields a value of approximately 1.6-1.8, indicating high self-similarity. By maintaining the fractal dimension, the reconstruction method employed by this application can retain the natural texture characteristics of the mountain edge even after a 10x magnification.

[0119] Step 3.2, construct the fractal dimension preservation constraint loss function;

[0120] The fractal dimension preservation constraint loss function is expressed as:

[0121]

[0122] Among them L fractal represents the fractal dimension preservation constraint loss function, |.| represents the absolute value operation, Represents the fractal dimension of the ridge wave structure of the reconstructed image (super-resolution image), Represents the fractal dimension of the ridgelet structure of the original high-resolution image.

[0123] This constraint ensures that the reconstructed image remains consistent with the original image in terms of complexity and self-similarity.

[0124] Calculate L fractal When the ridge wave structure of the reconstructed image is extracted regularly and its fractal dimension is calculated Due to the high computational cost of extracting ridge wave structure and calculating fractal dimension, it can be updated once at the end of each training cycle. values ​​instead of recalculating them on every gradient update.

[0125] Step 3.3, introduce the recursive connection structure into the MLP network;

[0126] In order to achieve consistent representation of fractal characteristics at different scales, skip connections are added between adjacent hidden layers so that the output of each layer is not only passed to the next layer, but also fused with the features of deeper layers to form a multi-scale feature representation. The mathematical expression is:

[0127]

[0128] in represents the output of the (i4+1)th layer, σ represents the activation function, represents the weight matrix connecting the i4th layer and the (i4+1)th layer, represents the output of layer i4, represents the bias vector of the (i4+1)th layer, Represents the parameter controlling the skip connection strength in layer i4.

[0129] In the specific implementation, the strength parameter of the jump connection is It can be set as a learnable parameter with an initial value of 0.1, which is adjusted adaptively during the training process. The modified network structure is as follows:

[0130] Input layer: receives the encoded feature vector γ(x, y), the number of neurons is 2m1L dim ;

[0131] Hidden layer 1: fully connected layer with 256 neurons and ReLU activation function;

[0132] Hidden layer 2: Fully connected layer with 256 neurons, using ReLU activation function, and receiving skip connections from hidden layer 1;

[0133] Hidden layer 3: Fully connected layer with 256 neurons, using ReLU activation function, and receiving skip connections from hidden layer 2;

[0134] Hidden layer 4: Fully connected layer with 256 neurons, using ReLU activation function, and receiving skip connections from hidden layer 3;

[0135] Output layer: fully connected layer, the number of neurons is c, and no activation function is used.

[0136] In medical image analysis, such as super-resolution reconstruction of lung CT scans, the structures of lung blood vessels and bronchi exhibit distinct fractal properties. By preserving the fractal dimensions of these structures, the proposed method can more accurately restore subtle lung structures, helping doctors identify early-stage lesions.

[0137] By preserving the fractal dimension constraint, we can ensure that the reconstructed image maintains similar complexity and structural properties to the original image at different scales, which is particularly suitable for processing natural images with rich textures and self-similar structures.

[0138] Step 4: Based on the continuous function representation, ridgelet constraint and fractal dimension constraint, a high-order derivative regularization term is constructed near the ridgelet.

[0139] This step builds on the results of the previous three steps by introducing a higher-order derivative regularization term to control the smoothness and sharpness of the function near the ridgelets. The continuous function representation in step 1, the ridgelet constraint in step 2, and the fractal dimension constraint in step 3 together provide the mathematical foundation for the calculation of higher-order derivatives in this step.

[0140] In this step, according to one embodiment of the present application, a high-order derivative regularization term is introduced to control the smoothness and sharpness of the function near the ridge wave to avoid overfitting and artifact generation.

[0141] The specific implementation is as follows:

[0142] Step 4.1, construct a regularization term based on function derivatives;

[0143] The regularization term based on function derivatives consists of two parts: the first-order derivative and the second-order derivative, which can be expressed as:

[0144]

[0145] Among them L reg represents the high-order derivative regularization loss function, represents the weight coefficient of the first-order derivative term, represents the L1 norm of the gradient of the mapping function F, represents the weight coefficient of the second-order derivative term, The square of the L2 norm of the Hessian matrix representing the mapping function F; represents the gradient of the mapping function F, represents the Hessian matrix of the mapping function F, and ||·||1 represents the L1 norm.

[0146] According to the embodiment of the present application, the weight coefficient and Adjustable according to image type:

[0147] For natural landscape images: To keep the edges sharp while preserving the natural texture;

[0148] For medical images: to reduce noise while preserving vital structures;

[0149] For document images: To enhance the sharpness of text edges.

[0150] Step 4.2, maintain the edge sharpness of the image;

[0151] First-order derivative regularization term The L1 norm is used to promote gradient sparsity, which is beneficial to maintaining the edge sharpness of the image; its calculation method is:

[0152]

[0153] in represents the L1 norm of the gradient of the mapping function F, Σ represents the summation symbol, represents the square root operation, represents the partial derivative of the mapping function F with respect to the horizontal coordinate x in the normalized spatial coordinates, Represents the partial derivative of the mapping function F with respect to the vertical coordinate y in the normalized spatial coordinates.

[0154] Calculate the partial derivative in the above formula ( and ), you can use step 2.3 to calculate the gradient In the central difference approximation method described above, the step size h1 can also be set to, for example, 0.001.

[0155] Step 4.3, prevent excessive oscillation;

[0156] Second-order derivative regularization term The L2 norm is used to control the curvature of the function and prevent excessive oscillation; its calculation method is:

[0157]

[0158] in represents the square of the L2 norm of the Hessian matrix of the mapping function F, ∑ represents the summation symbol, represents the second-order partial derivative of the mapping function F with respect to the horizontal coordinate x in the normalized spatial coordinates, represents the mixed partial derivative of the mapping function F with respect to the horizontal coordinate x and the vertical coordinate y in the normalized spatial coordinates, Represents the second-order partial derivative of the mapping function F with respect to the vertical coordinate y in the normalized spatial coordinates.

[0159] In actual calculations, the finite difference method can also be used to approximate the second-order partial derivatives:

[0160]

[0161] in Represents the second-order partial derivative of the mapping function F with respect to x at the normalized spatial coordinates (x, y), F(x+h2, y) represents the value of the mapping function F at the point (x+h2, y), F(x, y) represents the value of the mapping function F at the normalized spatial coordinates (x, y), F(x-h2, y) represents the value of the mapping function F at the point (x-h2, y), 2 represents the coefficient, Represents the square of the step size.

[0162]

[0163] in represents the second-order partial derivative of the mapping function F with respect to y at the normalized spatial coordinates (x, y), F(x, y+h2) represents the value of the mapping function F at the point (x, y+h2), and F(x, y-h2) represents the value of the mapping function F at the point (x, y-h2).

[0164]

[0165] in Represents the mixed partial derivative of the mapping function F with respect to x and y at the normalized spatial coordinates (x, y), F(x+h2, y+h2) represents the value of the mapping function F at the point (x+h2, y+h2), F(x+h2, y-h2) represents the value of the mapping function F at the point (x+h2, y-h2), F(x-h2, y+h2) represents the value of the mapping function F at the point (x-h2, y+h2), F(x-h2, y-h2) represents the value of the mapping function F at the point (x-h2, y+h2), F(x-h2, y-h2) represents the value of the mapping function F at the point (x-h2, y-h2), Indicates 4 times the square of the step size.

[0166] Step 4.4, control the smoothness and sharpness of the reconstructed image;

[0167] By adjusting the weight coefficient and The value of can control the smoothness and sharpness of the reconstructed image. It should be noted that a larger The value will enhance edge sharpness but may introduce noise. A value of 0 will increase smoothness but may cause loss of details. In practical applications, the appropriate weight value is usually selected according to the specific image type and requirements.

[0168] In video surveillance applications, such as license plate recognition, it is often necessary to extract and magnify the license plate area from low-resolution surveillance video. When applying this method, high-order derivative regularization can effectively eliminate aliasing and blurring artifacts during the magnification process while maintaining the sharp edges of the license plate characters, improving the accuracy of subsequent character recognition.

[0169] In addition, high-order derivative regularization makes the reconstruction function smooth and continuous, effectively suppressing common super-resolution artifacts such as jagged edges, ringing effects, and noise amplification.

[0170] Step 5: Integrate continuous function representation, ridgelet constraint, fractal dimension constraint, and high-order derivative regularization term to optimize network parameters through end-to-end training to achieve super-resolution image generation of arbitrary scale;

[0171] This step integrates all the constraints and optimization objectives of the previous four steps, optimizing the network parameters through end-to-end training. The constraints and loss functions established in steps 1-4 are unified and integrated in this step to form a complete optimization objective.

[0172] In this step, according to an embodiment of the present application, the aforementioned constraints are combined into a comprehensive loss function, and the network parameters are optimized through end-to-end training.

[0173] The specific implementation is as follows:

[0174] Step 5.1, construct a comprehensive loss function;

[0175] Integrating pixel reconstruction loss, ridgelet constraint loss, periodic consistency loss, fractal dimension loss and regularization loss, the comprehensive loss function is expressed as:

[0176] L total =L pixel +α2·L ridge +β2·L period +γ frac ·L fractal +δ2·L reg ;

[0177] Among them L total Represents the comprehensive loss function, L pixel represents the pixel reconstruction loss, α2 represents the weight coefficient of the ridgelet constraint loss, L ridge represents the ridgelet constraint loss, β2 represents the weight coefficient of the periodic consistency loss, L period represents the periodic consistency loss, γ frac Represents the weight coefficient of fractal dimension loss, L fractal represents the fractal dimension loss, δ2 represents the weight coefficient of regularization loss, L reg represents the regularization loss.

[0178] According to the embodiment of the present application, the reference values ​​of the weight coefficients are as follows:

[0179] α2=0.1: controls the strength of the ridgelet constraint;

[0180] β2=0.05: controls the strength of the periodic consistency constraint;

[0181] γ frac =0.2: controls the strength of the fractal dimension constraint;

[0182] δ2=0.15: controls the strength of high-order derivative regularization.

[0183] These parameters can be adjusted according to the specific application scenario and image characteristics. For example, for images with obvious periodic structures, the β2 value can be appropriately increased; for natural images with rich self-similar structures, the γ value can be appropriately increased. frac value.

[0184] Step 5.2: Use Adam optimizer to train the network.

[0185] It should be noted that the Adam optimizer combines the advantages of the momentum method and the adaptive learning rate method, and is suitable for handling optimization problems containing noise and sparse gradients. The learning rate is initially set to 0.0001 and gradually decays using a cosine annealing strategy:

[0186]

[0187] in represents the learning rate at step t2, η min represents the minimum learning rate, η max represents the maximum learning rate, cos represents the cosine function, π represents the pi ratio, t2 represents the current training step number, T2 represents the total training step number, represents the cosine annealing factor.

[0188] In actual implementation, η can be selected min =10 -6 , η max =10 -4 , the total number of training steps T2 is set to 100,000 steps. In addition, a weight decay coefficient of 10 -5 The L2 regularization prevents overfitting.

[0189] Step 5.3, batch processing of training process;

[0190] According to one embodiment of the present application, the training process adopts a batch processing method, and each batch randomly samples a group of coordinate points for forward calculation and back propagation. To accelerate convergence, an importance sampling strategy is adopted to increase the sampling probability of ridge wave areas and high-frequency detail areas.

[0191] This sampling strategy enables the network to pay more attention to the key structural features in the image and improve the reconstruction quality.

[0192] Step 5.4, regularly evaluate model performance;

[0193] During training, model performance is regularly evaluated on the validation set, including objective metrics such as peak signal-to-noise ratio and structural similarity. When performance metrics on the validation set stop improving for multiple consecutive cycles, training is terminated using an early stopping strategy to prevent overfitting.

[0194] According to an embodiment of the present application, the model performance can be evaluated on the validation set every 1,000 steps. If the PSNR index does not improve after 10 consecutive evaluations, the training is terminated early. In addition, in order to comprehensively evaluate the image quality, a weighted index can be used as the basis for early stopping:

[0195] Score = PSNR + 20·SSIM;

[0196] Among them, Score represents the comprehensive evaluation index, PSNR represents the peak signal-to-noise ratio, and SSIM represents the structural similarity.

[0197] Step 5.5, generating a reconstructed image of corresponding resolution;

[0198] After training is completed, the continuous function can be sampled at any resolution to generate a reconstructed image of the corresponding resolution;

[0199] For the super-resolution task with a magnification of S, the sampling point coordinates are:

[0200] (i5 / sW,j5 / sH)|i5=0,1,...,sW-1;j5=0,1,...,sH-1;

[0201] Where (i5 / sW, j5 / sH) represents the normalized sampling coordinates, i5 represents the pixel index in the horizontal direction, j5 represents the pixel index in the vertical direction, s represents the magnification, W represents the width of the original image, H represents the height of the original image, and | represents "satisfying the condition".

[0202] This method offers advantages in digital content creation applications, such as reconstructing low-resolution artwork into high-resolution images suitable for large-scale printing. By jointly optimizing multiple constraints, it maintains the overall style of the artwork while preserving detail fidelity, avoiding the oversmoothing or false detail issues common in traditional methods. In a user experience test, compared to multiple super-resolution methods, this method achieved the highest user satisfaction in visual quality scores.

[0203] As can be seen, through joint optimization training, the constraints work together to ensure that the reconstructed image maintains the details and structure of the original image while maintaining a natural and realistic visual effect. Furthermore, thanks to the continuous function representation and multiple constraints, this method can generate high-quality super-resolution images at arbitrary magnifications.

[0204] like Figures 2 to 6 As shown, the image reconstruction quality comparison results of different methods; the relationship between magnification and reconstruction quality; the relationship between computational efficiency and storage requirements; the evaluation results of technical effects in various application scenarios; and the impact of different constraint weights on reconstruction quality.

[0205] A computer storage medium includes a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement the above-mentioned image super-resolution reconstruction method based on ridgelet.

[0206] Here, the present invention provides an implementation example:

[0207] In the field of medical imaging, improving the quality of low-resolution scanned images is crucial for disease diagnosis. This embodiment is applied to super-resolution reconstruction of lung CT scan images to improve the clarity and detail fidelity of lung blood vessels and bronchial structures.

[0208] This application uses a set of low-dose lung CT scan images with a resolution of 256×256 pixels as input data. These low-resolution images are acquired to reduce radiation dose, making it difficult to clearly identify subtle lung structures. The method of this application is performed as follows:

[0209] First, the low-resolution CT images were normalized and preprocessed to map pixel values ​​to the interval [0, 1]. Coordinate-pixel value pairs were extracted from the images as training data. Then, an MLP network with four hidden layers (256 neurons per layer) was constructed, and periodic encoding was introduced to map the input coordinates into a 40-dimensional feature vector.

[0210] Because the pulmonary vascular and bronchial structures exhibit distinct fractal properties, this application places particular emphasis on preserving the fractal dimension. Using the box counting method to calculate the fractal dimension of the ridge wave structure of the original image, the fractal dimension of the pulmonary vascular network is approximately 1.7, indicating high self-similarity. Based on this characteristic, the weight coefficient for the fractal dimension constraint in the loss function is set to 0.25, higher than the standard setting (0.2), to better preserve the self-similarity of these structures.

[0211] According to the characteristics of medical images, the weight coefficient of high-order derivative regularization is set to and To reduce noise while preserving important structures, during training, the sampling probability of the pulmonary blood vessels and bronchial regions was increased to 60% (the standard is 50%), allowing the network to pay more attention to these key structures.

[0212] After the model training is completed, the input 256×256 low-resolution CT image is enlarged 4 times to generate a 1024×1024 high-resolution image. The entire reconstruction process takes only 2.5 seconds (on a standard GPU workstation).

[0213] Comparing the results of this method with those of traditional bicubic interpolation methods and standard deep learning super-resolution methods, this method performs well in the following indicators:

[0214] Structural fidelity: The fractal dimension of the reconstructed pulmonary vascular network differed by only 0.05 from that of standard high-resolution CT scans, while the difference reached 0.24 using conventional methods, indicating that this method better preserved the self-similarity of the lung structure.

[0215] Detail clarity: The method's visibility of tiny lung structures (blood vessels <1 mm in diameter) was scored 4.2 out of 5, 50% higher than the 2.8 of conventional methods, which is crucial for detecting early lesions.

[0216] Radiologists' subjective evaluation: Five senior radiologists conducted a blind evaluation of the reconstructed images. The average clarity score obtained by this method was 4.3 (out of 5), while the traditional method was 3.1, which increased diagnostic confidence by 38%.

[0217] Lesion detection rate: Using images reconstructed by this method, doctors were able to identify 92% of early-stage lung nodules, while using traditional methods they could only identify 76%, improving the accuracy of disease diagnosis.

[0218] Through the above verification, this method shows advantages in super-resolution reconstruction of medical images, especially in maintaining the fractal structure of complex vascular networks and improving the visibility of tiny structures, providing more reliable image support for clinical diagnosis.

[0219] Here, the present invention provides an implementation example:

[0220] Remote sensing monitoring is of great significance for environmental change analysis and land use planning, but is limited by satellite sensor resolution and transmission bandwidth. This embodiment is applied to super-resolution reconstruction of remote sensing images for agricultural monitoring, improving the accuracy of identifying crop distribution and growth status.

[0221] This application uses a dataset of multispectral satellite images with a resolution of 512×512 pixels, encompassing both visible and near-infrared bands. These images contain distinct periodic farmland structures and fractal features of natural landforms.

[0222] This application specifically enhances periodic encoding and ridgelet constraints to address the unique characteristics of remote sensing imagery. First, Fourier analysis is used to identify the primary periodic patterns in the image. Farmland areas typically exhibit a regularly arranged period of 20-30 pixels. Based on this characteristic, the base frequency f0 is set to 0.8 (standard: 1.0) in the periodic encoding to better match the periodic structure of farmland. Furthermore, the weight coefficient β of the periodic consistency constraint in the loss function is increased to 0.08 (standard: 0.05).

[0223] At the same time, for natural landform areas such as mountains and rivers, the fractal dimension of their ridge wave structure is about 1.6-1.8. While maintaining the fractal dimension constraint, an adaptive weight strategy is adopted for different areas: γ is 0.25 for natural landform areas and γ is 0.15 for farmland areas.

[0224] Since remote sensing images contain both regular artificial structures and complex natural textures, the high-order derivative regularization parameter is set to and To keep the edges sharp while preserving the natural texture. Model training uses regional importance sampling to increase the sampling probability of the boundaries of objects and areas with complex textures.

[0225] After training, the input 512×512 low-resolution remote sensing image is magnified 5 times to generate a 2560×2560 high-resolution image, with a processing time of approximately 3.8 seconds.

[0226] This method is compared with the existing remote sensing image super-resolution method, and the technical effect is verified as follows:

[0227] Periodic structure fidelity: In areas with regular farmland, the accuracy of the periodic structure reconstructed by this method reached 93%, while the traditional method only had 75%, indicating that this method better restored the regular arrangement pattern of the farmland.

[0228] Boundary clarity: This method improves the sharpness of boundary reconstruction by 42%, especially in the transition area between farmland and natural landforms, with boundary positioning accuracy improved to sub-pixel level (average deviation <0.5 pixel).

[0229] Land feature classification accuracy: Based on the reconstructed imagery for land cover classification, this method achieved a classification accuracy of 87%, a 14 percentage point improvement over traditional methods, improving the performance of downstream analysis tasks.

[0230] Change detection sensitivity: In crop growth status monitoring, this method can detect abnormal growth areas with an area of ​​only 2×2 original pixels, which is 4 times higher than the detection capability of traditional methods (4×4 pixels), providing important support for early crop disease and pest monitoring.

[0231] These results indicate that the proposed method has advantages in super-resolution reconstruction of remote sensing images, especially its ability to preserve periodic structures and fractal features, providing higher quality image data for agricultural monitoring and environmental analysis.

[0232] Here, the present invention provides an implementation example:

[0233] In the field of video surveillance, surveillance videos are typically captured and stored at lower resolutions due to storage and transmission bandwidth limitations. This embodiment is applied to super-resolution reconstruction of key areas in urban surveillance videos, improving the recognition rate of important features such as license plates and faces.

[0234] This application selected surveillance video frames with a resolution of 720×480 pixels, focusing on scenes containing vehicles and pedestrians. Unlike medical and remote sensing applications, surveillance video super-resolution places greater emphasis on edge clarity and text readability.

[0235] In view of the characteristics of monitoring scenarios, this application has specially optimized the high-order derivative regularization part. For text areas such as license plates and logos, set and (higher than the standard value 0.1) and (lower than the standard value 0.005), enhance edge sharpness; for areas that require natural transitions such as faces, use and Parameter configuration.

[0236] Because key areas in surveillance videos, such as license plates, are typically small (only about 20 × 10 pixels at their original resolution), this application employs a two-stage processing strategy: first, an object detection algorithm is used to locate the key areas. Then, this method is applied to these areas for 8x super-resolution reconstruction, generating high-definition regional images of 160 × 80 pixels.

[0237] During the training process, the sampling ratio of text edges and high-contrast areas was increased to 65%, and an additional ridge constraint weight (α2 = 0.15, higher than the standard value 0.1) was added to the license plate character area to better maintain the clarity of the character edges.

[0238] It takes an average of only 0.8 seconds to process the key area in a 720×480 video frame, meeting near real-time processing requirements.

[0239] This method is compared with a super-resolution method specifically used for surveillance videos, and the results are verified as follows:

[0240] License plate character recognition rate: Using this method to reconstruct license plate images, the automatic recognition system's character recognition accuracy reaches 94%, while the traditional method is only 78%, an improvement of 20.5%, which is crucial for automatic traffic violation recognition systems;

[0241] Facial feature point positioning accuracy: In facial images reconstructed using this method, the average positioning error of 68 key feature points was reduced by 35%, and the facial feature fidelity score increased by 0.6 points (on a 5-point scale), effectively improving the reliability of identity confirmation.

[0242] Edge fidelity: In high-contrast edge regions, the average edge localization deviation of our method is 0.4 pixels, compared to 1.2 pixels for traditional methods, improving accuracy by 66.7%. This improvement is particularly noticeable in scenes with low light conditions.

[0243] Storage efficiency: Compared to directly storing high-resolution video, this method only needs to store the low-resolution original video and the coordinate information of the key areas, reducing storage requirements by 86% while maintaining the high-definition reconstruction capability of the key areas;

[0244] Through the above verification, this method performs well in video surveillance super-resolution applications, especially in maintaining text clarity and edge accuracy, providing practical technical solutions for fields such as security monitoring and traffic management.

[0245] In summary, the three implementation examples above fully demonstrate the adaptability and superiority of this method in different scenarios. By adjusting the weights and parameter configurations of different constraints, this method can be optimized for the characteristics of different application scenarios. It achieves technical results in maintaining fractal characteristics, periodic structure, and edge clarity, verifying its wide value in practical applications.

[0246] The above describes an embodiment of the present invention, but this embodiment is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Ordinary technicians in this field can also make more forms of equivalent embodiments based on the inspiration of this embodiment, all of which are protected by this embodiment.

Claims

1. A ridgelet-based image super-resolution reconstruction method, characterized in that: include: The input low-resolution image is modeled as a continuous function, and the pixel values ​​are directly predicted from the spatial coordinates through a multi-layer perceptron network to obtain a continuous function representation of the entire image; Based on the continuous function representation, the periodic features of the spatial coordinates are extracted. The network is enhanced with the periodic encoding of the Fourier transform to improve the expression of periodic structures. The periodic feature output is further used to impose ridgelet constraints to ensure that the reconstructed image can maintain the ridgelet features of the original image. Based on continuous function representation and ridgelet constraints, the fractal dimension of the ridgelet structure of the original image is analyzed, and fractal dimension preservation constraints are introduced accordingly to ensure that the reconstructed result has the multi-scale self-similar structure characteristics of the original image; Based on continuous function representation, ridgelet constraint and fractal dimension constraint, a high-order derivative regularization term is constructed near the ridgelet. By integrating continuous function representation, ridgelet constraints, fractal dimension constraints and high-order derivative regularization terms, the network parameters are optimized through end-to-end training to achieve super-resolution image generation of arbitrary scale.

2. The image super-resolution reconstruction method based on ridgelet according to claim 1, characterized in that: The multi-layer perceptron network includes an input layer, multiple hidden layers and an output layer. The input layer receives normalized spatial coordinates, and the output layer generates pixel values ​​corresponding to the coordinate points.

3. The image super-resolution reconstruction method based on ridgelet according to claim 2, characterized in that: After the introduction of periodic coding, the input layer is changed to receive the encoded feature vector, and the input dimension is changed from 2 to 2m1L dim , m1 represents the total number of frequencies, L dim Indicates the coordinate dimension.

4. The image super-resolution reconstruction method based on ridgelet according to claim 1, characterized in that: The ridgelet-constrained loss function ensures that the second-order derivative of the reconstruction function in the gradient direction is close to zero, that is, the rate of change of the function along the gradient direction reaches an extreme value at the extreme point.

5. The image super-resolution reconstruction method based on ridgelet according to claim 4, characterized in that: The ridge wave characteristics maintained by the ridge wave constraint have periodic consistency. A ridge wave periodic consistency constraint needs to be applied so that the ridge wave characteristics of the reconstruction function remain consistent after a periodic offset occurs. The periodicity is determined by Fourier analysis.

6. The image super-resolution reconstruction method based on ridgelet according to claim 1, characterized in that: The fractal dimension is calculated by a box counting method, in which a linear regression is performed on the number of boxes of different scales required to cover the ridge wave structure in a double logarithmic coordinate system, and the slope of the regression line is obtained as an estimated value of the fractal dimension.

7. The image super-resolution reconstruction method based on ridgelet according to claim 2, characterized in that: Jump connections are added between adjacent hidden layers of the multi-layer perceptron network so that the output of each layer is not only passed to the next layer, but also fused with the features of deeper layers to form a multi-scale feature representation.

8. The image super-resolution reconstruction method based on ridgelet according to claim 1, characterized in that: The high-order derivative regularization term includes two parts: the first-order derivative and the second-order derivative. The first-order derivative regularization term uses the L1 norm to promote gradient sparsity, and the second-order derivative regularization term uses the L2 norm to control the curvature of the function.

9. The image super-resolution reconstruction method based on ridgelet according to claim 1, characterized in that: The training process of the multi-layer perceptron network adopts a batch processing method and uses an importance sampling strategy to increase the sampling probability of ridge wave areas and high-frequency detail areas, so that the network pays more attention to key structural features in the image.

10. A computer storage medium, characterized in that The invention comprises a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement a ridgelet-based image super-resolution reconstruction method according to any one of claims 1 to 9.

Images