An image interpolation method based on high dimension

Abstract

The application relates to the technical field of computer vision and image processing, and discloses an image interpolation method based on high dimensionality, which comprises a video decoder module, a timestamp analysis module, a feature extraction module, a time sequence correlation modeling module, a time sequence correlation modeling module, a convolutional neural network module, an edge smoothing processing module, an edge detection module, an edge enhancement module, an image output module and a quality evaluation module; the video decoder module extracts images; the timestamp analysis module records time points and positions; the feature extraction module extracts image edge feature data; the time sequence correlation modeling module captures time and space correlation data; the convolutional neural network module performs pixel-level interpolation on image frames; the edge smoothing processing module optimizes the interpolation results; the edge detection module identifies pixel points; the edge enhancement module is applied to perform enhancement processing on edge pixel points; the image output module generates high-quality interpolated image frames; and the quality evaluation module checks the quality of the generated images.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision and image processing, in particular to an image interpolation method based on high dimension. BACKGROUND

[0002] In the field of image interpolation, although the image interpolation method based on high dimension can improve the resolution and detail restoration ability of the image to some extent, there are still some significant defects in the prior art. Among them, the most prominent problem is that when the image magnification is high, these methods may introduce artifacts and noise, thereby reducing the quality and readability of the image.

[0003] The generation of artifacts and noise is mainly due to the inaccurate estimation of image pixel values during the interpolation process. In high-dimensional image interpolation, the algorithm needs to estimate the value of unknown pixels based on the value of known pixels, which often involves complex mathematical models and a large amount of calculation. However, due to the complexity and diversity of image data, as well as the limitations of the interpolation algorithm itself, it is difficult to ensure that the estimation of all pixels is completely accurate. Therefore, in the interpolated image, there may be some pixel value changes that do not originally exist, i.e. artifacts. At the same time, due to the introduction of random errors or rounding errors during the interpolation process, noise may also appear in the image.

[0004] The appearance of artifacts and noise not only affects the visual effect of the image, making it look more blurred or distorted, but also may bring difficulties to subsequent image processing and analysis. For example, in medical image analysis, artifacts and noise may interfere with the doctor's judgment of the lesion; in remote sensing image processing, they may affect the recognition and classification accuracy of ground objects. Therefore, how to effectively suppress or eliminate artifacts and noise is one of the important problems that the high-dimensional image interpolation method needs to solve. SUMMARY

[0005] (I) Technical problems solved

[0006] In view of the shortcomings of the prior art, the present application provides an image interpolation method based on high dimension, which has the advantages of effectively suppressing or eliminating artifacts and noise, and solves the problem that the prior art cannot effectively suppress or eliminate artifacts and noise when in use.

[0007] (II) Technical solutions

[0008] To achieve the above-mentioned purpose, the present application provides the following technical solutions: an image interpolation method based on high dimension, comprising the following steps:

[0009] Step one, establish video decoder module, timestamp analysis module, feature extraction module, timing correlation modeling module, timing correlation modeling module, convolutional neural network module, edge smoothing processing module, edge detection module, edge enhancement module, image output module and quality assessment module;

[0010] Step two, using video decoder module, frame by frame from the original video file to extract images;

[0011] Step three, through the timestamp analysis module, record the specific time point and position of each frame, provide reference for subsequent interpolation operation;

[0012] Step four, using feature extraction module, extract edge and texture image edge feature data from each frame of image;

[0013] Step five, using timing correlation modeling module, based on the image edge feature data of the previous step, construct a multi-dimensional timing correlation model, capture the time and space correlation data between adjacent frames;

[0014] Step six, using convolutional neural network module, pixel level interpolation is carried out on the image frame, and the value of unknown pixel point is predicted and filled;

[0015] Step seven, edge smoothing processing module further optimizes the interpolation result, ensures the smooth transition and detail recovery of edge pixel points;

[0016] Step eight, using edge detection module, identify the blurred or distorted edge pixel points generated in the interpolation process;

[0017] Step nine, using edge enhancement module, enhance the edge pixel points;

[0018] Step ten, using image output module, generate high quality interpolated image frame;

[0019] Step eleven, through the quality assessment module, check the quality of the generated image.

[0020] Preferably, the feature extraction module extracts 9 groups of image edge feature data about edge and texture from each frame of image, and carries out statistics and numbering on the 9 groups of image edge feature data, the 9 groups of image edge feature data are numbered as According to the numbered 9 groups of image edge feature data, an original image matrix is formed as follows:

[0021] ;

[0022] The original image matrix is 3x3, , represents the first group of edge feature data extracted from the image, represents the second set of edge feature data extracted from the image, represents the third set of edge feature data extracted from the image.

[0023] Preferably, the original image matrix The edge intensity is calculated The calculation formula is as follows:

[0024] In the formula, represents the generated edge intensity, respectively represents the 9 sets of image edge feature data, represents the gradient in the horizontal axis direction, represents the gradient in the vertical axis direction.

[0025] Preferably, the edge intensity can reflect the edge degree of the acquired image. When the edge intensity is high, it corresponds to the edge region in the image; and when the edge intensity is low, it corresponds to the non-edge region in the image.

[0026] Preferably, the time sequence correlation modeling module is used to construct a multi-dimensional time sequence correlation model based on the image edge feature data of the previous step, capture the time and space correlation data between adjacent frames, record and count the time and space correlation data between adjacent frames, and perform data numbering. The time correlation data between adjacent frames is numbered as , and the space correlation data between adjacent frames is numbered as .

[0027] Preferably, the time correlation data between adjacent frames and the space correlation data between adjacent frames are used to calculate the pixel value The calculation formula is as follows:

[0028] ;

[0029] In the formula, represents the pixel value, represents the time correlation data between adjacent frames, represents the space correlation data between adjacent frames, represents the time and space correlation coefficient between adjacent frames, represents the amount of statistical data.

[0030] Preferably, the calculation of the pixel value can be dynamically adjusted according to the changes in image content, and is suitable for different image sequences and scenes.

[0031] Preferably, the convolutional neural network module, the convolutional neural network module, pixel-level interpolation is carried out on the image frame, the value of the unknown pixel point is predicted and filled, the pixel-level interpolation and the predicted value of the unknown pixel point of the image frame are recorded and counted, and data numbering is carried out, the pixel-level interpolation of the image frame is numbered as , and the predicted value of the unknown pixel point is numbered as .

[0032] Preferably, the pixel-level interpolation and the predicted value of the unknown pixel point of the image frame are jointly calculated to generate a high-dimensional prediction model pixel difference index , and the calculation formula is as follows:

[0033] ;

[0034] In the formula, indicates the high-dimensional prediction model pixel difference index, indicates the pixel-level interpolation of the image frame, indicates the pixel-level interpolation of the i-th image frame, indicates the predicted value of the unknown pixel point, indicates the i-th predicted value of the unknown pixel point, indicates the number of counted values.

[0035] Preferably, the high-dimensional prediction model pixel difference index can be used as a feedback signal to dynamically adjust the interpolation strategy or parameters to adapt to different image contents and complexities.

[0036] Compared with the prior art, the present application provides an image interpolation method based on high dimension, which has the following beneficial effects:

[0037] 1. The present application calculates the edge strength , and through the comparison between the threshold value and the edge strength , part of the artifacts and noise can be effectively inhibited or eliminated. After the edge points are identified, filtering technology (such as Gaussian filtering, median filtering, etc.) is further applied to smooth the image and reduce residual noise. At the same time, morphological operations (such as dilation, erosion, opening operation, closing operation, etc.) are used to clean up the details in the image and remove isolated noise points, so as to maintain the integrity of the edge. Finally, the processed image is visually inspected to verify the effect of edge detection. If there are still artifacts or noise that have not been completely eliminated, more complex post-processing technology will be switched.

[0038] 2. The present application calculates the pixel value , which can be dynamically adjusted according to the change of image content, so as to adapt to different image sequences and scenes, thereby improving the universality of interpolation. A higher value indicates that the pixel is of high importance in the image, and also suggests that the pixel is located at the edge of the image, in a textured area, or in an area with significant color variation. This helps the model reconstruct key regions more accurately during interpolation, thereby improving the overall image quality. Conversely, when the calculated pixel values... A lower value indicates that the pixel is less important in the image, and also suggests that the pixel is located in the background or an area with little color variation. These pixels do not require much attention during interpolation because they have little impact on the overall image quality. The pixel values ​​mentioned above... This calculation method not only improves the accuracy of interpolation but also preserves the key features of the image. Attached Figure Description

[0039] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0040] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0041] Please see Figure 1 A high-dimensional image interpolation method includes the following steps:

[0042] Step 1: Establish the following modules: video decoder, timestamp analysis, feature extraction, temporal correlation modeling, convolutional neural network (CNN), edge smoothing, edge detection, edge enhancement, image output, and quality assessment.

[0043] Step 2: Use the video decoder module to extract images frame by frame from the original video file;

[0044] Step 3: Record the specific time and position of each frame using the timestamp analysis module to provide a reference for subsequent interpolation operations;

[0045] Step 4: Using the feature extraction module, extract image edge feature data of edges and textures from each frame of the image;

[0046] Step 5: Using the temporal correlation modeling module, construct a multi-dimensional temporal correlation model based on the image edge feature data from the previous step to capture the temporal and spatial correlation data between adjacent frames;

[0047] Step six, using the convolutional neural network module, pixel-level interpolation is performed on the image frames to predict and fill in the values of unknown pixels;

[0048] Step seven, the edge smoothing processing module further optimizes the interpolation results to ensure smooth transition and detail recovery of edge pixels;

[0049] Step eight, use the edge detection module to identify blurred or distorted edge pixels generated during the interpolation process;

[0050] Step nine, apply the edge enhancement module to enhance the edge pixels (improve the clarity and readability of the overall image);

[0051] Step ten, use the image output module to generate high-quality interpolated image frames (ensure resolution improvement and maintain edge clarity and detail richness of the original image);

[0052] Step eleven, through the quality evaluation module, the generated image is checked for quality (to ensure that it meets the needs of subsequent processing and analysis tasks, and outputs the interpolated image frames).

[0053] The feature extraction module extracts 9 groups of edge and texture image edge feature data from each image, and performs statistical and numbering on the 9 groups of image edge feature data, which are numbered as According to the numbered 9 groups of image edge feature data, an original image matrix is formed as follows:

[0054] ;

[0055] The original image matrix is 3x3, represents the first group of edge feature data extracted from the image, represents the second group of edge feature data extracted from the image, represents the third group of edge feature data extracted from the image.

[0056] The original image matrix calculates the generated edge intensity , whose calculation formula is as follows:

[0057] ; in the formula, represents the generated edge intensity, represents the 9 groups of image edge feature data, represents the gradient in the horizontal direction, represents the gradient in the vertical direction.

[0058] The advantage is that the edge intensity is calculated Using a single gradient magnitude as the edge intensity, the nine numbered sets of image edge feature data are combined into an original image matrix to generate multidimensional feature data, thus calculating the edge intensity. It can comprehensively reflect the edge information of an image, based on edge intensity. Input the data features and historical model data, and set an edge strength. The threshold when the edge strength When the value is higher than this threshold, it corresponds to an edge region in the image, and the pixel is considered an edge point; while when the edge intensity is higher... Points below this threshold correspond to non-edge regions in the image and are considered non-edge points. This threshold is then compared with edge strength. The comparison can effectively suppress or eliminate some artifacts and noise. After identifying edge points, further filtering techniques (such as Gaussian filtering, median filtering, etc.) are applied to smooth the image and reduce residual noise. At the same time, morphological operations (such as dilation, erosion, opening, closing, etc.) are used to clean up details in the image, remove isolated noise points, and maintain the integrity of the edges. Finally, the processed image is visually inspected to verify the edge detection effect. If there are still artifacts or noise that have not been completely eliminated, more complex post-processing techniques will be switched.

[0059] Edge strength It can reflect the edge strength of the acquired image, when the edge strength When the edge intensity is high, it corresponds to the edge region in the image; while when the edge intensity is low... When the value is low, it corresponds to a non-edge region in the image.

[0060] The temporal correlation modeling module constructs a multi-dimensional temporal correlation model based on the image edge feature data from the previous step. It captures the temporal and spatial correlation data between adjacent frames (temporal correlation between adjacent frames refers to the similarity between consecutive frames in an image sequence; since objects in a video or image sequence do not change drastically in a short period, the pixel values ​​between adjacent frames will be similar). The module records and statistically analyzes the temporal and spatial correlation data between adjacent frames and assigns data numbers. The temporal correlation data between adjacent frames is numbered as follows: The spatial correlation data between adjacent frames is numbered as follows: .

[0061] The pixel value is generated by combining the temporal correlation data and the spatial correlation data between adjacent frames. The calculation formula is as follows:

[0062] ;

[0063] In the formula, pixel value, temporal correlation data between adjacent frames, spatial correlation data between adjacent frames, temporal and spatial correlation coefficients between adjacent frames, statistical data amount.

[0064] The advantage is that the pixel value can be dynamically adjusted according to the changes of image content, so as to adapt to different image sequences and scenes, thereby improving the universality of interpolation. When the calculated pixel value is high, it indicates that the pixel point has high importance in the image, and at the same time indicates that the pixel point is located at the edge, texture area or area with large color change of the image. High pixel value helps the model to more accurately reconstruct the key area in the interpolation process, thereby improving the overall quality of the image. On the contrary, when the calculated pixel value is low, it indicates that the importance of the pixel point in the image is low, and at the same time indicates that the pixel point is located in the background area or the area with small color change. For the pixel point with low pixel value , it does not need to be paid too much attention in the interpolation process, because its influence on the overall quality of the image is small. The above calculation method of pixel value not only improves the accuracy of interpolation, but also preserves the key features of the image.

[0065] The calculation of pixel value can be dynamically adjusted according to the changes of image content, so as to adapt to different image sequences and scenes (thereby improving the universality and effect of interpolation).

[0066] The convolutional neural network module, the convolutional neural network module, pixel-level interpolation is performed on the image frame, the value of the unknown pixel point is predicted and filled, the pixel-level interpolation of the image frame and the predicted value of the unknown pixel point are recorded and counted, and the data is numbered. The pixel-level interpolation of the image frame is numbered , and the predicted value of the unknown pixel point is numbered .

[0067] The pixel-level interpolation of the image frame and the predicted value of the unknown pixel point are jointly calculated to generate a high-dimensional prediction model pixel difference index , and the calculation formula is as follows:

[0068] ;

[0069] In the formula, represents the high-dimensional prediction model pixel difference index, represents the pixel-level interpolation of the image frame, represents the i-th pixel-level interpolation of the image frame, represents the predicted unknown pixel point value, represents the i-th predicted unknown pixel point value, represents the statistical value quantity.

[0070] The advantage is that the high-dimensional prediction model pixel difference index The interpolation method can better cope with various types of image sequences and scenes, whether static or dynamic image scenes, and can obtain high-quality interpolation results, thereby enhancing the universality of the method, the high-dimensional prediction model pixel difference index can help improve the quality of the reconstructed frame, dynamically adjust the interpolation parameters to reduce artifacts and noise, thereby retaining more detailed information to improve the visual effect of the image, and the high-dimensional prediction model pixel difference index can also enhance the robustness of the interpolation model, allowing the model to maintain stable performance in different environments, thereby processing complex and variable image content to improve system reliability.

[0071] The high-dimensional prediction model pixel difference index can be used as a feedback signal to dynamically adjust the interpolation strategy or parameters to adapt to different image content and complexity.

[0072] Although embodiments of the present application have been shown and described, it will be understood by those of ordinary skill in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the present application, and the scope of the present application is defined by the appended claims and their equivalents.

Claims

1. A high-dimensional based image interpolation method, characterized by, Comprising the following steps: Step one, establish video decoder module, timestamp analysis module, feature extraction module, timing correlation modeling module, timing correlation modeling module, convolutional neural network module, edge smoothing processing module, edge detection module, edge enhancement module, image output module and quality assessment module; Step two, using video decoder module, extract image frame by frame from the original video file; Step three, through the timestamp analysis module, record the specific time point and position of each frame, provide reference for subsequent interpolation operation; Step four, use feature extraction module to extract edge and texture image edge feature data from each frame of image; Step five, using timing correlation modeling module, based on the image edge feature data of the previous step, construct multi-dimensional timing correlation model, capture the time and space correlation data between adjacent frames, the time correlation between adjacent frames refers to the similarity between consecutive frames in image sequence, because the object in video or image sequence will not change dramatically in a short time, so the pixel value between adjacent frames will be similar; Step six, using convolutional neural network module, pixel level interpolation is carried out on image frame, the value of unknown pixel point is predicted and filled; Step seven, edge smoothing processing module further optimizes the interpolation result, ensures the smooth transition and detail recovery of edge pixel points; Step eight, using edge detection module, identify the blurred or distorted edge pixel points generated in the interpolation process; Step nine, apply edge enhancement module to enhance the edge pixel points; Step ten, using image output module, generate high quality interpolated image frame; Step eleven, through quality assessment module, check the quality of the generated image; The time sequence correlation modeling module is used for constructing a multi-dimensional time sequence correlation model based on the image edge feature data of the previous step, capturing time and space correlation data between adjacent frames, recording and counting the time and space correlation data between adjacent frames, and numbering the data, wherein the time correlation data between adjacent frames is numbered as 、 、 , … , and the space correlation data between adjacent frames is numbered as 、 、 , … ; The temporal correlation data between adjacent frames and the spatial correlation data between adjacent frames are jointly calculated to generate a pixel value The calculation formula is: In the formula, represents a pixel value, , , ,... represents time correlation data between adjacent frames, , , ,... represents spatial correlation data between adjacent frames, , , ,... represents a time and spatial correlation coefficient between adjacent frames, represents a statistical data amount.

2. The high-dimensional based image interpolation method of claim 1, wherein, The feature extraction module extracts nine sets of image edge feature data related to edges and textures from each frame of the image, and statistically analyzes and numbers these nine sets of image edge feature data. The nine sets of image edge feature data are respectively numbered as follows: , , , , , , , , An original image matrix is ​​formed based on the nine sets of image edge feature data that have been numbered. ,as follows: original image matrix is 3x3, represents a first set of edge feature data extracted from the image, represents a second set of edge feature data extracted from the image, represents a third set of edge feature data extracted from the image.

3. The high-dimensional image interpolation method of claim 2, wherein: the original image matrix calculating the edge strength and the calculation formula is as follows: In the formula, denotes the generated edge strength, , , , , , , , , denote 9 groups of image edge feature data respectively, denotes the gradient in the horizontal axis direction, denotes the gradient in the vertical axis direction.

4. The high-dimensional image interpolation method of claim 3, wherein: the edge strength is able to reflect the degree of edges in the acquired image, when the edge strength is high, corresponds to an edge region in the image; while when the edge strength is low, corresponds to a non-edge region in the image.

5. The high-dimensional image interpolation method of claim 1, wherein: The pixel value The calculation can be dynamically adjusted according to the changes of image content, and is adaptive to different image sequences and scenes.

6. The high-dimensional image interpolation method of claim 1, wherein: The convolutional neural network module performs pixel-level interpolation on the image frame, predicts and fills in the value of the unknown pixel point, records and counts the pixel-level interpolation of the image frame and the predicted value of the unknown pixel point, and performs data numbering, wherein the pixel-level interpolation of the image frame is numbered as 、 、 、… , and the predicted value of the unknown pixel point is numbered as 、 、 、… .

7. The high-dimensional image interpolation method of claim 6, wherein: The image frame pixel-level interpolation and the predicted unknown pixel point value jointly calculate a high-dimension prediction model pixel difference index The calculation formula is as follows: In the formula, represents the high-dimensional prediction model pixel difference index, , , , represents the image frame pixel-level interpolation, represents the i-th image frame pixel-level interpolation, , , , represents the predicted unknown pixel point value, represents the i-th predicted unknown pixel point value, represents the number of statistical values.

8. The high-dimensional image interpolation method of claim 7, wherein: The high-dimensional prediction model pixel difference index , can be used as a feedback signal to dynamically adjust the interpolation strategy or parameters to adapt to different image content and complexity.

Images