Image resolution improving method and device, equipment and medium

Abstract

The invention discloses an image resolution improving method and device, equipment and a medium, and relates to the technical field of image resolution improvement, and the method comprises the steps: obtaining known pixel points and to-be-filled pixel points in a low-resolution image with the resolution to be improved; partitioning the low-resolution image, extracting environment color features of each region, and detecting to obtain an edge boundary of the low-resolution image; based on the environment color features of the area where the current to-be-filled pixel point is located in the low-resolution image and the color values of known pixel points which are located in the neighborhood of the low-resolution image and located on the same side of the same edge boundary, the filling value of the current to-be-filled pixel point is calculated and the current to-be-filled pixel point is filled; therefore, the resolution of the low-resolution image is improved. According to the method, known pixel information on the same side of the boundary is strictly used during filling, cross-boundary mixing of colors is prevented, and the overall hue of the amplified colors is coordinated by using the environmental color features for filling.

Description

Technical Field

[0001] This application relates to the field of image resolution enhancement technology, specifically to an image resolution enhancement method, apparatus, device, and medium. Background Technology

[0002] Traditional interpolation methods can lead to blurred edges, loss of detail, and jagged edges or ringing effects when upscaling low-resolution images. Learning-based super-resolution performs well, but requires large amounts of training data and powerful computing resources, resulting in large model parameters and making it difficult to deploy on resource-constrained embedded devices. It may also produce artifacts that do not conform to physical reality. Existing methods often perform poorly when upscaling from extremely low to high resolution, especially in areas lacking sufficient texture information. They typically lack a global perception of the overall color environment of the image and fail to fully utilize perceptual characteristics to optimize display effects. Their handling of boundaries is not intelligent enough, easily leading to blurred or distorted edges. Summary of the Invention

[0003] This application provides an image resolution enhancement method, apparatus, device, and medium, which can solve the technical problem of poor magnification effect of low-resolution images in the prior art.

[0004] In a first aspect, embodiments of this application provide an image resolution enhancement method, the image resolution enhancement method comprising: Obtain the known pixels and the pixels to be filled in the low-resolution image that needs to be upgraded; The low-resolution image is divided into regions, the environmental color features of each region are extracted, and the edge boundaries of the low-resolution image are detected. Based on the environmental color features of the region where the pixel to be filled is located in the low-resolution image, and the color values ​​of known pixels located in its neighborhood and on the same side of the same edge boundary, the filling value of the current pixel to be filled is calculated and the current pixel to be filled is filled, thereby achieving the resolution improvement of the low-resolution image.

[0005] In conjunction with the first aspect, in one implementation, obtaining known pixels and pixels to be filled in a low-resolution image whose resolution needs to be upgraded includes: Acquire the low-resolution image whose resolution needs to be upgraded, perform image noise reduction, color correction and white balance processing on the low-resolution image whose resolution needs to be upgraded, and set the target high-resolution size. The low-resolution image is enlarged to the target high-resolution size using interpolation to obtain the enlarged low-resolution image; In the enlarged low-resolution image, the pixels corresponding to the original pixel positions in the unenlarged low-resolution image are identified as known pixels, and the remaining pixels in the enlarged low-resolution image are identified as pixels to be filled.

[0006] In conjunction with the first aspect, in one implementation, partitioning the low-resolution image and extracting the environmental color features of each region includes: The enlarged low-resolution image is divided into multiple regions according to a preset size; Calculate the average lightness, the average first chromaticity, and the average second chromaticity of all pixels in each region in the Lab color space; The triplet obtained based on the average brightness, the first average chromaticity, and the second average chromaticity will be used as the environmental color feature of each region.

[0007] In conjunction with the first aspect, in one implementation, detecting the edge boundaries of the low-resolution image includes: An edge detection algorithm is used to process and extract the initial edge pixels of the magnified low-resolution image; Morphological processing is performed on the initial edge pixels to connect the broken edges and form a continuous edge boundary line.

[0008] In conjunction with the first aspect, in one implementation, before the step of considering the ambient color features of the region where the currently filled pixel is located in the low-resolution image and the color values ​​of known pixels located in its neighborhood and on the same side of the same edge boundary, the following steps are included: Determine the neighborhood of the current pixel to be filled, centered on the current pixel to be filled. Obtain the Euclidean distance between the current pixel to be filled and the nearest edge boundary. Calculate the reliability of the current pixel to be filled based on the Euclidean distance. The specific calculation method is as follows:

[0009] in, Indicates the current pixel to be filled. Reliability, Indicates the current pixel to be filled. The number of known pixels in the neighborhood. This represents the Euclidean distance between the current pixel to be filled and the nearest edge boundary. Represents a minimal constant; Sort all pixels to be filled from highest to lowest reliability, and determine the filling order of each pixel in turn, so that pixels with higher reliability are filled first.

[0010] In conjunction with the first aspect, in one implementation, the step of calculating the fill value of the current pixel to be filled based on the environmental color features of the region where the current pixel to be filled is located in the low-resolution image and the color values ​​of known pixels located in its neighborhood and on the same side of the same edge boundary line includes: Known pixels located within the neighborhood of the current pixel to be filled and on the same side of the same edge boundary are selected as valid reference pixels. The fusion weight of each valid reference pixel relative to the current pixel to be filled is calculated as follows:

[0011]

[0012]

[0013]

[0014] in, Indicates valid reference pixels Relative to the fusion weights of the current pixels to be filled, Indicates spatial distance weights. Indicates color similarity weight. Indicates the boundary distance weight. This represents the spatial Euclidean distance between the valid reference pixel and the current pixel to be filled. Indicates the spatial attenuation coefficient. , Indicates valid reference pixels color value, This represents the average color of all valid reference pixels within the neighborhood of the pixel to be filled. Indicates the color attenuation coefficient. , This indicates the distance between the valid reference pixel and the nearest edge boundary. This represents the attenuation coefficient of the edge boundary line. ; Based on the fusion weights and the color values ​​of each effective reference pixel, a weighted average is calculated on the color values ​​of all effective reference pixels to obtain a local reference value. The specific calculation method is as follows:

[0015] in, Indicates a local reference value. Indicates valid reference pixels The color value; Obtain the color smoothness of the region where the current pixel to be filled is located, and calculate the dynamic fusion coefficient of the current pixel to be filled. The specific calculation method is as follows:

[0016] in, Represents the dynamic fusion coefficient. Indicates empirical weights, , This indicates the color smoothness of the area where the current pixel to be filled is located. Indicates the normalization factor; Based on the local reference value, the dynamic fusion coefficient of the current pixel to be filled, and the environmental color features of the region where the current pixel to be filled is located, the filling value of the current pixel to be filled is calculated. The specific calculation method is as follows:

[0017] in, Indicates the current pixel to be filled. The fill value, It represents the environmental color characteristics of the area where the pixel to be filled is located.

[0018] In conjunction with the first aspect, in one implementation, after filling the current pixels to be filled, thereby achieving a resolution increase in the low-resolution image, the process includes: Unsharpened mask sharpening is applied to the edge boundaries in the high-resolution image. Gaussian filtering noise reduction is performed on smooth regions in the high-resolution image where the color variance is below a preset threshold. The processed high-resolution image is output to the display device; The high-resolution image is a low-resolution image that has been upgraded.

[0019] Secondly, embodiments of this application provide an image resolution enhancement device, the image resolution enhancement device comprising: The acquisition module is used to acquire known pixels and pixels to be filled in a low-resolution image whose resolution needs to be upgraded. The partitioning module is used to partition the low-resolution image, extract the environmental color features of each region, and detect the edge boundaries of the low-resolution image. The calculation module is used to calculate the fill value of the current pixel to be filled based on the environmental color features of the area where the current pixel to be filled is located in the low-resolution image and the color values ​​of known pixels located in its neighborhood and on the same side of the same edge boundary, and then fill the current pixel to be filled, thereby realizing the resolution improvement of the low-resolution image.

[0020] Thirdly, embodiments of this application provide an image resolution enhancement device, which includes a processor, a memory, and an image resolution enhancement program stored in the memory and executable by the processor. When the image resolution enhancement program is executed by the processor, it implements the steps of the image resolution enhancement method as described in any of the above embodiments.

[0021] Fourthly, embodiments of this application provide a computer-readable storage medium storing an image resolution enhancement program, wherein when the image resolution enhancement program is executed by a processor, it implements the steps of the image resolution enhancement method as described in any of the above embodiments.

[0022] The beneficial effects of the technical solutions provided in this application include: This application's embodiments prevent color mixing across boundaries by detecting edge boundaries and strictly using known pixel information on the same side of the boundary during filling, thus maintaining sharp edges after magnification. This application's embodiments also extract and utilize environmental color features from different regions for filling, ensuring overall color harmony after magnification, solving the problems of traditional methods lacking global perception and prone to color banding or inconsistency. This application's embodiments are based on a pixel-level filling algorithm using environmental features and boundary constraints, eliminating the need for large deep learning models and their training processes. Therefore, it greatly reduces the requirement for large amounts of memory and training data, enabling it to run on embedded devices with limited computing power. Attached Figure Description

[0023] Figure 1 This is a flowchart illustrating the image resolution enhancement method of this application; Figure 2 This is a schematic diagram of the functional modules of the image resolution enhancement device of this application; Figure 3 This is a schematic diagram of the hardware structure of the image resolution enhancement device involved in the embodiments of this application. Detailed Implementation

[0024] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0025] This application provides an image resolution enhancement method, apparatus, device, and medium, which can solve the technical problem of poor magnification effect of low-resolution images in the prior art.

[0026] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0027] In a first aspect, embodiments of this application provide a method for improving image resolution.

[0028] In one embodiment, reference is made to Figure 1 , Figure 1 This is a flowchart illustrating the image resolution enhancement method of this application. Figure 1 As shown, the image resolution enhancement method specifically includes the following steps: Step S1: Obtain the known pixels and the pixels to be filled in the low-resolution image whose resolution needs to be upgraded.

[0029] In this embodiment of the application, step S1 specifically includes the following steps: Step S11: Obtain the low-resolution image whose resolution needs to be upgraded, perform image noise reduction, color correction and white balance processing on the low-resolution image whose resolution needs to be upgraded, and set the target high-resolution size.

[0030] Specifically, low-resolution images can be images with a resolution between 160×120 pixels and 640×480 pixels, which is lower than the standard resolution of modern display devices. Spatial or temporal filters are applied to denoise the low-resolution images to suppress random noise introduced by sensor signal-to-noise ratio limitations. A color correction matrix compensates for sensor spectral response deviations, and a white balance algorithm corrects the color temperature to ensure that image color reproduction matches human visual perception. Simultaneously, a specific target high-resolution size is clearly defined based on the physical pixel array of the output display device.

[0031] Step S12: Enlarge the low-resolution image to the target high-resolution size using interpolation to obtain the enlarged low-resolution image.

[0032] Specifically, bilinear interpolation or bicubic interpolation algorithms are used to map and resample the pixel grid of the low-resolution image preprocessed in step S11 onto the target high-resolution pixel grid. The image processed in step S12 meets the final output requirements in terms of geometric dimensions.

[0033] Step S13: Determine the pixels in the magnified low-resolution image that correspond to the original pixel positions in the unmagnified low-resolution image as known pixels, and determine the remaining pixels in the magnified low-resolution image as pixels to be filled.

[0034] Specifically, pixels whose spatial location precisely corresponds to pixels in the original low-resolution image are identified as known pixels. The color values ​​of these known pixels are directly derived from the original low-resolution image and are considered reliable information sources. Pixels in the low-resolution image enlarged to the target high-resolution size that do not have a direct counterpart in the original image are identified as pixels to be filled. The color values ​​of these pixels are either empty or only preliminary interpolation estimates, and they are the targets that need to be filled later.

[0035] Step S2: Divide the low-resolution image into regions, extract the environmental color features of each region, and detect the edge boundaries of the low-resolution image.

[0036] In this embodiment of the application, step S2, which involves partitioning the low-resolution image and extracting the environmental color features of each region, specifically includes the following steps: Step S211: Divide the enlarged low-resolution image into multiple regions according to a preset size.

[0037] Specifically, the magnified low-resolution image generated in step S1 is a two-dimensional matrix composed of pixels. Using a square window of a preset size as the unit, the two-dimensional matrix is ​​evenly divided into several non-overlapping local rectangular regions, each containing the same number of pixels. The preset size can be 16 pixels × 16 pixels.

[0038] Step S212: Calculate the average value of lightness, the average value of first chromaticity, and the average value of second chromaticity for all pixels in each region in the Lab color space.

[0039] Specifically, the color value of each pixel in each region is converted from the original RGB representation to the Lab color space. In the Lab color space, the L component represents lightness, with a value range from 0 (pure black) to 100 (pure white), quantifying the human eye's perception of brightness; the a component represents red-green hue, with negative values ​​tending towards green and positive values ​​towards red; and the b component represents yellow-blue hue, with negative values ​​tending towards blue and positive values ​​towards yellow.

[0040] After the conversion is completed, the arithmetic mean of the three components of all pixels in the current region is calculated. Specifically, the lightness average of the current region is obtained by summing the L components of all pixels in the region and dividing by the total number of pixels in the current region; the first chromaticity average of the current region is obtained by summing the a components of all pixels in the region and dividing by the total number of pixels in the current region; and the second chromaticity average of the current region is obtained by summing the b components of all pixels in the region and dividing by the total number of pixels in the current region.

[0041] Step S213: Use the triplet obtained based on the average value of lightness, the average value of the first chromaticity, and the average value of the second chromaticity as the environmental color feature of each region.

[0042] Specifically, the average value of lightness, the average value of the first chromaticity, and the average value of the second chromaticity are combined in sequence into a three-dimensional vector, namely (L avg a avg b avg ), where L avg a represents the average brightness. avg b represents the average value of the first chromaticity. avg This represents the average value of the second chromaticity. This three-dimensional vector is the environmental color feature that is finally extracted from the current area. The environmental color feature can represent the overall brightness level and chromaticity tendency of the colors in the current area.

[0043] Furthermore, the Otsu method (maximum inter-class variance method) can be used to divide the entire image into multiple dark and light regions. When performing color calculation on a pixel to be filled, it can first be determined whether the point belongs to a dark or light region, and then the corresponding environmental color features can be called for fusion.

[0044] In this embodiment of the application, the detection of the edge boundary line of the low-resolution image in step S2 specifically includes the following steps: Step S221: Use an edge detection algorithm to process and extract the initial edge pixels of the magnified low-resolution image.

[0045] Specifically, the Canny edge detection operator is used to perform convolution operations on the magnified low-resolution image. First, the input image is Gaussian blurred to suppress sensor noise and minute textures that might be falsely detected as edges. Then, the gradient magnitude and direction at each pixel location are calculated. The gradient magnitude reflects the intensity of local gray-level changes, and the direction indicates the normal direction of the edge. Next, a non-maximum suppression operation is used to set the gradient magnitude at non-edge peak locations to zero, achieving edge refinement. The Canny edge algorithm employs a dual-threshold strategy: a high threshold is used to extract strong edge pixels, and a low threshold is used to connect edge segments, ultimately confirming the portion of weak edge points connected to strong edge points as edges. In this embodiment, pixels identified as edges are assigned a value of "1", and all non-edge pixels are assigned a value of "0". The set of pixels with a value of 1 is the initial edge pixels.

[0046] Step S222: Perform morphological processing on the initial edge pixels to connect the broken edges and form a continuous edge boundary line.

[0047] Specifically, morphological closing is a standard shape processing method in digital image processing, which includes a composite operation of dilation followed by erosion. The dilation operation uses a 3×3 square structuring element to expand the initial edge pixels to their 8-neighborhood, connecting the endpoints of adjacent broken edges. The erosion operation shrinks the dilated edge contour to approximately the original width, thereby connecting the originally broken initial edge pixels to form a continuous edge boundary.

[0048] Step S3: Based on the environmental color features of the area where the current pixel to be filled is located in the low-resolution image and the color values ​​of known pixels located in its neighborhood and on the same side of the same edge boundary, calculate the filling value of the current pixel to be filled and fill the current pixel to be filled, thereby achieving resolution improvement of the low-resolution image.

[0049] In this embodiment of the application, before step S3, the following steps are included: Step A1: Determine the neighborhood of the current pixel to be filled, centered on the current pixel to be filled.

[0050] Specifically, a square neighborhood window is established on the image grid of the target high-resolution size, centered on the pixel to be filled. The default size is 5×5 pixels. The spatial range covered by this neighborhood window defines the local spatial context in which the reliability calculation of the pixel to be filled is performed. In some other embodiments of this application, the window size can be adaptively expanded. When the number of known pixels is insufficient, it is gradually increased to 7×7 pixels or 9×9 pixels to ensure statistical validity.

[0051] Step A2: Obtain the Euclidean distance between the current pixel to be filled and the nearest edge boundary. Calculate the reliability of the current pixel to be filled based on the minimum Euclidean distance. The specific calculation method is as follows:

[0052] in, Indicates the current pixel to be filled. Reliability, Indicates the current pixel to be filled. The number of known pixels in the neighborhood. This represents the Euclidean distance between the current pixel to be filled and the nearest edge boundary. This represents a very small constant.

[0053] Specifically, in the binary image of the edge boundary zone, all pixels marked as edges are traversed, the Euclidean distance from the current point to be filled to each edge boundary line is calculated, and the minimum value is taken as the boundary line. . It can reflect the richness of local information; the higher the value, the more reliable reference data there is. It can reflect the constraint strength of the edge boundary line. The closer to the boundary, the higher the risk of cross-boundary color mixing, and the reliability should be reduced.

[0054] Step A3: Sort all the pixels to be filled from high to low reliability, and determine the filling order of each pixel in turn, so that the pixels with high reliability are filled first.

[0055] Specifically, since the pixels to be filled first have higher reliability, that is, more reference information and clearer boundary constraints, the filling value may be more accurate. These accurately filled pixels will become known pixels in the calculation of subsequent pixels, thereby improving the quality of subsequent filling.

[0056] In this embodiment of the application, step S3, which calculates the filling value of the current pixel to be filled based on the environmental color features of the region where the pixel to be filled is located in the low-resolution image and the color values ​​of known pixels located in its neighborhood and on the same side of the same edge boundary, specifically includes the following steps: Step S31: Select known pixels that are located in the neighborhood of the current pixel to be filled and are on the same side of the same edge boundary as the current pixel to be filled as valid reference pixels. Calculate the fusion weight of each valid reference pixel relative to the current pixel to be filled. The specific calculation method is as follows:

[0057]

[0058]

[0059]

[0060] in, Indicates valid reference pixels Relative to the fusion weights of the current pixels to be filled, Indicates spatial distance weights. Indicates color similarity weight. Indicates the boundary distance weight. This represents the spatial Euclidean distance between the valid reference pixel and the current pixel to be filled. Indicates the spatial attenuation coefficient. , Indicates valid reference pixels color value, This represents the average color of all valid reference pixels within the neighborhood of the pixel to be filled. Indicates the color attenuation coefficient. , This indicates the distance between the valid reference pixel and the nearest edge boundary. This represents the attenuation coefficient of the edge boundary line. .

[0061] Specifically, the selection of valid reference pixels ensures that only known information with the same semantic region as the point to be filled is used during the filling process, preventing cross-boundary color contamination. Spatial distance weight measures the physical proximity of the valid reference pixel to the current pixel to be filled; the closer the distance, the greater the weight. Color similarity weight measures the similarity between the valid reference pixel and the average color of valid reference pixels in the neighborhood of the current pixel to be filled; the more similar the colors, the greater the weight. Boundary distance weight measures the distance between the valid reference pixel and the nearest edge boundary; the farther the distance, the greater the weight.

[0062] Step S32: Based on the fusion weights and the color values ​​of each effective reference pixel, perform a weighted average of the color values ​​of all effective reference pixels to obtain a local reference value. The specific calculation method is as follows:

[0063] in, Indicates a local reference value. Indicates valid reference pixels The color value.

[0064] Specifically, Indicates valid reference pixels Color values ​​in the Lab color space. Local reference values ​​reflect the locally optimal color estimate of the pixel to be filled, inferred from the effective reference pixel, under the combined effects of spatial proximity, color similarity, and boundary constraints.

[0065] Step S33: Obtain the color smoothness of the region where the current pixel to be filled is located, and calculate the dynamic fusion coefficient of the current pixel to be filled. The specific calculation method is as follows:

[0066] in, Represents the dynamic fusion coefficient. Indicates empirical weights, , This indicates the color smoothness of the area where the current pixel to be filled is located. This represents the normalization factor.

[0067] Specifically, the color variance of the region where the current pixel to be filled is located is calculated, normalized, and used as... The smaller the variance, the higher the smoothness. The calculation of the dynamic fusion coefficient of the current pixel to be filled makes pixels far from the edge boundary and in the smooth area more dependent on the ambient color features, and pixels near the edge boundary or in the complex texture area more dependent on the local reference value.

[0068] Step S34: Based on the local reference value, the dynamic fusion coefficient of the current pixel to be filled, and the environmental color features of the region where the current pixel to be filled is located, calculate the fill value of the current pixel to be filled. The specific calculation method is as follows:

[0069] in, Indicates the current pixel to be filled. The fill value, It represents the environmental color characteristics of the area where the pixel to be filled is located.

[0070] Specifically, when When the value is close to 1, the fill value is mainly based on the local reference value. When the value is close to 0, the fill value is mainly based on the environmental color features, so that the final fill value achieves the optimal trade-off between local estimation and global prior, maintaining both the authenticity of local structure and the consistency of regional color.

[0071] In this embodiment of the application, after step S3, the following is included: Step B1: Perform unsharpened mask sharpening on the edge boundaries in the high-resolution image.

[0072] Specifically, a high-resolution image is a low-resolution image that has been upscaled. A low-pass filter is applied to the high-resolution image to obtain a blurred copy that retains only low-frequency information and has smoothed edges. The original high-resolution image is then subtracted pixel-by-pixel from this blurred copy to obtain a mask image. This mask image primarily contains the high-frequency details from the original image, i.e., edge and texture information. The calculated mask image is multiplied by an intensity coefficient, and then the weighted high-frequency detail information is added back to the original high-resolution image. This compensates for any edge sharpness and contrast that might have been lost during the infilling process.

[0073] Step B2: Perform Gaussian filtering noise reduction on smooth regions in the high-resolution image where the color variance is below a preset threshold.

[0074] Specifically, the color variance of each region in the high-resolution image is calculated. If the variance is lower than a preset threshold, the region is marked as a smooth region. Gaussian filtering is then applied to the pixels marked as smooth regions to eliminate random noise and banding artifacts that may have been introduced in the previous steps, while maintaining constant brightness in the smooth regions. The preset threshold can be 25.

[0075] Step B3: Output the processed high-resolution image to a display device; wherein the high-resolution image is a low-resolution image after resolution enhancement.

[0076] Specifically, the fully processed high-resolution image data is transmitted to the target display device for display via a graphics output interface.

[0077] This application's embodiments prevent color mixing across boundaries by detecting edge boundaries and strictly using known pixel information on the same side of the boundary during filling, thus maintaining sharp edges after magnification. This application's embodiments also extract and utilize environmental color features from different regions for filling, ensuring overall color harmony after magnification, solving the problems of traditional methods lacking global perception and prone to color banding or inconsistency. This application's embodiments are based on a pixel-level filling algorithm using environmental features and boundary constraints, eliminating the need for large deep learning models and their training processes. Therefore, it greatly reduces the requirement for large amounts of memory and training data, enabling it to run on embedded devices with limited computing power.

[0078] Secondly, embodiments of this application also provide an image resolution enhancement device.

[0079] In one embodiment, reference is made to Figure 2 , Figure 2 This is a schematic diagram of the functional modules of the image resolution enhancement device of this application. Figure 2 As shown, the image resolution enhancement device includes: The acquisition module is used to acquire known pixels and pixels to be filled in a low-resolution image whose resolution needs to be upgraded. The partitioning module is used to partition the low-resolution image, extract the environmental color features of each region, and detect the edge boundaries of the low-resolution image. The calculation module is used to calculate the fill value of the current pixel to be filled based on the environmental color features of the area where the current pixel to be filled is located in the low-resolution image and the color values ​​of known pixels located in its neighborhood and on the same side of the same edge boundary, and then fill the current pixel to be filled, thereby realizing the resolution improvement of the low-resolution image.

[0080] The functions of each module in the above-mentioned image resolution enhancement device correspond to the steps in the above-mentioned image resolution enhancement method embodiment, and their functions and implementation processes will not be described in detail here.

[0081] Thirdly, embodiments of this application provide an image resolution enhancement device, which includes a processor, a memory, and an image resolution enhancement program stored in the memory and executable by the processor. When the image resolution enhancement program is executed by the processor, it implements the steps of the image resolution enhancement method as described in any of the above embodiments.

[0082] Image resolution enhancement devices can be personal computers (PCs), laptops, servers, or other devices with data processing capabilities.

[0083] Reference Figure 3 , Figure 3 This is a schematic diagram of the hardware structure of the image resolution enhancement device involved in the embodiments of this application. In the embodiments of this application, the image resolution enhancement device may include a processor, a memory, a communication interface, and a communication bus.

[0084] The communication bus can be of any type and is used to interconnect the processor, memory, and communication interface.

[0085] Communication interfaces include input / output (I / O) interfaces, physical interfaces, and logical interfaces used for interconnecting internal components of the image resolution enhancement device, as well as interfaces used for interconnecting the image resolution enhancement device with other devices (such as other computing devices or user equipment). Physical interfaces can be Ethernet interfaces, fiber optic interfaces, ATM interfaces, etc.; user equipment can be displays, keyboards, etc.

[0086] Memory can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), flash memory, optical storage, hard disk, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), etc.

[0087] The processor can be a general-purpose processor, which can call the image resolution enhancement program stored in memory and execute the image resolution enhancement method provided in the embodiments of this application. For example, the general-purpose processor can be a central processing unit (CPU). The method executed when the image resolution enhancement program is called can be referred to in the various embodiments of the image resolution enhancement method of this application, and will not be repeated here.

[0088] Those skilled in the art will understand that Figure 3 The hardware structure shown does not constitute a limitation of this application and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0089] Fourthly, embodiments of this application provide a computer-readable storage medium storing an image resolution enhancement program, wherein when the image resolution enhancement program is executed by a processor, it implements the steps of the image resolution enhancement method as described in any of the above embodiments.

[0090] The present application has a computer-readable storage medium storing an image resolution enhancement program, wherein when the image resolution enhancement program is executed by a processor, the steps of the image resolution enhancement method described above are implemented.

[0091] The method implemented when the image resolution enhancement procedure is executed can be referred to in various embodiments of the image resolution enhancement method of this application, and will not be repeated here.

[0092] It should be noted that the sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0093] The terms "comprising" and "having," and any variations thereof, in the specification, claims, and accompanying drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus. The terms "first," "second," and "third," etc., are used to distinguish different objects, etc., and do not indicate a sequence, nor do they limit "first," "second," and "third" to different types.

[0094] In the description of the embodiments of this application, terms such as "exemplary," "for example," or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplary," "for example," or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary," "for example," or "for instance" is intended to present the relevant concepts in a concrete manner.

[0095] In the description of the embodiments of this application, unless otherwise stated, " / " means "or". For example, A / B can mean A or B. The "and / or" in the text is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in the description of the embodiments of this application, "multiple" means two or more.

[0096] In some processes described in the embodiments of this application, multiple operations or steps are included in a specific order. However, it should be understood that these operations or steps may not be executed in the order they appear in the embodiments of this application, or they may be executed in parallel. The sequence number of the operation is only used to distinguish different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or fewer operations, and these operations or steps may be executed sequentially or in parallel, and these operations or steps may be combined.

[0097] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device to execute the methods described in the various embodiments of this application.

[0098] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for improving image resolution, characterized in that, The image resolution enhancement method includes: Obtain the known pixels and the pixels to be filled in the low-resolution image that needs to be upgraded; The low-resolution image is divided into regions, the environmental color features of each region are extracted, and the edge boundaries of the low-resolution image are detected. Based on the environmental color features of the region where the pixel to be filled is located in the low-resolution image, and the color values ​​of known pixels located in its neighborhood and on the same side of the same edge boundary, the filling value of the current pixel to be filled is calculated and the current pixel to be filled is filled, thereby achieving the resolution improvement of the low-resolution image.

2. The image resolution enhancement method according to claim 1, characterized in that, Obtain the known pixels and the pixels to be filled in the low-resolution image to be upgraded, including: Acquire the low-resolution image whose resolution needs to be upgraded, perform image noise reduction, color correction and white balance processing on the low-resolution image whose resolution needs to be upgraded, and set the target high-resolution size. The low-resolution image is enlarged to the target high-resolution size using interpolation to obtain the enlarged low-resolution image; In the enlarged low-resolution image, the pixels corresponding to the original pixel positions in the unenlarged low-resolution image are identified as known pixels, and the remaining pixels in the enlarged low-resolution image are identified as pixels to be filled.

3. The image resolution enhancement method according to claim 2, characterized in that, The step of partitioning the low-resolution image and extracting the environmental color features of each region includes: The enlarged low-resolution image is divided into multiple regions according to a preset size; Calculate the average lightness, the average first chromaticity, and the average second chromaticity of all pixels in each region in the Lab color space; The triplet obtained based on the average brightness, the first average chromaticity, and the second average chromaticity will be used as the environmental color feature of each region.

4. The image resolution enhancement method according to claim 3, characterized in that, The detection of the edge boundaries of the low-resolution image includes: An edge detection algorithm is used to process and extract the initial edge pixels of the magnified low-resolution image; Morphological processing is performed on the initial edge pixels to connect the broken edges and form a continuous edge boundary line.

5. The image resolution enhancement method according to claim 4, characterized in that, Before the environmental color features of the region where the pixel to be filled is located in the low-resolution image and the color values ​​of known pixels located in its neighborhood and on the same side of the same edge boundary, the following is included: Determine the neighborhood of the current pixel to be filled, centered on the current pixel to be filled. Obtain the Euclidean distance between the current pixel to be filled and the nearest edge boundary. Calculate the reliability of the current pixel to be filled based on the Euclidean distance. The specific calculation method is as follows: in, Indicates the current pixel to be filled. Reliability, Indicates the current pixel to be filled. The number of known pixels in the neighborhood. This represents the Euclidean distance between the current pixel to be filled and the nearest edge boundary. Represents a minimal constant; Sort all pixels to be filled from highest to lowest reliability, and determine the filling order of each pixel in turn, so that pixels with higher reliability are filled first.

6. The image resolution enhancement method according to claim 5, characterized in that, The filling value of the current pixel to be filled is calculated based on the environmental color features of the region where the pixel to be filled is located in the low-resolution image and the color values ​​of known pixels located in its neighborhood and on the same side of the same edge boundary. This includes: Known pixels located within the neighborhood of the current pixel to be filled and on the same side of the same edge boundary are selected as valid reference pixels. The fusion weight of each valid reference pixel relative to the current pixel to be filled is calculated as follows: in, Indicates valid reference pixels Relative to the fusion weight of the current pixel to be filled, Indicates spatial distance weights. Indicates color similarity weight, Indicates the boundary distance weight. This represents the spatial Euclidean distance between the valid reference pixel and the current pixel to be filled. Indicates the spatial attenuation coefficient. , Indicates valid reference pixels color value, This represents the average color of all valid reference pixels within the neighborhood of the pixel to be filled. Indicates the color attenuation coefficient. , This indicates the distance between the valid reference pixel and the nearest edge boundary. This represents the attenuation coefficient of the edge boundary line. ; Based on the fusion weights and the color values ​​of each effective reference pixel, a weighted average is calculated on the color values ​​of all effective reference pixels to obtain a local reference value. The specific calculation method is as follows: in, Indicates a local reference value. Indicates valid reference pixels The color value; Obtain the color smoothness of the region where the current pixel to be filled is located, and calculate the dynamic fusion coefficient of the current pixel to be filled. The specific calculation method is as follows: in, Represents the dynamic fusion coefficient. Indicates empirical weights, , This indicates the color smoothness of the area where the current pixel to be filled is located. Indicates the normalization factor; Based on the local reference value, the dynamic fusion coefficient of the current pixel to be filled, and the environmental color features of the region where the current pixel to be filled is located, the filling value of the current pixel to be filled is calculated. The specific calculation method is as follows: in, Indicates the current pixel to be filled. The fill value, It represents the environmental color characteristics of the area where the pixel to be filled is located.

7. The image resolution enhancement method according to claim 6, characterized in that, After filling the current pixels to be filled, thereby increasing the resolution of the low-resolution image, the process includes: Unsharpened mask sharpening is applied to edge boundaries in high-resolution images. Gaussian filtering noise reduction is performed on smooth regions in the high-resolution image where the color variance is below a preset threshold. The processed high-resolution image is output to the display device; The high-resolution image is a low-resolution image that has been upgraded.

8. An image resolution enhancement device, characterized in that, The image resolution enhancement device includes: The acquisition module is used to acquire known pixels and pixels to be filled in a low-resolution image whose resolution needs to be upgraded. The partitioning module is used to partition the low-resolution image, extract the environmental color features of each region, and detect the edge boundaries of the low-resolution image. The calculation module is used to calculate the fill value of the current pixel to be filled based on the environmental color features of the area where the current pixel to be filled is located in the low-resolution image and the color values ​​of known pixels located in its neighborhood and on the same side of the same edge boundary, and then fill the current pixel to be filled, thereby realizing the resolution improvement of the low-resolution image.

9. An image resolution enhancement device, characterized in that, The image resolution enhancement device includes a processor, a memory, and an image resolution enhancement program stored in the memory and executable by the processor, wherein when the image resolution enhancement program is executed by the processor, it implements the steps of the image resolution enhancement method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores an image resolution enhancement program, wherein when the image resolution enhancement program is executed by a processor, it implements the steps of the image resolution enhancement method as described in any one of claims 1 to 7.

Images