An image processing method, apparatus, device, medium and product

Abstract

The application provides an image processing method, device, equipment, medium and product, and relates to the technical field of image processing. The method is characterized in that: an intermediate resolution image is obtained by interpolating an original image based on a preset magnification, and then edge enhancement processing is performed on the intermediate resolution image. The method effectively improves the problem that the traditional interpolation method is poor in restoring image texture and edges and low in definition, avoids visual defects such as edge discontinuity, blocking and blurring caused by one-step interpolation to a target resolution, and performs end interpolation processing to obtain a target image when the resolution of the edge enhanced image meets a preset resolution condition. The method can effectively maintain the detail features of the original image in a high magnification scenario, improve the overall definition, and make the image edge transition natural.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image processing method, apparatus, device, medium and product. Background Technology

[0002] Currently, image upscaling is an important topic in digital image processing. Its goal is to convert low-resolution original images into higher-resolution images through interpolation methods, thereby meeting the needs of various application scenarios for image visual effects and detail presentation.

[0003] Existing technologies often employ bilinear interpolation, bicubic interpolation, and other methods to directly interpolate to the target resolution in one step. However, this approach not only fails to address the inherent problems of low clarity and poor edge reproduction in traditional interpolation methods, but also introduces new visual defects. The magnified image is prone to discontinuous and segmented edges, and the problem of blurred edges still exists, resulting in harsh edge transitions and poor overall visual effects. Summary of the Invention

[0004] This application provides an image processing method, apparatus, device, medium, and product to solve the problems in the prior art.

[0005] In a first aspect, this application provides an image processing method, comprising:

[0006] Obtain the original image to be processed;

[0007] The original image is interpolated based on a preset magnification ratio to obtain an intermediate resolution image; wherein the preset magnification ratio is an integer or non-integer magnification ratio greater than 1.

[0008] The intermediate resolution image is subjected to edge enhancement processing to obtain an edge-enhanced image;

[0009] Determine whether the resolution of the edge-enhanced image meets the preset resolution condition;

[0010] If the resolution of the edge-enhanced image meets the preset resolution condition, the edge-enhanced image is subjected to end interpolation processing to obtain the target image.

[0011] In one possible design, the interpolation process performed on the original image based on a preset magnification to obtain an intermediate resolution image includes:

[0012] Local gradient calculations are performed on the original image along multiple directions to obtain local gradient values ​​in each direction;

[0013] Based on the local gradient values ​​in each direction, the local orientation of the pixel region of the original image is determined to identify the local orientation corresponding to each pixel region.

[0014] Based on the pixel information in the local direction and the gradient change information between pixels, the pixel values ​​of the interpolation points in the original image are calculated to obtain the target pixel values ​​of each interpolation point.

[0015] Based on the target pixel value of each interpolation point and the preset magnification, the original image is interpolated to obtain the intermediate resolution image.

[0016] In one possible design, the pixel value calculation for the interpolation points of the original image includes:

[0017] The local gradient values ​​in each direction are compared with preset gradient thresholds to distinguish the flat areas, textured areas and edge areas of the original image.

[0018] Based on the weights corresponding to the flat region, the texture region, and the edge region, local pixels in the corresponding regions are selected to calculate the pixel values ​​of the interpolation points, thereby completing the pixel value calculation for the interpolation points in each region.

[0019] In one possible design, the edge enhancement processing of the intermediate resolution image to obtain an edge-enhanced image includes:

[0020] Obtain the current pixel value of the intermediate resolution image, and calculate the pixel difference between the current point and the surrounding points within the pixel block of the intermediate resolution image;

[0021] Based on the pixel difference, a weighted processing is performed using the weights corresponding to the multi-dimensional image feature information to obtain weighted difference data; wherein, the multi-dimensional image feature information includes frequency information, contrast information and brightness information;

[0022] The weighted difference data is superimposed on the current pixel value to obtain the enhanced pixel value of each pixel.

[0023] The intermediate resolution image is reconstructed based on the enhanced pixel values ​​of each pixel to obtain the edge-enhanced image.

[0024] In one possible design, the step of performing end-interpolation on the edge-enhanced image to obtain the target image includes:

[0025] Distinguish between the flat areas, textured areas, and edge areas of the edge-enhanced image;

[0026] Interpolation calculations are performed in the flat region and the textured region using an isotropic interpolation kernel function.

[0027] Interpolation calculations are performed in the edge region using an interpolation kernel function along the direction.

[0028] In one possible design, the interpolation calculation in the edge region using an interpolation kernel function along the direction includes:

[0029] The horizontal and vertical gradients are calculated for local pixels within the image block to be processed in the edge region, and the Hessian matrix corresponding to the image block to be processed is obtained based on the horizontal and vertical gradients.

[0030] The Hessian matrix is ​​subjected to eigenvalue decomposition to obtain the eigenvectors corresponding to the Hessian matrix;

[0031] The interpolation kernel's stretching strength along the edge direction, the interpolation kernel's contraction strength perpendicular to the edge direction, and the distance vector from the pixel point within the image block to the point to be interpolated are obtained to obtain the interpolation calculation feature parameter set.

[0032] Based on the eigenvectors corresponding to the Hessian matrix and the interpolation calculation feature parameter set, the interpolation weights of each pixel in the image block to be processed are solved to obtain the interpolation weights corresponding to each pixel.

[0033] Based on the interpolation weights corresponding to each pixel, an interpolation kernel function along the direction is used to perform interpolation operations on the edge region to complete the interpolation kernel function interpolation calculation of the edge region along the direction.

[0034] Secondly, this application provides an image processing apparatus, comprising:

[0035] The acquisition module is used to acquire the original image to be processed;

[0036] The first interpolation module is used to interpolate the original image based on a preset magnification ratio to obtain an intermediate resolution image; wherein the preset magnification ratio is an integer or non-integer magnification ratio greater than 1.

[0037] An image enhancement module is used to perform edge enhancement processing on the intermediate resolution image to obtain an edge-enhanced image;

[0038] The judgment module is used to determine whether the resolution of the edge enhancement image meets the preset resolution condition;

[0039] The second interpolation module is used to perform end interpolation processing on the edge enhancement image to obtain the target image, provided that the resolution of the edge enhancement image meets the preset resolution condition.

[0040] Thirdly, this application provides an electronic device, including: a processor, and a memory communicatively connected to the processor;

[0041] The memory stores computer-executed instructions;

[0042] The processor executes computer execution instructions stored in the memory to implement the method as described in any of the first aspects.

[0043] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any of the first aspects.

[0044] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the method described in any of the first aspects.

[0045] This application provides an image processing method, apparatus, device, medium, and product. The method first interpolates the original image based on a preset magnification to obtain an intermediate resolution image, and then performs edge enhancement processing on the intermediate resolution image. This effectively improves the problems of poor image texture and edge restoration and low clarity caused by traditional interpolation methods, and avoids visual defects such as discontinuous edges, blockages, and blurring caused by one-step interpolation to the target resolution. In addition, when the resolution of the edge-enhanced image meets the preset resolution condition, the final interpolation processing is performed to obtain the target image. This ensures that the image can effectively maintain the details of the original image in high-magnification scenarios, improve the overall clarity, and make the image edge transition natural, resulting in a good overall visual effect. It can be adapted to application scenarios with stringent requirements for image edge performance and clarity, such as mobile terminals, autonomous driving, and medical imaging. Attached Figure Description

[0046] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0047] Figure 1 This is an application scenario diagram corresponding to an image processing method provided in an embodiment of this application;

[0048] Figure 2 A schematic flowchart of an image processing method provided in an embodiment of this application;

[0049] Figure 3 A schematic flowchart of an image processing method provided in another embodiment of this application;

[0050] Figure 4 This is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of this application;

[0051] Figure 5This is a structural example diagram of an electronic device provided in an embodiment of this application.

[0052] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0053] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0054] To clearly understand the technical solution of this application, the solutions of the prior art will be described in detail first.

[0055] Image magnification, a crucial topic in digital image processing, aims to convert low-resolution original images into higher-resolution images using interpolation methods, thereby meeting the demands for improved visual effects and detail reproduction in various application scenarios. Currently, traditional interpolation methods suffer from poor texture and edge reproduction when magnifying images, resulting in generally lower overall clarity and an inability to effectively preserve the details of the original image. Furthermore, this deficiency intensifies with increasing magnification; the higher the magnification, the more severe the loss of image details and blurring, making it difficult to meet the high-resolution requirements of high-magnification scenarios.

[0056] In pursuit of direct and efficient image magnification, existing technologies often employ bilinear interpolation, bicubic interpolation, and other methods to directly interpolate to the target resolution in one step. However, this approach not only fails to address the inherent problems of low clarity and poor edge reproduction in traditional interpolation methods but also introduces new visual defects. The magnified image is prone to discontinuous and fragmented edges, and the problem of blurred edges still exists, resulting in harsh edge transitions and poor overall visual effects. This makes it unsuitable for applications such as mobile terminals, autonomous driving, and medical imaging, which have stringent requirements for image edge performance and clarity.

[0057] Figure 1 An application scenario diagram corresponding to an image processing method provided in an embodiment of this application is shown, such as... Figure 1As shown, the application scenario provided in this embodiment includes: image source device 10, image processing device 11 and target terminal 12. Image source device 10 and image processing device 11 interact through data transmission bus or wireless communication link, and image processing device 10 and target terminal 12 establish connection through adapter interface.

[0058] Specifically, the image source device 10 acquires the low-resolution original image to be processed (such as a low-resolution photo taken by a mobile phone, a low-resolution road condition map captured by a vehicle camera, or a low-resolution medical scan image), and transmits the original image data to the image processing device 11. The image processing device 11 performs interpolation processing on the original image based on a preset magnification (an integer or non-integer greater than 1) to generate an intermediate resolution image, avoiding edge defects caused by one-step interpolation to the target resolution. Subsequently, the image processing device 11 performs edge enhancement processing on the intermediate resolution image to enhance image texture and edge features, making up for the detail loss problem of traditional interpolation, and obtaining an edge-enhanced image. Then, the image processing device 11 detects in real time whether the resolution of the edge-enhanced image meets the preset resolution condition. If it is determined to meet the condition, the edge-enhanced image is subjected to end interpolation processing to accurately magnify to the target resolution, generating a target image with continuous edges and clear details. Finally, the image processing device 11 outputs the target image to the corresponding target terminal 12 to meet the visual effect and detail presentation requirements of high-resolution images in various scenarios.

[0059] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0060] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or modules, and may be electrical, mechanical, or other forms.

[0061] Figure 2 This is a schematic flowchart of an image processing method provided in an embodiment of this application, as shown below. Figure 2As shown, the execution subject of this embodiment is an image processing device. This device can be implemented by a computer program, or by a medium storing the relevant computer program, such as a USB flash drive and / or optical disc; alternatively, it can be implemented by a physical device integrating or installing the relevant computer program, such as a chip or electronic device. The electronic device may be a computer or a server, etc. The image processing method provided in this embodiment includes the following steps:

[0062] S201. Obtain the original image to be processed.

[0063] In this step, the original image to be processed is a low-resolution image. The original image can come from application scenarios such as high-definition display, machine vision, and medical imaging. Specifically, it can be acquired by image acquisition devices (such as cameras, CT scanners, ultrasound imaging devices, etc.), or it can be read from a storage device to read stored low-resolution image data, or it can be received from externally transmitted low-resolution image data through a data transmission link.

[0064] The original image format includes, but is not limited to, JPEG, PNG, and TIFF. The resolution of the original image is determined based on the actual application scenario and is not specifically limited, as long as it meets the condition of "low resolution that cannot directly meet the subsequent display / analysis / recognition requirements". In addition, the original image can be a grayscale image or a color image. If it is a color image, it can be converted to a color space such as YUV or Lab first. The image processing steps of this application are performed on the luminance channel (such as the Y channel and L channel), and the chrominance channel is processed using conventional interpolation methods to balance processing efficiency and color fidelity.

[0065] Optionally, after acquiring the original image, preprocessing operations can be performed on the original image. Preprocessing operations include, but are not limited to, noise removal (such as using Gaussian filtering, median filtering, etc. to remove salt-and-pepper noise and Gaussian noise in the original image to avoid noise affecting subsequent interpolation and edge enhancement effects) and grayscale correction (such as adjusting the grayscale value range of the original image to make the grayscale value distribution uniform and improve image contrast). Preprocessing operations can be flexibly selected according to the quality of the original image.

[0066] S202. Interpolate the original image based on a preset magnification ratio to obtain an intermediate resolution image; wherein the preset magnification ratio is an integer or non-integer magnification ratio greater than 1.

[0067] This step is the first interpolation process, which aims to initially improve the resolution of the original image to obtain an intermediate resolution image. By reasonably setting the preset magnification, the blurring defects caused by a single high-magnification interpolation are avoided.

[0068] The preset magnification factor is an integer or non-integer magnification factor greater than 1, ranging from 1.2 to 3.0 times (e.g., 1.5x, 1.8x, 3.0x). The selection of this preset magnification factor is based on the initial resolution and target resolution of the original image. A preset magnification factor that is too large (e.g., exceeding 3.0x) will cause significant blurring and artifacts in a single interpolation, which cannot be completely corrected by subsequent edge enhancement steps. A preset magnification factor that is too small (e.g., close to 1.0x) will increase the number of subsequent interpolation steps, reducing processing efficiency. Therefore, a preset magnification factor of 1.2 to 3.0x strikes a balance between interpolation quality and processing efficiency.

[0069] S203. Perform edge enhancement processing on the intermediate resolution image to obtain an edge-enhanced image.

[0070] The purpose of this step is to promptly correct defects such as edge blurring and detail distortion that occur during the S202 interpolation process, enhance the edge contrast and detail clarity of the intermediate resolution image, lay the foundation for subsequent interpolation steps, and prevent defects from being further amplified.

[0071] Optionally, during the edge enhancement process, a Gaussian filtering algorithm can be combined for denoising. While enhancing edges and details, noise generated during edge enhancement is removed to ensure the clarity and purity of the edge-enhanced image. The intensity of edge enhancement can be flexibly adjusted according to the quality of the intermediate resolution image. If the edges of the intermediate resolution image are severely blurred, the enhancement intensity can be appropriately increased. If the intermediate resolution image has a lot of noise, the enhancement intensity can be appropriately reduced to balance the enhancement and denoising effects.

[0072] The edge-enhanced image obtained in this step has better edge sharpness and detail clarity than the intermediate resolution image obtained in S202, and has corrected the blurring defects generated during the S202 interpolation process, providing a high-quality image foundation for subsequent interpolation steps.

[0073] S204. Determine whether the resolution of the edge enhancement image meets the preset resolution conditions.

[0074] The purpose of this step is to flexibly adjust the interpolation process to avoid over- or under-interpolation and ensure that the final target image meets the needs of practical applications.

[0075] The preset resolution condition is the minimum resolution standard required for the target application scenario, which is determined based on the actual application scenario. For example, the preset resolution condition for a high-definition display scenario can be 1920×1080 pixels (1080P), the preset resolution condition for a medical imaging scenario (such as CT images) can be 2048×2048 pixels, and the preset resolution condition for a machine vision scenario (such as target recognition) can be 1280×720 pixels. The preset resolution condition can be flexibly set by the user according to actual needs, without specific restrictions.

[0076] Optionally, the resolution parameters (horizontal pixels × vertical pixels) of the edge enhancement image are read, and the resolution parameters are compared with the preset resolution conditions. If the resolution of the edge enhancement image is greater than or equal to the resolution corresponding to the preset resolution conditions, it can be determined that the preset resolution conditions are met; if the resolution of the edge enhancement image is less than the resolution corresponding to the preset resolution conditions, it can be determined that the preset resolution conditions are not met.

[0077] Furthermore, if it is determined that the preset resolution condition is not met, the process returns to step S202 and re-interpolates the current edge-enhanced image based on the preset magnification. At this time, the input image is the edge-enhanced image, not the original image, resulting in a new intermediate resolution image. Then, step S203 is executed to perform edge enhancement processing, followed by step S204 to determine the resolution again. This process is repeated until the resolution of the edge-enhanced image meets the preset resolution condition, at which point the process proceeds to step S205. During the loop, the preset magnification can remain unchanged or be flexibly adjusted according to the resolution of the current image. For example, if the resolution is close to the preset resolution condition, the preset magnification can be appropriately reduced to avoid over-interpolation.

[0078] S205. If the resolution of the edge enhancement image meets the preset resolution condition, perform end interpolation processing on the edge enhancement image to obtain the target image.

[0079] The purpose of this step is to perform precise interpolation based on the edge-enhanced image so that the resolution of the target image exactly matches the preset resolution conditions, while further optimizing the image quality to ensure that the target image has no obvious defects.

[0080] Optionally, the end interpolation process can adopt an adaptive interpolation strategy, that is, first identify the flat area, texture area and edge area in the edge enhancement image, and then use different interpolation algorithms for different areas to ensure that no new artifacts are introduced during the end interpolation process, and that the edges and details are not blurred, while ensuring that the resolution of the target image accurately matches the preset requirements.

[0081] Optionally, after the interpolation at the end is completed, post-processing operations can be performed on the obtained target image. Post-processing operations include, but are not limited to, contrast adjustment, artifact removal (using an adaptive filtering algorithm to remove slight artifacts that may be generated during the interpolation at the end), and grayscale normalization (adjusting the grayscale value range of the target image to adapt it to subsequent display, analysis, or recognition devices). Post-processing operations can be flexibly selected according to the actual application scenario.

[0082] The image processing method provided in this application first interpolates the original image based on a preset magnification to obtain an intermediate resolution image, and then performs edge enhancement processing on the intermediate resolution image. This effectively improves the problems of poor image texture and edge restoration and low clarity caused by traditional interpolation methods, and avoids visual defects such as discontinuous edges, blockiness and blurring caused by one-step interpolation to the target resolution. In addition, when the resolution of the edge-enhanced image meets the preset resolution condition, the final interpolation processing is performed to obtain the target image. This ensures that the image can effectively maintain the details of the original image in high-magnification scenarios and improve the overall clarity, while also making the image edge transition natural and the overall visual effect good. It can be adapted to application scenarios with strict requirements for image edge performance and clarity, such as mobile terminals, autonomous driving, and medical imaging.

[0083] As an optional implementation, based on any of the above embodiments, interpolation processing is performed on the original image based on a preset magnification to obtain an intermediate resolution image, including the following steps:

[0084] First, local gradient calculations are performed on the original image along multiple directions to obtain the local gradient values ​​in each direction.

[0085] The purpose of this step is to capture the intensity of grayscale changes of each pixel in the original image in different directions, providing basic data support for subsequent local direction determination and pixel value calculation of the interpolation point. Among them, the local gradient value is used to characterize the rate of grayscale change of a pixel in a certain direction. The larger the gradient value, the more drastic the pixel grayscale change in that direction; the smaller the gradient value, the more gradual the pixel grayscale change in that direction.

[0086] Specifically, first, determine the multiple directions to be calculated. These multiple directions can be eight: 0° (horizontal to the right), 45° (upper right to lower left), 90° (vertical upward), 135° (upper left to lower right), 180° (horizontal to the left), 225° (lower left to upper right), 270° (vertical downward), and 315° (lower right to upper left). These eight directions can comprehensively cover all possible directions of pixel change in the image, ensuring the comprehensiveness and accuracy of local gradient calculation. Alternatively, other numbers of directions, such as four or six, can be flexibly selected based on the characteristics of the original image.

[0087] Optionally, local gradient calculation can be implemented using a gradient operator, such as the Sobel operator, Prewitt operator, or Roberts operator, or other conventional gradient calculation algorithms. The specific choice depends on the noise level of the original image and the required gradient calculation accuracy. For example, the Sobel operator is used when the original image has a lot of noise; the Prewitt operator can be used when higher gradient calculation accuracy is required.

[0088] Optionally, taking each pixel in the original image as the center, a local neighborhood of a preset size (such as a 3×3 neighborhood or a 5×5 neighborhood) is selected, and the gradient value of the pixel in the above multiple directions is calculated using the gradient operator to obtain multiple local gradient values ​​corresponding to each pixel. For pixels at the edge of the original image (whose local neighborhood exceeds the image range), the neighborhood pixel values ​​are supplemented by an edge completion strategy (such as zero filling, mirror completion, or copy completion) before local gradient calculation is performed to avoid deviations in the gradient calculation of edge pixels.

[0089] For example, when using the Sobel operator to calculate the local gradient value of a pixel in the 0° direction (horizontally to the right), the Sobel operator template in the horizontal direction is convolved with the 3×3 local neighborhood of the pixel to obtain the gradient value in that direction; similarly, the gradient values ​​of the pixel in the other 7 directions are calculated in turn to obtain the 8 local gradient values ​​of the pixel.

[0090] Secondly, based on the local gradient values ​​in each direction, the local orientation of the pixel regions in the original image is determined to identify the local orientation corresponding to each pixel region.

[0091] The purpose of this step is to determine the dominant change direction (i.e., local direction) of each pixel region based on the obtained local gradient values ​​in each direction. This local direction is the direction in which the pixel grayscale changes most drastically, corresponding to features such as the edge direction and texture extension direction of pixels in the image, providing a directional basis for the accurate calculation of the pixel values ​​of the subsequent interpolation points.

[0092] In this context, a pixel region can be defined as a single pixel, determining the local direction pixel by pixel, or it can be defined as a pixel block of a preset size, such as a 2×2 pixel block or a 3×3 pixel block. This is suitable for areas with relatively smooth textures and can reduce computational complexity. The specific choice depends on the detail density of the original image and the processing efficiency requirements. For densely detailed areas (such as textured areas and edge areas), a single pixel can be used as the pixel region to ensure the accuracy of local direction determination; for areas with relatively smooth details (such as flat areas), pixel blocks can be used as the pixel region to improve processing efficiency.

[0093] Optionally, for each pixel region, the local gradient values ​​of all pixels in the pixel region in multiple directions are extracted, and the average gradient value corresponding to each direction is calculated. If it is a single pixel, the local gradient value of the pixel is the average gradient value of the corresponding direction. The average gradient values ​​of multiple directions are compared, and the direction with the smallest average gradient value is selected as the local direction corresponding to the pixel region.

[0094] Furthermore, to avoid local gradient value anomalies caused by noise affecting the accuracy of local direction determination, local gradient values ​​can be filtered before calculating the average gradient value, such as by using median filtering or mean filtering to remove abnormal gradient values. In addition, a gradient threshold can be set. If the average gradient value in all directions is less than the gradient threshold, it indicates that the pixel area is a flat area with no obvious gray-level change direction. Its local direction can be set to any direction (such as the default 0° direction), and then a simple interpolation algorithm can be used to calculate the pixel value of the point to be interpolated.

[0095] For example, the local gradient values ​​of a certain pixel region (a single pixel) in eight directions are as follows: the local gradient value in the 0° direction is 5; the local gradient value in the 45° direction is 28; the local gradient value in the 90° direction is 7; the local gradient value in the 135° direction is 32; the local gradient value in the 180° direction is 4; the local gradient value in the 225° direction is 25; the local gradient value in the 270° direction is 6; and the local gradient value in the 315° direction is 29. After comparison, it can be seen that the gradient value in the 180° direction is the smallest (4). Therefore, the local direction of this pixel region is determined to be 180°, indicating that the direction of the smoothest grayscale change of this pixel is 180°, which corresponds to the direction of a certain edge in the image being 180°.

[0096] Next, based on the pixel information in the local direction and the gradient change information between pixels, the pixel values ​​of the interpolation points in the original image are calculated to obtain the target pixel values ​​of each interpolation point.

[0097] The purpose of this step is to accurately calculate the target pixel value of the point to be interpolated based on a determined local direction, combined with the original pixel information and gradient change rules in the local direction, to avoid edge blurring and texture distortion, ensure that the pixel value of the point to be interpolated is consistent with the change rules of the surrounding pixels, and improve the detail fidelity of the interpolated image.

[0098] Optionally, firstly, based on a preset magnification, the size of the interpolated intermediate resolution image is calculated. The size difference between the original image and the intermediate resolution image is compared to determine the number of interpolation points to be added and the corresponding position of each interpolation point in the original image, i.e., the distribution of original pixels around the interpolation point. The interpolation points are evenly distributed and together with the original pixels constitute the pixel matrix of the intermediate resolution image. Secondly, for each interpolation point, a pixel region within a preset range (e.g., a 3×3 neighborhood, a 5×5 neighborhood) is determined. The local directions corresponding to these pixel regions are extracted, and the dominant local direction corresponding to the interpolation point is comprehensively determined, i.e., the dominant direction of grayscale change in the region where the interpolation point is located. The original pixel information (including the grayscale value and position coordinates of the original pixels) and the gradient change information between these original pixels (i.e., the gradient difference and gradient change trend between adjacent original pixels) are extracted along this dominant local direction.

[0099] Then, based on the original pixel grayscale values ​​in the dominant local direction and combined with the gradient change information between pixels, a linear or nonlinear interpolation algorithm is used to calculate the target pixel value of the point to be interpolated. The gradient change information is used to adjust the interpolation weights so that the target pixel value better matches the local grayscale change pattern. In regions with drastic gradient changes (such as edge regions), the weight difference between adjacent original pixels is increased to ensure edge sharpness; in regions with gentle gradient changes (such as textured or flat regions), the weight difference between adjacent original pixels is decreased to ensure the smoothness and continuity of details.

[0100] Optionally, the pixel value of the point to be interpolated is calculated using the following interpolation formula:

[0101]

[0102] Where p is the target pixel value of the point to be interpolated; , These are adjustable weights that can be adjusted according to the characteristics of the image region; is the average pixel value in the local direction; g is the pixel gradient value in that local direction.

[0103] Optionally, if there are multiple local directions in the area where the interpolation point is located (such as the edge intersection area), the original pixel information and gradient change information of each local direction are extracted, the candidate pixel value corresponding to each local direction is calculated, and then different weights are assigned according to the magnitude of the gradient value of each local direction. The candidate pixel values ​​are then weighted and summed to obtain the final target pixel value, ensuring the interpolation effect in the intersection area.

[0104] Finally, based on the target pixel values ​​of each interpolation point and the preset magnification, the original image is interpolated to obtain an intermediate resolution image.

[0105] The purpose of this step is to fuse the calculated target pixel values ​​of all interpolation points with the original pixel values ​​of the original image, construct a complete intermediate resolution image according to a preset magnification, and complete this interpolation process.

[0106] Optionally, a blank pixel matrix (with the same size as the intermediate resolution image) is constructed according to the intermediate resolution image size determined by a preset magnification; the original pixels of the original image are filled into the corresponding coordinates of the blank pixel matrix according to their corresponding positions; the calculated target pixel values ​​of each interpolation point are filled into the corresponding interpolation positions in the blank pixel matrix; after filling, a complete intermediate resolution image is obtained.

[0107] Optionally, after the filling is completed, a simple smoothing process can be performed on the intermediate resolution image (such as using Gaussian filtering with a filter kernel size of 2×2) to remove abrupt changes in grayscale between the interpolation point and the original pixel point, thereby improving the overall smoothness of the intermediate resolution image; if the grayscale distribution of the intermediate resolution image is uneven, simple grayscale adjustments can be made.

[0108] The image processing method provided in this application calculates local gradient values ​​along multiple directions of the original image to accurately determine the local direction of each pixel region. Then, it calculates the pixel value of the point to be interpolated based on the pixel and gradient change information in the local direction. This can better match the texture and edge direction of the image itself, effectively avoiding the edge blurring and detail loss problems caused by traditional interpolation methods that do not consider local characteristics. The resulting intermediate resolution image is clearer and more accurate in terms of texture and edge presentation.

[0109] As an optional implementation, based on any of the above embodiments, the pixel values ​​of the interpolation points in the original image are calculated, including the following steps:

[0110] First, the local gradient values ​​in each direction are compared with preset gradient thresholds to distinguish the flat areas, textured areas and edge areas of the original image.

[0111] The purpose of this step is to achieve accurate segmentation of image regions, providing a foundation for subsequent differential weight settings and pixel value calculation. Specifically, pixel gray values ​​in edge regions change abruptly, with the largest local gradient value; pixel gray values ​​in textured regions change regularly, with the local gradient value in the middle; and pixel gray values ​​in flat regions change gradually, with the smallest local gradient value.

[0112] Optionally, the local gradient values ​​in each direction of each pixel region (a single pixel or a pixel block of a preset size) are first extracted, and the maximum gradient value of each pixel region is calculated, which is the maximum value of the local gradient values ​​in all calculation directions of the pixel region. This maximum gradient value is used to characterize the degree of grayscale change in the pixel region. Subsequently, two gradient thresholds are preset, namely the first gradient threshold (low threshold) and the second gradient threshold (high threshold), wherein the first gradient threshold is less than the second gradient threshold. The specific values ​​of the two gradient thresholds can be flexibly adjusted according to the quality of the original image, the detail density and the actual application scenario, without fixed restrictions.

[0113] Optionally, if the maximum gradient value of a pixel region is less than the first gradient threshold, it indicates that the pixel grayscale value changes smoothly in the region, with no obvious details or edges, and is determined to be a flat region; if the maximum gradient value of a pixel region is greater than or equal to the first gradient threshold and less than or equal to the second gradient threshold, it indicates that the pixel grayscale value in the region changes regularly and has continuous texture details, and is determined to be a textured region; if the maximum gradient value of a pixel region is greater than the second gradient threshold, it indicates that the pixel grayscale value in the region changes abruptly and has obvious edge features, and is determined to be an edge region.

[0114] Furthermore, to avoid gradient value anomalies caused by noise affecting the accuracy of region division, local gradient values ​​in each direction of a pixel region can be filtered (e.g., median filtering, mean filtering) before calculating the maximum gradient value of that pixel region to remove abnormal gradient values. In addition, for "transition regions" that appear during region division—where adjacent pixel regions belong to different types of regions—a neighborhood voting method can be used for correction. The region type with the highest proportion in the surrounding 3×3 neighborhood is selected as the final type of the transition region, ensuring the continuity and integrity of region division and avoiding region fragmentation.

[0115] For example, if the first gradient threshold is preset to 8 and the second gradient threshold is preset to 25, and the maximum local gradient value of a certain pixel region (a single pixel) is 5 in each direction, then since 5 < 8, the pixel region is determined to be a flat region; if the maximum gradient value of a certain pixel region is 18, then since 8 ≤ 18 ≤ 25, the pixel region is determined to be a textured region; if the maximum gradient value of a certain pixel region is 30, then since 30 > 25, the pixel region is determined to be an edge region.

[0116] Optionally, the Harris corner detection algorithm can be used to distinguish flat areas, textured areas, and edge areas in the original image. Specifically, the Harris response value of each pixel region is calculated to quantify the corner features and grayscale variation characteristics of the pixel region. A double threshold is preset for the Harris response value, and the flat, textured, and edge regions of the original image are divided based on the comparison between the Harris response value and the double threshold. The Harris response value accurately characterizes the degree of grayscale variation and corner features of a pixel region, providing a quantitative basis for the division of flat, textured, and edge regions. A pixel region can be a single pixel or a 3×3 pixel block of a preset size. Image regions with dense details (such as suspected edge or textured areas) are calculated using a single pixel to reduce region division errors; image regions with smooth details are calculated using pixel blocks to reduce computation and improve processing efficiency.

[0117] Secondly, based on the weights corresponding to the flat area, texture area, and edge area, local pixels in the corresponding areas are selected to calculate the pixel values ​​of the interpolation points, so as to complete the pixel value calculation of the interpolation points in each area.

[0118] The purpose of this step is to set differentiated weights for pixel features in different regions, select suitable local pixels for interpolation calculations, and ensure the accuracy of pixel values ​​at the interpolation points in each region. Specifically, edge regions should prioritize edge sharpness and continuity, textured regions should prioritize preserving texture details, and flat regions should prioritize ensuring smoothness and the absence of artifacts.

[0119] Specifically, based on the pixel features and interpolation requirements of each region, corresponding interpolation weights are set for the flat region, texture region, and edge region, with weight values ​​ranging from 0 to 1. The sum of the weights of all local pixels participating in the calculation around the same interpolation point is 1, ensuring the rationality and standardization of the interpolation calculation. The core requirement for the edge region is to ensure edge sharpness and continuity, therefore the highest weight is set, and local pixels from the edge region are used in the calculation of the pixel value of the interpolation point; the weight value range can be 0.6 to 0.8. The core requirement for the texture region is to preserve texture details, so a medium weight is set to balance detail preservation and calculation stability; the weight value range can be 0.3 to 0.5. The core requirement for the flat region is to ensure smoothness and no artifacts, so the lowest weight is set to avoid introducing unnecessary grayscale abrupt changes; the weight value range can be 0.1 to 0.2. It should be noted that the weight values ​​can be flexibly adjusted according to the region where the interpolation point is located. If the interpolation point is located near the edge region, the weight of local pixels in the edge region should be further increased and the weight of flat regions should be decreased. If the interpolation point is located in the center of the texture region, the weight of the texture region can be appropriately increased. If the interpolation point is located in the flat region, only local pixels in the flat region can be used for calculation, simplifying the process and ensuring smoothness.

[0120] Optionally, firstly, determine the computational neighborhood of each interpolation point (a local neighborhood of a preset size, such as a 3×3 or 5×5 neighborhood); then, identify all local pixels within this computational neighborhood, and distinguish local pixels belonging to flat regions, textured regions, and edge regions based on the region types defined above; finally, select local pixels to participate in the calculation based on the weights corresponding to each region. Prioritize selecting local pixels from regions with higher weights. If the number of local pixels from regions with higher weights is insufficient, such as when the interpolation point is located in a flat region and there are no edge or textured region pixels in the neighborhood, then select local pixels from the core region to participate in the calculation, ensuring the effectiveness of the calculation.

[0121] For example, if the point to be interpolated is located near the edge region, the calculation neighborhood is a 3×3 neighborhood, containing 5 edge region pixels, 3 texture region pixels, and 1 flat region pixel. If the weights of the edge region are set to 0.7, the texture region to 0.2, and the flat region to 0.1, then all 5 edge region pixels, 3 texture region pixels, and 1 flat region pixel are selected to participate in the calculation, and the calculation weights are allocated according to their corresponding weights.

[0122] Optionally, based on the grayscale value of the selected local pixels and the corresponding region weight, a linear interpolation algorithm or a nonlinear interpolation algorithm is used to calculate the target pixel value of the point to be interpolated.

[0123] The image processing method provided in this application accurately distinguishes between flat areas, textured areas, and edge areas of the original image by comparing local gradient values ​​with preset gradient thresholds, and assigns corresponding weights to different areas to select local pixels for calculating the pixel values ​​of the points to be interpolated. Thus, it fully considers the characteristics of different regions of the image, avoiding unnecessary complex calculations in flat areas, better preserving texture details in textured areas, and effectively enhancing edge clarity in edge areas, thereby specifically improving the interpolation effect in each region.

[0124] As an optional implementation, based on any of the above embodiments, edge enhancement processing is performed on the intermediate resolution image to obtain an edge-enhanced image, including the following steps:

[0125] First, obtain the current pixel value of the intermediate resolution image, and calculate the pixel difference between the current point and the surrounding points within the pixel block of the intermediate resolution image.

[0126] The purpose of this step is to obtain the basic features of the current pixel and the overall gray level of the surrounding pixels, directly quantify the gray level difference between the current pixel and the surrounding pixels, provide accurate basic data for subsequent weighted processing, and ensure the effectiveness of the gray level difference representation.

[0127] Optionally, first traverse each pixel of the intermediate resolution image and obtain the current pixel value of each pixel one by one. The current pixel value is the original grayscale value of the pixel without enhancement processing. Then, take each current pixel as the center and select a pixel block of a preset size as the calculation neighborhood (such as a 3×3 pixel block). This pixel block is the surrounding pixel area of ​​the current pixel. Finally, count the grayscale values ​​of all surrounding pixels in the pixel block except the current pixel, calculate their arithmetic mean, obtain the pixel value of the surrounding points in the pixel block, and directly calculate the pixel difference between the current pixel value and the pixel value.

[0128] Optionally, for pixels at the edge of an intermediate resolution image (whose pixel blocks extend beyond the image area), an edge completion strategy (such as mirror completion or copy completion) is used to fill in the missing pixel values ​​around the image before calculating the pixel difference. This avoids deviations in pixel value calculation and ensures consistent processing of all pixels in the image. If the image has slight noise, median filtering can be applied to the grayscale values ​​of surrounding pixels within the pixel block before calculating the pixel value to remove noise interference and ensure the authenticity of the pixel value.

[0129] Furthermore, a difference threshold can be set (e.g., 5~10, which can be flexibly adjusted according to image quality). If the absolute value of the difference between the current pixel value and the surrounding pixels is less than the difference threshold, it indicates that the grayscale difference between the current pixel and the surrounding pixels is small, and it is likely a pixel in a flat area, thus avoiding over-enhancing the flat area and introducing artifacts. If the absolute value of the difference between the current pixel value and the surrounding pixel value is greater than or equal to the difference threshold, it indicates that the current pixel is likely an edge or texture detail pixel, which needs to be enhanced to ensure the sharpness of edges and details.

[0130] Alternatively, the pixel difference can be calculated using the following formula:

[0131]

[0132] Where diff is the pixel difference; , Adjustable weights for edge enhancement; The pixel value of the current point; The pixel value is the value of the surrounding points within the pixel block.

[0133] Alternatively, the maximum and minimum pixel values ​​of the points surrounding the current point within a pixel block of an intermediate resolution image can be calculated. Based on the current pixel value and the maximum and minimum pixel values ​​of the surrounding points, a pixel difference value is calculated, and then fused using adjustable weights to obtain the pixel difference.

[0134] Next, based on the pixel difference, the weights corresponding to the multi-dimensional image feature information are used for weighted processing to obtain weighted difference data; among which, the multi-dimensional image feature information includes frequency information, contrast information and brightness information.

[0135] This step combines multi-dimensional image feature information to assign differentiated weights. It can distinguish high-frequency details from low-frequency regions through frequency information, distinguish high-contrast edges from low-contrast textures through contrast information, and adapt to the enhancement needs of different brightness regions through brightness information, thus avoiding edge blurring, texture distortion, or artifacts caused by single-dimensional enhancement.

[0136] Specifically, firstly, multi-dimensional image feature information (frequency information, contrast information, brightness information) corresponding to the current pixel is extracted; then, corresponding weights are assigned to the feature information of each dimension, and the pixel difference is weighted based on the weights to obtain weighted difference data.

[0137] Optionally, Fourier transform or wavelet transform algorithms are used to extract the frequency features of the pixel block where the current pixel is located to obtain frequency information. The frequency information is used to distinguish between high-frequency and low-frequency regions. High-frequency regions correspond to the edges, textures and other details of the image, while low-frequency regions correspond to the flat areas of the image. The frequency value is normalized to a range of 0 to 1. The closer the frequency value is to 1, the more likely the current pixel is to be located in a high-frequency detail region.

[0138] Optionally, with the current pixel as the center, the standard deviation of the gray values ​​of all pixels in the pixel block is calculated, and the standard deviation is normalized to serve as the contrast information. The contrast information is used to characterize the gray-level fluctuation of the area surrounding the current pixel. The larger the contrast value, the more intense the gray-level fluctuation of the surrounding pixels, corresponding to the edge or dense texture area. The smaller the contrast value, the smoother the gray-level of the surrounding pixels, corresponding to the flat area. The range of the contrast value is normalized to 0~1.

[0139] Optionally, the current pixel value of the current pixel is normalized to obtain brightness information. The brightness information is used to adapt to the enhancement needs of different brightness areas, avoiding overexposure due to excessive enhancement in bright areas and loss of detail due to insufficient enhancement in low-brightness areas. The value range of the brightness information is normalized to 0~1. The closer the brightness information is to 1, the higher the brightness of the current pixel. The closer the brightness information is to 0, the lower the brightness of the current pixel.

[0140] Optionally, based on the physical meaning of the multi-dimensional image feature information, corresponding weights are assigned to frequency information, contrast information, and brightness information respectively. The weight values ​​range from 0 to 1, and the sum of the weights of the three dimensions is 1.

[0141] Frequency information weights are used to enhance high-frequency detail areas, with a weight value ranging from 0.4 to 0.6; contrast information weights are used to enhance the enhancement effect of high-contrast edges, with a weight value ranging from 0.2 to 0.3; and brightness information weights are used to balance the enhancement effect of different brightness areas, with a weight value ranging from 0.1 to 0.3.

[0142] Then, the weighted difference data is superimposed on the current pixel value to obtain the enhanced pixel value of each pixel.

[0143] The purpose of this step is to enhance the grayscale of the current pixel by superimposing weighted difference data, thereby strengthening edge and texture details and avoiding over-enhancement.

[0144] Specifically, the weighted difference data obtained in the above steps is superimposed on the current pixel value to obtain the enhanced pixel value of each pixel.

[0145] Optionally, to avoid the enhanced pixel value from exceeding the reasonable range of image grayscale values, the enhanced pixel value is subjected to amplitude limiting: if the enhanced pixel value is >255, the enhanced pixel value is truncated to 255; if the enhanced pixel value is <0, the enhanced pixel value is truncated to 0; if the enhanced pixel value is in the range of 0~255, it remains unchanged, ensuring that the enhanced pixel value is legal and without distortion.

[0146] Optionally, if the enhanced pixel value is greater than the maximum value within the pixel block, the enhanced pixel value is truncated to the maximum value within the pixel block; if the enhanced pixel value is less than the minimum value within the pixel block, the enhanced pixel value is truncated to the minimum value within the pixel block; if the enhanced pixel value is within the range of the maximum and minimum values ​​within the pixel block, it remains unchanged, ensuring that the enhanced pixel value is valid and without distortion.

[0147] Optionally, the weighted difference data is superimposed on the current pixel value to obtain the pixel value after edge enhancement. The calculation formula is as follows:

[0148]

[0149] Where pz is the pixel value after edge enhancement; , , These are the weights corresponding to frequency information, contrast information, and brightness information, respectively.

[0150] Finally, pixel reconstruction is performed on the intermediate resolution image based on the enhanced pixel values ​​of each pixel to obtain the edge-enhanced image.

[0151] The purpose of this step is to integrate the enhanced pixel values ​​of all pixels, reconstruct a complete edge-enhanced image, and complete this edge enhancement process.

[0152] Optionally, each pixel of the intermediate resolution image is traversed, and the enhanced pixel value corresponding to each pixel is filled into the corresponding coordinate position in the blank pixel matrix with the same size as the original intermediate resolution image. After the enhanced pixel values ​​of all pixels are filled, the initial edge enhancement image is obtained. Subsequently, the initial edge enhancement image can be subjected to simple smoothing processing (2×2 Gaussian filtering can be used) to remove slight gray-level abrupt changes that may occur during pixel enhancement, improve the overall smoothness of the image, and finally obtain the edge enhancement image.

[0153] Optionally, a gradient enhancement algorithm can be used to calculate the gray-level gradient of pixels in the edge area, amplify the gray-level gradient, enhance the sharpness of the edge, and suppress edge blurring and jagged distortion. A detail enhancement algorithm can be used to extract the detailed features of the texture area (such as texture contour and texture density), enhance the detailed features, preserve the continuous details of the texture area, and avoid the details being smoothed excessively. For flat areas, no additional edge enhancement processing is performed, only the smoothness of the flat area is preserved, avoiding unnecessary artifacts introduced by the enhancement processing and ensuring that there is no obvious distortion in the flat area.

[0154] Optionally, during the edge enhancement process, a Gaussian filtering algorithm can be combined for denoising to enhance edges and details while removing noise generated during edge enhancement, ensuring the clarity and purity of the edge-enhanced image. The intensity of edge enhancement can be flexibly controlled by adjusting the enhancement intensity coefficient. If the edges of the intermediate resolution image are severely blurred, the value of the enhancement intensity coefficient can be appropriately increased. If the intermediate resolution image has a lot of noise, the value of the enhancement intensity coefficient can be appropriately decreased, balancing the enhancement effect and the denoising effect.

[0155] The image processing method provided in this application first obtains the current pixel value and the pixel values ​​of surrounding points in an intermediate resolution image and calculates the difference. Then, it combines the weights of multi-dimensional image feature information such as frequency, contrast, and brightness to weight the pixel difference to obtain weighted difference data. This weighted difference data is then superimposed on the current pixel value to obtain an enhanced pixel value, and the image is reconstructed. Thus, by comprehensively considering multi-dimensional features, image edges are accurately identified, and edge information is highlighted through weighted processing, effectively enhancing the clarity and contrast of image edges, improving image edge blurring, and avoiding over-enhancement that leads to noise amplification. This results in higher quality and better visual effects for the edge-enhanced image.

[0156] As an optional implementation, based on any of the above embodiments, the edge enhancement image is subjected to end interpolation processing to obtain the target image, including the following steps:

[0157] First, distinguish the flat areas, textured areas, and edge areas in the edge enhancement image.

[0158] The purpose of this step is to accurately identify the different regional features of the edge-enhanced image, providing a foundation for the subsequent use of differentiated interpolation kernel functions and ensuring that the interpolation kernel function is highly compatible with the regional characteristics.

[0159] Optionally, an edge detection algorithm (such as the Canny operator) is used to extract edge information from the edge-enhanced image to determine the location and extent of the edge region; a texture extraction algorithm (such as gray-level co-occurrence matrix or LBP operator) is used to extract texture information from the edge-enhanced image to determine the location and extent of the texture region (after edge enhancement, texture details are clearer, and dense texture areas and smooth areas can be accurately distinguished); the remaining areas with gentle gray-level changes and no obvious edges or textures are determined to be flat areas. Optionally, the Harris corner detection algorithm can also be used to distinguish between flat areas, textured areas, and edge areas, as described in the above embodiments and will not be repeated here.

[0160] Secondly, isotropic interpolation kernel functions are used for interpolation calculations in flat and textured regions.

[0161] The purpose of this step is to ensure the smoothness and artifact-free nature of flat areas and the detail integrity of textured areas. The advantage of isotropic interpolation kernel functions is that they treat pixel grayscale changes in all directions equally, without obvious directional bias. This is suitable for the characteristics of flat areas ("no dominant direction, smooth grayscale") and textured areas ("multi-directional texture distribution, need to retain overall regularity"), avoiding the introduction of unnecessary grayscale abrupt changes or directional distortion.

[0162] Optionally, suitable isotropic interpolation kernel functions are selected for flat areas and textured areas respectively. The size of the interpolation kernel function can be flexibly adjusted according to the area detail density (such as 3×3 or 5×5 kernel function).

[0163] Optionally, a Gaussian interpolation kernel function (kernel size 3×3) can be used for flat regions. This kernel function has the advantages of good smoothness and strong noise resistance, which can further ensure the smoothness of flat regions and avoid artifacts introduced by interpolation at the end. The standard deviation of the Gaussian interpolation kernel function can be selected from 0.8 to 1.2.

[0164] Optionally, a cubic convolution interpolation kernel function (kernel size 5×5) can be used for the texture area. This kernel function has the advantages of strong detail preservation and high interpolation accuracy, and can accurately capture the detailed patterns of the texture area, avoiding the smoothing of texture details.

[0165] Optionally, a local neighborhood with the same size as the interpolation kernel function is selected as the center of each interpolation point, and the isotropic interpolation kernel function is convolved with the pixel values ​​in the local neighborhood to obtain the enhanced pixel value of the interpolation point. For flat areas, the weight magnitude of the interpolation kernel function can be appropriately reduced to reduce the risk of artifacts. For textured areas, the original weight of the interpolation kernel function can be retained to ensure that texture details are not lost.

[0166] Optionally, within the flat and textured regions, the interpolation weight of the (i,j)th pixel within the block is calculated using the following isotropic interpolation kernel function:

[0167]

[0168] in, f( ) is the interpolation weight for the (i,j)th pixel; The kernel function can be a Gaussian function, a bilinear interpolation function, or a bicubic interpolation function, etc. This is the horizontal distance from the pixel to the point to be interpolated. This is the vertical distance from the pixel to the point to be interpolated.

[0169] Finally, interpolation calculations are performed in the edge region using an interpolation kernel function along the direction.

[0170] The purpose of this step is to ensure the directional continuity and edge sharpness of the edge region, avoiding edge blurring and jagged distortion caused by end interpolation. The enhanced edge region has a clear dominant direction (edge ​​direction). The interpolation kernel function along the direction can perform interpolation calculations in accordance with the dominant edge direction, giving priority to the pixel information in the edge direction, suppressing grayscale abrupt changes perpendicular to the edge direction, and further enhancing edge sharpness and continuity.

[0171] Optionally, the dominant direction of each edge pixel region in the edge enhancement image is determined by combining the local direction information obtained from the interpolation above (or by recalculating the local gradient values ​​of edge region pixels along multiple directions and selecting the direction with the largest gradient value as the dominant edge direction). The dominant edge direction is the direction of the edge, typically 0°, 45°, 90°, 135°, etc., but can also be flexibly determined according to the actual edge direction. Based on the dominant edge direction, a corresponding interpolation kernel function is selected along the direction to ensure that the dominant direction of the interpolation kernel function is consistent with the dominant edge direction.

[0172] Optionally, a 3×3 local neighborhood is selected centered on each interpolation point in the edge region. The interpolation kernel function along the direction is convolved with the pixel values ​​in the local neighborhood to obtain the enhanced pixel value of the interpolation point. Optionally, an edge protection threshold is set. If the difference between the gray value of the interpolation point and the gray value of the edge region pixel exceeds the edge protection threshold, the enhancement intensity of the interpolation kernel function is appropriately reduced to avoid excessive enhancement leading to edge artifacts. The sum of the weights of the interpolation kernel function along the direction is kept at 1 to ensure the rationality and standardization of the interpolation calculation.

[0173] Optionally, within the edge region, the interpolation weight of the (i,j)th pixel within the block can be calculated using the following interpolation kernel function along the direction:

[0174]

[0175] in, This is the distance vector from the pixel to the point to be interpolated; , These are the eigenvectors obtained from the eigenvalue decomposition of the Hessian matrix. The tensile strength of the interpolation kernel along the edge direction. The contraction intensity is the value of the interpolation kernel in the direction perpendicular to the edge.

[0176] The image processing method provided in this application distinguishes between flat areas, textured areas, and edge areas in the edge enhancement image, and uses different interpolation kernel functions for different areas to perform end interpolation processing. In flat and textured areas, the use of isotropic interpolation kernel functions can ensure smooth transitions within the region and reduce noise and artifacts. In edge areas, the use of direction-dependent interpolation kernel functions can better fit the edge direction, effectively maintain the clarity and continuity of the edge, and avoid edge blurring and jaggedness, thereby improving the quality of the target image.

[0177] Figure 3 A flowchart illustrating an image processing method provided in another embodiment of this application is shown below. Figure 3 As shown, as an optional implementation, based on any of the above embodiments, interpolation calculation is performed in the edge region using an interpolation kernel function along the direction, including the following steps:

[0178] S301. Calculate the horizontal and vertical gradients of local pixels within the image block to be processed in the edge region, and obtain the Hessian matrix corresponding to the image block to be processed based on the horizontal and vertical gradients.

[0179] The purpose of this step is to quantify the local gray-level variation characteristics of the image patch to be processed in the edge region through gradient calculation and Hessian matrix construction, providing data support for subsequent edge direction recognition and interpolation weight calculation. The image patch to be processed is a local neighborhood selected centered on the point to be interpolated within the edge region. It can be a 3×3 or 5×5 image patch, and its selection range is limited to the edge region to ensure that the calculated data are all pixel features of the edge region and avoid interference from pixels in non-edge regions.

[0180] Optionally, firstly, select an image block to be processed centered on the point to be interpolated within the edge region, traverse all local pixels within the image block, and use gradient operators to calculate the horizontal and vertical gradients of each local pixel. During gradient calculation, if the image block to be processed exceeds the edge enhancement image range, a mirror completion strategy is used to supplement the missing pixel values ​​to ensure the integrity and accuracy of gradient calculation.

[0181] Secondly, based on the calculated horizontal and vertical gradients, the Hessian matrix corresponding to the image patch to be processed is solved. The Hessian matrix is ​​used to characterize the second-order variation characteristics of pixel gray levels within the image patch, and can accurately reflect the direction and curvature of the edges.

[0182] Alternatively, the formula for calculating the Hessian matrix is:

[0183]

[0184] in, It is the gradient in the horizontal direction. It is the gradient in the vertical direction.

[0185] S302. Perform eigenvalue decomposition on the Hessian matrix to obtain the corresponding eigenvectors.

[0186] The purpose of this step is to extract the dominant edge direction information of the image patch to be processed in the edge region through Hessian matrix eigenvalue decomposition. The feature vector directly corresponds to the vertical direction of the edge direction, providing a directional basis for subsequent direction adaptation of the interpolation kernel function and interpolation weight solution.

[0187] Specifically, the Hessian matrix obtained by solving is subjected to eigenvalue decomposition. The Hessian matrix is ​​a 2×2 symmetric matrix, and after decomposition, two eigenvalues ​​and two corresponding mutually perpendicular eigenvectors are obtained. Further, the dominant edge direction is determined by the magnitude of the eigenvalues. The eigenvector corresponding to the eigenvalue with the larger absolute value is perpendicular to the edge direction; the eigenvector corresponding to the eigenvalue with the smaller absolute value is parallel to the edge direction (i.e., the dominant edge direction). This method accurately locates the dominant edge direction, providing a clear basis for the direction adaptation of the subsequent interpolation kernel function and avoiding interpolation distortion caused by deviations in edge direction determination.

[0188] S303. Obtain the tensile strength of the interpolation kernel along the edge direction, the contraction strength of the interpolation kernel in the direction perpendicular to the edge, and the distance vector from the pixel point in the image block to the point to be interpolated, so as to obtain the interpolation calculation feature parameter set.

[0189] The purpose of this step is to construct the feature parameter set required for interpolation calculation, control the directional adaptability of the interpolation kernel function through stretching strength and shrinkage strength, and control the rationality of the interpolation weights through distance vector, so as to ensure that the interpolation kernel function can accurately fit the dominant edge direction, take into account the distance correlation between the pixel and the point to be interpolated, and improve the interpolation accuracy.

[0190] The stretching intensity of the interpolation kernel along the edge direction is used to control the stretching degree of the interpolation kernel function in the direction parallel to the edge. The greater the stretching intensity, the wider the coverage of the interpolation kernel in the edge direction, and the more the pixel information in the edge direction can be utilized. Its value range can be 1.2~2.0, which can be flexibly adjusted according to the edge sharpness of the edge area.

[0191] The shrinkage intensity of the interpolation kernel in the vertical edge direction is used to control the degree of shrinkage of the interpolation kernel function in the direction perpendicular to the edge. The greater the shrinkage intensity, the narrower the coverage of the interpolation kernel in the vertical edge direction, and the better it can suppress gray-scale abrupt changes in the vertical edge direction, thus avoiding edge blurring. Its value range can be 0.3~0.8.

[0192] Optionally, for each local pixel in the image block to be processed, a two-dimensional coordinate system is established with the interpolation point as the origin. The difference between the coordinates of the pixel and the coordinates of the interpolation point constitutes a distance vector. The distance vector is used to characterize the spatial positional relationship between the local pixel and the interpolation point, providing a basis for the differentiated allocation of subsequent interpolation weights. The closer the pixel is, the higher the subsequent interpolation weight is assigned.

[0193] S304. Based on the eigenvectors corresponding to the Hessian matrix and the interpolation calculation of the feature parameter set, the interpolation weights of each pixel in the image block to be processed are solved to obtain the interpolation weights corresponding to each pixel.

[0194] The purpose of this step is to combine edge direction features (Hessian matrix eigenvectors) and interpolation kernel parameters (feature parameter set) to solve for the differentiated interpolation weights of each local pixel, ensuring that pixels in the edge direction receive higher weights and pixels in the vertical edge direction receive lower weights.

[0195] Optionally, based on the eigenvectors of the Hessian matrix, the distance vector of each local pixel is decomposed to obtain the components of the distance vector in the direction parallel to the edge and the components in the direction perpendicular to the edge.

[0196] Optionally, the distance component is corrected by combining the tensile strength and contraction strength in the feature parameter set of the interpolation calculation to obtain the corrected distance component. The purpose of the correction is to amplify the influence of the distance component in the edge direction by using the tensile strength and reduce the influence of the distance component in the vertical edge direction by using the contraction strength, so that the corrected distance component is more in line with the requirements of interpolation along the direction and strengthens the weight advantage of pixels in the edge direction.

[0197] Optionally, based on the corrected distance components, a Gaussian weighting function is used to solve for the initial interpolation weights of each local pixel. The Gaussian weighting function has good smoothness and avoids interpolation artifacts caused by abrupt changes in weights.

[0198] Optionally, the initial interpolation weights of all local pixels are normalized to obtain the final interpolation weights for each pixel, ensuring that the sum of the weights of all pixels is 1, thus guaranteeing the rationality and standardization of the interpolation calculation. Normalization ensures that the weight allocation of each pixel is reasonable, avoiding interpolation distortion caused by abnormal weight summation.

[0199] S305. Based on the interpolation weights corresponding to each pixel, the interpolation kernel function along the direction is used to perform interpolation calculation on the edge region to complete the interpolation kernel function interpolation calculation along the direction of the edge region.

[0200] The purpose of this step is to combine the obtained differential interpolation weights with the interpolation kernel function along the direction to complete the pixel value calculation of the point to be interpolated in the edge region, ensuring that the interpolation result fits the edge direction, enhancing edge continuity and sharpness, and avoiding edge blurring and jagged distortion.

[0201] Optionally, based on the edge dominance direction obtained from the eigenvalue decomposition of the Hessian matrix, a corresponding interpolation kernel function (with the kernel function size consistent with the image block to be processed) is selected to ensure that the dominance direction of the interpolation kernel function is consistent with the edge dominance direction, thereby achieving the adaptation of the interpolation kernel function to the edge direction. Secondly, the gray value of each local pixel in the image block to be processed is multiplied by the corresponding interpolation weight, and then all product results are summed to obtain the target pixel value of the point to be interpolated.

[0202] Repeat steps S301 to S305 above to perform interpolation calculations along the direction kernel function for each interpolation point in the edge region, complete all interpolation processing in the edge region, and ensure that each interpolation point in the edge region can fit the dominant edge direction, thereby improving edge continuity and sharpness.

[0203] The image processing method provided in this application first obtains the Hessian matrix by calculating the horizontal and vertical gradients of the image block to be processed in the edge region and decomposes it into feature vectors. Then, it combines parameters such as the stretching intensity along the edge direction, the contraction intensity perpendicular to the edge direction, and the distance vector from the pixel to the point to be interpolated to solve for the interpolation weight of each pixel. Finally, it completes the interpolation calculation using the direction-based interpolation kernel function. Therefore, it can accurately capture edge direction information, flexibly adjust the interpolation weights according to edge characteristics, make the interpolation process more closely follow the edge direction, effectively avoid edge blurring and jaggedness, and significantly improve the interpolation effect in the edge region.

[0204] Figure 4 This is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of this application, as shown below. Figure 4 As shown, the image processing device provided in this embodiment is located in an electronic device. The image processing device 40 provided in this embodiment includes: an acquisition module 41, a first interpolation module 42, an image enhancement module 43, a judgment module 44, and a second interpolation module 45.

[0205] Specifically, the acquisition module 41 is used to acquire the original image to be processed; the first interpolation module 42 is used to interpolate the original image based on a preset magnification to obtain an intermediate resolution image; wherein, the preset magnification is an integer or non-integer magnification greater than 1; the image enhancement module 43 is used to perform edge enhancement processing on the intermediate resolution image to obtain an edge-enhanced image; the judgment module 44 is used to judge whether the resolution of the edge-enhanced image meets the preset resolution condition; and the second interpolation module 45 is used to perform end interpolation processing on the edge-enhanced image if the resolution of the edge-enhanced image meets the preset resolution condition to obtain the target image.

[0206] Optionally, the first interpolation module 42, when interpolating the original image based on a preset magnification to obtain an intermediate resolution image, specifically performs the following: calculates local gradients of the original image along multiple directions to obtain local gradient values ​​in each direction; determines the local direction of pixel regions of the original image based on the local gradient values ​​in each direction to determine the local direction corresponding to each pixel region; calculates pixel values ​​of the points to be interpolated in the original image based on pixel information in the local direction and gradient change information between pixels to obtain target pixel values ​​for each point to be interpolated; and interpolates the original image based on the target pixel values ​​of each point to be interpolated and the preset magnification to obtain an intermediate resolution image.

[0207] Optionally, the first interpolation module 42, when calculating the pixel value of the interpolation point in the original image, specifically performs the following: comparing the local gradient values ​​in each direction with a preset gradient threshold to distinguish the flat area, texture area and edge area of ​​the original image; and selecting local pixels in the corresponding area to calculate the pixel value of the interpolation point based on the weights corresponding to the flat area, texture area and edge area, so as to complete the pixel value calculation of the interpolation point in each area.

[0208] Optionally, the image enhancement module 43, when performing edge enhancement processing on the intermediate resolution image to obtain an edge-enhanced image, specifically performs the following steps: obtaining the current pixel value of the intermediate resolution image and calculating the pixel values ​​of the current point and surrounding points within the pixel block of the intermediate resolution image; performing weighted processing based on the pixel difference using the weights corresponding to the multi-dimensional image feature information to obtain weighted difference data; wherein, the multi-dimensional image feature information includes frequency information, contrast information, and brightness information; superimposing the weighted difference data onto the current pixel value to obtain the enhanced pixel value of each pixel; and performing pixel reconstruction on the intermediate resolution image based on the enhanced pixel values ​​of each pixel to obtain the edge-enhanced image.

[0209] Optionally, the second interpolation module 45, when performing end interpolation processing on the edge enhancement image to obtain the target image, is specifically used to: distinguish the flat area, texture area and edge area of ​​the edge enhancement image; perform interpolation calculation using an isotropic interpolation kernel function in the flat area and texture area; and perform interpolation calculation using an interpolation kernel function along the direction in the edge area.

[0210] Optionally, the second interpolation module 45, when performing interpolation calculations using an interpolation kernel function along the direction in the edge region, specifically performs the following: calculates the horizontal and vertical gradients of local pixels within the image block to be processed in the edge region; obtains the Hessian matrix corresponding to the image block to be processed based on the horizontal and vertical gradients; performs eigenvalue decomposition on the Hessian matrix to obtain the eigenvector corresponding to the Hessian matrix; obtains the stretching intensity of the interpolation kernel along the edge direction, the contraction intensity of the interpolation kernel perpendicular to the edge direction, and the distance vector from the pixel point in the image block to the point to be interpolated, to obtain the interpolation calculation feature parameter set; based on the eigenvector corresponding to the Hessian matrix and the interpolation calculation feature parameter set, calculates the interpolation weights for each pixel point in the image block to be processed, to obtain the interpolation weights corresponding to each pixel point; and performs interpolation operations on the edge region using an interpolation kernel function along the direction based on the interpolation weights corresponding to each pixel point, to complete the interpolation calculation of the interpolation kernel function along the direction in the edge region.

[0211] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application, as shown below. Figure 5 As shown, the electronic device 50 provided in this embodiment includes: a processor 51 and a memory 52 communicatively connected to the processor 51.

[0212] The memory 52 stores computer-executed instructions; the processor 51 executes the computer-executed instructions stored in the memory 52 to implement an image processing method provided in any of the above embodiments.

[0213] The program may include program code, which includes computer-executable instructions. Memory 52 may include high-speed RAM, and may also include non-volatile memory, such as at least one disk storage device.

[0214] In this embodiment, the memory 52 and the processor 51 are connected via a bus. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 5 The bus is represented by a single straight line, but this does not mean that there is only one bus or one type of bus.

[0215] This application also provides a computer-readable storage medium, which stores computer-executable instructions. When executed by a processor, the computer-executable instructions are used to implement an image processing method provided in any of the above embodiments.

[0216] This application also provides a computer program product, including a computer program that, when executed by a processor, implements an image processing method provided in any of the above embodiments.

[0217] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to implement the solution of this embodiment according to actual needs.

[0218] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing unit, or each module can exist physically separately, or two or more modules can be integrated into one unit. The unit composed of the above modules can be implemented in hardware or in the form of hardware plus software functional units.

[0219] The integrated modules described above, implemented as software functional modules, can be stored in a computer-readable storage medium. These software functional modules, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods of the various embodiments of this application.

[0220] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly manifested as execution by a hardware processor, or execution by a combination of hardware and software modules within the processor.

[0221] The memory may include high-speed RAM, and may also include non-volatile storage (NVM), such as at least one disk storage device, and may also be a USB flash drive, external hard drive, read-only memory, disk or optical disc, etc.

[0222] The aforementioned storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0223] An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Alternatively, the storage medium can be an integral part of the processor. The processor and storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and storage medium can exist as discrete components in an electronic control unit or main control device.

[0224] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0225] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. An image processing method, characterized in that, include: Obtain the original image to be processed; The original image is interpolated based on a preset magnification ratio to obtain an intermediate resolution image; wherein the preset magnification ratio is an integer or non-integer magnification ratio greater than 1. The intermediate resolution image is subjected to edge enhancement processing to obtain an edge-enhanced image; Determine whether the resolution of the edge-enhanced image meets the preset resolution condition; If the resolution of the edge-enhanced image meets the preset resolution condition, the edge-enhanced image is subjected to end interpolation processing to obtain the target image.

2. The method according to claim 1, characterized in that, The step of interpolating the original image based on a preset magnification to obtain an intermediate resolution image includes: Local gradient calculations are performed on the original image along multiple directions to obtain local gradient values ​​in each direction; Based on the local gradient values ​​in each direction, the local orientation of the pixel region of the original image is determined to identify the local orientation corresponding to each pixel region. Based on the pixel information in the local direction and the gradient change information between pixels, the pixel values ​​of the interpolation points in the original image are calculated to obtain the target pixel values ​​of each interpolation point. Based on the target pixel value of each interpolation point and the preset magnification, the original image is interpolated to obtain the intermediate resolution image.

3. The method according to claim 2, characterized in that, The step of calculating the pixel value of the interpolation point in the original image includes: The local gradient values ​​in each direction are compared with preset gradient thresholds to distinguish the flat areas, textured areas and edge areas of the original image. Based on the weights corresponding to the flat region, the texture region, and the edge region, local pixels in the corresponding regions are selected to calculate the pixel values ​​of the interpolation points, thereby completing the pixel value calculation for the interpolation points in each region.

4. The method according to claim 1, characterized in that, The process of performing edge enhancement processing on the intermediate resolution image to obtain an edge-enhanced image includes: Obtain the current pixel value of the intermediate resolution image, and calculate the pixel difference between the current point and the surrounding points within the pixel block of the intermediate resolution image; Based on the pixel difference, a weighted processing is performed using the weights corresponding to the multi-dimensional image feature information to obtain weighted difference data; wherein, the multi-dimensional image feature information includes frequency information, contrast information and brightness information; The weighted difference data is superimposed on the current pixel value to obtain the enhanced pixel value of each pixel. The intermediate resolution image is reconstructed based on the enhanced pixel values ​​of each pixel to obtain the edge-enhanced image.

5. The method according to claim 1, characterized in that, The step of performing end interpolation processing on the edge-enhanced image to obtain the target image includes: Distinguish between the flat areas, textured areas, and edge areas of the edge-enhanced image; Interpolation calculations are performed in the flat region and the textured region using an isotropic interpolation kernel function. Interpolation calculations are performed in the edge region using an interpolation kernel function along the direction.

6. The method according to claim 5, characterized in that, The interpolation calculation in the edge region using an interpolation kernel function along the direction includes: The horizontal and vertical gradients are calculated for local pixels within the image block to be processed in the edge region, and the Hessian matrix corresponding to the image block to be processed is obtained based on the horizontal and vertical gradients. The Hessian matrix is ​​subjected to eigenvalue decomposition to obtain the eigenvectors corresponding to the Hessian matrix; The interpolation kernel's stretching strength along the edge direction, the interpolation kernel's contraction strength perpendicular to the edge direction, and the distance vector from the pixel point within the image block to the point to be interpolated are obtained to obtain the interpolation calculation feature parameter set. Based on the eigenvectors corresponding to the Hessian matrix and the interpolation calculation feature parameter set, the interpolation weights of each pixel in the image block to be processed are solved to obtain the interpolation weights corresponding to each pixel. Based on the interpolation weights corresponding to each pixel, an interpolation kernel function along the direction is used to perform interpolation operations on the edge region to complete the interpolation kernel function interpolation calculation of the edge region along the direction.

7. An image processing apparatus, characterized in that, include: The acquisition module is used to acquire the original image to be processed; The first interpolation module is used to interpolate the original image based on a preset magnification ratio to obtain an intermediate resolution image; wherein the preset magnification ratio is an integer or non-integer magnification ratio greater than 1. An image enhancement module is used to perform edge enhancement processing on the intermediate resolution image to obtain an edge-enhanced image; The judgment module is used to determine whether the resolution of the edge enhancement image meets the preset resolution condition; The second interpolation module is used to perform end interpolation processing on the edge enhancement image to obtain the target image, provided that the resolution of the edge enhancement image meets the preset resolution condition.

8. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-6.

10. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method of any one of claims 1-6.

Images