Image processing method, electronic device and storage medium
By determining the edge direction according to the gradient direction in image processing for interpolation, the problems of image blurring and contour information loss during image scaling in the prior art are solved, and a high-quality image processing effect is achieved.
Patent Information
- Application Number
- CN202310558777.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-17
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2043-05-17
AI Technical Summary
The existing image processing methods fail to effectively consider the edge characteristics of the image during the scaling process, resulting in blurred image after processing, which cannot reflect the true outline information of the image, affecting image quality.
By determining the edge direction of each pixel point to be interpolated in the original image and interpolated in the edge direction, the continuity of the interpolated image is improved and more image details are preserved.
It effectively avoids the appearance of stripes in the image, improves the quality of the processed image, makes it clear and lossless, and can truly reflect the outline information of the image.
Smart Images

Figure CN117726557B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of images, and in particular to an image processing method, an electronic device and a storage medium. Background Art
[0002] With the emergence and widespread application of digital images, people's requirements for image quality are getting higher and higher. Improving image resolution in different application scenarios has gradually become one of the hot issues in the field of image processing.
[0003] At present, when we scale images, we usually use traditional image processing methods, such as bilinear interpolation. However, using traditional image processing methods to process images easily leads to blurred images after processing, which seriously affects the quality of the images. Summary of the invention
[0004] The present application provides an image processing method, an electronic device and a storage medium. In the process of processing an image, the edge characteristics of the image can be taken into account, so that the target image obtained after processing retains more image details, can reflect the real contour information of the image, improves the quality of the processed image, and ensures that the processed image is clear and lossless.
[0005] In a first aspect, the present application provides an image processing method, the method comprising: determining the edge direction of each pixel to be interpolated according to the gradient direction of each pixel to be interpolated corresponding to the original image, and determining the grayscale value of each pixel to be interpolated; in the edge direction of each pixel to be interpolated, interpolating based on the grayscale value of the pixel to be interpolated to obtain a target image corresponding to the original image.
[0006] Among them, the edge direction is perpendicular to the gradient direction.
[0007] The original image can be an RGB image, a YUV image, or a grayscale image.
[0008] The gradient direction of the pixel to be interpolated indicates the direction of change of the grayscale value of the pixel to be interpolated. For the pixel to be interpolated, the grayscale value changes fastest along the gradient direction, and the grayscale value changes slowest along the edge direction. Therefore, if interpolation is performed in the edge direction, the continuity of the interpolated image is higher and the interpolation loss is smaller, thereby retaining more image details, making the final target image clear, lossless and high-quality.
[0009] It should be understood that when interpolation is performed based on the grayscale value of each pixel to be interpolated in the edge direction of each pixel to be interpolated, different interpolation methods correspond to different image processing requirements.
[0010] Optionally, when it is necessary to enlarge the image, up-interpolation is performed based on the grayscale value of each pixel to be interpolated in the edge direction of each pixel to be interpolated, so as to obtain a target image corresponding to the original image.
[0011] Optionally, when it is necessary to reduce the image, downward interpolation is performed based on the grayscale value of each pixel to be interpolated in the edge direction of the pixel to be interpolated to obtain a target image corresponding to the original image.
[0012] The image processing method provided in the first aspect first determines the edge direction and grayscale value of each pixel to be interpolated according to the gradient direction of each pixel to be interpolated corresponding to the original image. Since the gradient direction and the edge direction are perpendicular to each other, when the grayscale value in the gradient direction changes fastest, the grayscale value in the corresponding edge direction changes slowest. Based on this, in the edge direction of each pixel to be interpolated, interpolation is performed based on the grayscale value of the pixel to be interpolated, which can improve the continuity of the grayscale value change of the interpolated image, reduce the interpolation loss, and thus make the target image obtained after processing retain more image details.
[0013] Since the present application takes into account the edge characteristics of the image, the image processing method provided by the present application, in the process of processing the image, combines the edge characteristics of the image, can make the target image obtained after processing reflect the real contour information of the image, improve the quality of the processed image, and ensure that the processed image is clear and lossless. Compared with the related art, it effectively avoids the stripe (or aliasing) phenomenon in the processed image.
[0014] In a possible implementation, the image processing method provided by the present application further includes:
[0015] Determine the gradient direction of each pixel to be interpolated corresponding to the original image.
[0016] In this implementation, the gradient direction of each pixel to be interpolated corresponding to the original image is determined, which is conducive to the subsequent rapid and accurate determination of the edge direction according to the gradient direction.
[0017] In a possible implementation, determining the gradient direction of each pixel to be interpolated corresponding to the original image includes: determining the grayscale gradient component of each pixel to be interpolated; performing vector synthesis on multiple grayscale gradient components of each pixel to be interpolated to obtain the grayscale gradient of each pixel to be interpolated.
[0018] The grayscale gradient component indicates the grayscale gradient of the pixel to be interpolated in a first direction, the first direction is the direction from the neighboring pixel points of the pixel to be interpolated to the pixel to be interpolated, and the grayscale gradient includes the gradient direction.
[0019] In this implementation, the grayscale gradient of the pixel to be interpolated is obtained by vector synthesis of multiple grayscale gradient components of the pixel to be interpolated. Since adjacent pixels are distributed in different directions, the method of vector synthesis through multiple grayscale gradient components can take into account the grayscale change characteristics of the pixel to be interpolated in different directions. The gradient direction of the pixel to be interpolated thus determined is more accurate, which is conducive to the subsequent accurate determination of the edge direction of the pixel to be interpolated based on the gradient direction.
[0020] In one possible implementation, the gradient direction of each pixel to be interpolated corresponding to the original image is determined, including: determining the weight value between each pixel to be interpolated and each adjacent pixel, determining the grayscale gradient component of each pixel to be interpolated according to each weight value, and performing vector synthesis on multiple grayscale gradient components of each pixel to be interpolated to obtain the grayscale gradient of each pixel to be interpolated.
[0021] In this implementation, when determining the grayscale gradient of the pixel to be interpolated, not only the grayscale change characteristics of the adjacent pixels of the pixel to be interpolated in different directions are taken into account, but also the grayscale values of the adjacent pixels are adjusted based on the weight values corresponding to different adjacent pixels, so that the determined grayscale gradient component is more accurate, and thus the gradient direction synthesized based on the grayscale gradient component is also more accurate.
[0022] In a possible implementation, determining the grayscale gradient component of each pixel to be interpolated includes:
[0023] For each pixel to be interpolated, multiple adjacent pixels adjacent to the pixel to be interpolated in the original image and the grayscale value of each adjacent pixel are determined; based on the initial grayscale value of the pixel to be interpolated and the grayscale values of multiple adjacent pixels, multiple grayscale gradient components of the pixel to be interpolated are calculated.
[0024] In this implementation, the grayscale gradient component of the pixel to be interpolated is determined based on multiple adjacent pixels adjacent to the pixel to be interpolated and the grayscale value of each adjacent pixel. In this process, the grayscale change characteristics of the adjacent pixels of the pixel to be interpolated in different directions are taken into account, so that the determined grayscale gradient component is more accurate, which is conducive to the subsequent accurate synthesis of the gradient direction of the pixel to be interpolated based on the grayscale gradient component.
[0025] In a possible implementation, determining the grayscale value of each pixel to be interpolated includes: determining the edge direction angle of each pixel to be interpolated; and calculating the grayscale value of each pixel to be interpolated based on the grayscale value of each adjacent pixel, the edge direction of each pixel to be interpolated, and the edge direction angle of each pixel to be interpolated.
[0026] The edge direction angle refers to the angle formed by the x-axis and the straight line where the edge direction of the pixel to be interpolated is located, which is determined by using the x-axis direction as the standard direction.
[0027] Optionally, the edge direction angle can be used to determine the slope of the straight line where the edge direction lies. Therefore, any angle formed by the x-axis and the straight line where the edge direction of the pixel to be interpolated lies can be used as the edge direction angle of the pixel to be interpolated as long as the slope of the straight line where the edge direction lies can be determined.
[0028] Optionally, the edge direction angle of each pixel to be interpolated may be determined by determining the gradient direction angle of each pixel to be interpolated, and determining the edge direction angle of each pixel to be interpolated according to the gradient direction angle of each pixel to be interpolated.
[0029] The gradient direction angle is the angle between the pixel to be interpolated in the horizontal direction and the gradient direction.
[0030] In this implementation, the gray value of each pixel to be interpolated is accurately calculated based on the gray value of each adjacent pixel, the edge direction of each pixel to be interpolated, and the edge direction angle of each pixel to be interpolated. Therefore, when interpolation is performed based on the gray value, the continuity of the interpolated image can be higher, the interpolation loss can be smaller, more image details can be retained, and the final target image can be clear, lossless, and of high quality.
[0031] In a possible implementation, the gray value of each pixel to be interpolated is calculated according to the gray value of each adjacent pixel, the edge direction of each pixel to be interpolated, and the edge direction angle of each pixel to be interpolated, including:
[0032] For each pixel to be interpolated, a first intersection and a second intersection are determined according to the edge direction of the pixel to be interpolated and multiple adjacent pixels; a first distance and a second distance are determined according to the edge direction angle of the pixel to be interpolated; grayscale values of the first intersection and the second intersection are calculated based on the grayscale value of each adjacent pixel; and the grayscale value of the pixel to be interpolated is calculated according to the first distance, the second distance, the grayscale value of the first intersection, and the grayscale value of the second intersection.
[0033] The first intersection point, the second intersection point and the pixel point to be interpolated are on a straight line where the edge direction is located, the first distance represents the distance between the pixel point to be interpolated and the first intersection point, and the second distance represents the distance between the pixel point to be interpolated and the second intersection point.
[0034] Optionally, based on the grayscale value of each adjacent pixel point, the grayscale value of the first intersection point is calculated by a first linear interpolation polynomial.
[0035] Optionally, based on the grayscale value of each adjacent pixel point, the grayscale value of the second intersection point is calculated by a second linear interpolation polynomial.
[0036] In this implementation, the gray value of each pixel to be interpolated is accurately calculated based on the gray value of each adjacent pixel, the edge direction of each pixel to be interpolated, and the edge direction angle of each pixel to be interpolated. Therefore, when interpolation is performed based on the gray value, the continuity of the interpolated image can be higher, the interpolation loss can be smaller, more image details can be retained, and the final target image can be clear, lossless, and of high quality.
[0037] In a possible implementation, the image processing method provided by the present application further includes:
[0038] For each pixel to be interpolated, the grayscale value of the pixel to be interpolated is determined as the initial grayscale value of the pixel to be interpolated; based on the initial grayscale value and the grayscale values of multiple adjacent pixels, multiple grayscale gradient components of the pixel to be interpolated are calculated; based on the multiple grayscale gradient components, the grayscale gradient of the pixel to be interpolated is determined; according to the grayscale gradient, the edge direction of the pixel to be interpolated is determined; the edge direction angle of the pixel to be interpolated is determined; according to the grayscale value of each adjacent pixel, the edge direction of the pixel to be interpolated and the edge direction angle of the pixel to be interpolated, the grayscale value of the pixel to be interpolated is calculated; repeating the above steps, when the edge direction angle converges, in the edge direction of each pixel to be interpolated, based on the grayscale value of the pixel to be interpolated, to obtain a target image corresponding to the original image, including: in the edge direction of each pixel to be interpolated, according to the grayscale value of the pixel to be interpolated obtained when the edge direction angle converges, interpolation is performed to obtain the target image.
[0039] In this implementation, due to continuous iteration, the determined gradient direction, edge direction and grayscale value of the pixel to be interpolated are more accurate. Therefore, when interpolation is performed based on the edge direction and grayscale value after the iteration, the interpolated image can retain more details and be more accurate, further improving the quality of the target image.
[0040] In a possible implementation, the image processing method provided by the present application, before determining the edge direction of each pixel to be interpolated according to the gradient direction of each pixel to be interpolated corresponding to the original image, further includes:
[0041] Detecting operations for zooming;
[0042] In response to the operation, a scaling ratio of the original image is determined; and each pixel point to be interpolated corresponding to the original image is determined according to the scaling ratio and an interpolation algorithm.
[0043] The operation for zooming in and out may be a zoom-in operation or a zoom-out operation.
[0044] If the operation for zooming is generated by the user touching the screen, the zoom ratio can be determined by detecting the position touched by the user's finger. If the operation for zooming is triggered by an electronic device, the zoom ratio can be determined by the screen resolution of the electronic device.
[0045] In this implementation, each pixel to be interpolated corresponding to the original image is first determined by a traditional interpolation algorithm, and the initial grayscale value of each pixel to be interpolated is determined, which provides a basis for the subsequent rapid and accurate determination of the gradient direction, edge direction and grayscale value of the pixel to be interpolated, thereby ensuring that a high-quality image is obtained after interpolation based on the gradient direction and grayscale value of the pixel to be interpolated.
[0046] In a second aspect, the present application provides an electronic device, comprising: one or more processors; one or more memories; a module with multiple applications installed; the memory stores one or more programs, and when one or more programs are executed by the processor, the electronic device executes the method in the above-mentioned first aspect and any possible implementation thereof.
[0047] In a third aspect, the present application provides a chip, including a processor. The processor is used to read and execute a computer program stored in a memory to perform the method in the first aspect and any possible implementation thereof.
[0048] Optionally, the chip also includes a memory, and the memory is connected to the processor via a circuit or wire.
[0049] Optionally, the chip also includes a communication interface.
[0050] In a fourth aspect, the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the processor executes the method in the first aspect and any possible implementation thereof.
[0051] In a fifth aspect, the present application provides a computer program product, which includes: a computer program code, which, when executed on an electronic device, enables the electronic device to execute the method in the first aspect and any possible implementation thereof.
[0052] The technical effects obtained by the above-mentioned second, third, fourth and fifth aspects are similar to the technical effects obtained by the corresponding technical means in the above-mentioned first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS
[0053] Figure 1 is a schematic diagram of a magnified image scene shown in an exemplary embodiment of the present application;
[0054] Figure 2 is another schematic diagram of an enlarged image scene shown in an exemplary embodiment of the present application;
[0055] Figure 3 is a schematic diagram of the hardware structure of an electronic device shown in an exemplary embodiment of the present application;
[0056] Figure 4 It is a flowchart of an image processing method provided in an embodiment of the present application;
[0057] Figure 5 is a schematic diagram of interpolation of an amplified image provided in an embodiment of the present application;
[0058] Figure 6 is a schematic diagram of a bilinear interpolation algorithm provided in an embodiment of the present application;
[0059] Figure 7 It is a flowchart of an image processing method provided in an embodiment of the present application;
[0060] Figure 8 is a flowchart of another image processing method provided in an embodiment of the present application;
[0061] Fig. 9 is a schematic diagram of a grayscale gradient component provided in an embodiment of the present application;
[0062] Fig.10 It is a vector synthesis schematic diagram provided in an embodiment of the present application;
[0063] Fig.11 is a schematic diagram of a process for calculating the grayscale value of a pixel to be interpolated provided in an embodiment of the present application;
[0064] Fig.12 is a schematic diagram of an edge direction angle provided in an embodiment of the present application;
[0065] Fig.13 It is a schematic diagram of an intersection provided in an embodiment of the present application;
[0066] Fig.14 is another intersection schematic diagram provided in an embodiment of the present application;
[0067] Fig.15 This is another intersection schematic diagram provided in an embodiment of the present application;
[0068] Fig.16 It is another intersection schematic diagram provided in an embodiment of the present application;
[0069] Fig.17 This is another intersection schematic diagram provided in an embodiment of the present application;
[0070] Fig.18It is another intersection schematic diagram provided in an embodiment of the present application;
[0071] Fig.19 It is a flowchart of an image processing method provided in an embodiment of the present application;
[0072] Fig. 20 is a schematic diagram of edge direction angle convergence provided in an embodiment of the present application;
[0073] Fig.21 is a schematic diagram of a scene in which an image is enlarged using the image processing method provided by the present application;
[0074] Fig. 22 It is a schematic diagram of the structure of a chip provided in an embodiment of the present application. DETAILED DESCRIPTION
[0075] The technical solution in this application will be described below in conjunction with the accompanying drawings.
[0076] In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.
[0077] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this embodiment, unless otherwise specified, "plurality" means two or more.
[0078] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.
[0079] It should be noted that the image processing method provided in the embodiment of the present application can be applied to any electronic device with image processing function.
[0080] In some embodiments of the present application, the electronic device may be a mobile phone, a tablet computer, a wearable device, a television, a vehicle-mounted device, an augmented reality (AR) / virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), etc. It may also be various camera devices such as a SLR camera and a compact camera, or it may be other devices or apparatuses capable of performing image processing. The embodiments of the present application do not impose any restrictions on the specific type of the electronic device.
[0081] In order to better understand the image processing method provided in the embodiments of the present application, some terms involved in the embodiments of the present application are first explained below to facilitate understanding by those skilled in the art.
[0082] 1. Bilinear interpolation
[0083] Also known as bilinear interpolation. Mathematically, bilinear interpolation is an extension of linear interpolation with an interpolation function of two variables. Its core idea is to perform linear interpolation in two directions respectively.
[0084] In image processing, bilinear interpolation is a type of image interpolation algorithm, which is performed by weighted calculation of the pixel values of the four pixels closest to the pixel to be calculated to obtain the value of the pixel to be calculated.
[0085] 2. Grayscale
[0086] The image is displayed with black as the base color and different degrees of black saturation. Each grayscale object has a brightness value from 0% (black) to 100% (white). Images generated using black and white or grayscale scanners are usually displayed in grayscale.
[0087] 3. Grayscale gradient
[0088] If we regard the image as a two-dimensional discrete function, the grayscale gradient is actually the derivative of this two-dimensional discrete function. We use difference instead of differential to obtain the grayscale gradient of the image.
[0089] 4. Gradient direction
[0090] In mathematics, the original meaning of gradient is a vector, which means that the directional derivative of a function at a certain point reaches its maximum value along that direction, that is, the function changes fastest along that direction at that point and the rate of change is the largest. In images, gradient is used to represent the degree and direction of change of grayscale values.
[0091] 5. Edge direction
[0092] In an image, the edge refers to the location where the grayscale intensity changes the weakest. The gradient direction is perpendicular to the edge direction.
[0093] 6. Grayscale images
[0094] Grayscale images are images with only one sampled color per pixel. Such images are usually displayed as grayscale from the darkest black to the brightest white. Grayscale images are different from black and white images. In the field of computer graphics, black and white images only have two colors, black and white, while grayscale images have many levels of color depth between black and white.
[0095] 7. RGB (Red, Green, Blue) images
[0096] RGB refers to a color model related to the structure of the human visual system. According to the structure of the human eye, all colors are regarded as different combinations of red, green, and blue.
[0097] RGB images refer to images displayed using the RGB color mode.
[0098] 8. YUV images
[0099] YUV refers to a color encoding method, where Y represents luminance or luma, and U and V represent chrominance or chroma. The above RGB color mode focuses on the human eye's perception of color, while the YUV color mode focuses on the visual sensitivity to brightness. The RGB color mode and the YUV color mode can be converted to each other.
[0100] YUV images are images displayed in YUV color mode. If an image has only Y component but no UV component, then the image is represented as a black and white image.
[0101] The above is a brief introduction to the terms involved in the embodiments of the present application, which will not be repeated below.
[0102] With the development of digital images, people use images more and more frequently, and at the same time, their requirements for image quality are getting higher and higher.
[0103] For example, when using electronic devices (such as mobile phones), we often encounter scenes where we need to enlarge or reduce images. At this time, we hope that the enlarged or reduced images still maintain high quality. However, at present, when we scale images, we usually use traditional image processing methods. Among them, traditional image processing methods include nearest neighbor interpolation, bilinear interpolation, bicubic interpolation, and multi-phase interpolation. Because these interpolation methods are used to process images, the edge characteristics of the image are not taken into account. For example, with the pixel to be found as the center, the characteristics of the pixel to be found in various directions (such as changes in pixel grayscale, texture, etc.) are usually different. However, when these traditional interpolation methods are used to process images, the principle of isotropy is used for processing, and the characteristics of the pixel to be found in various directions are not taken into account, resulting in blurred images after processing, unclear image details, and serious impact on the quality of the image.
[0104] In some possible application scenarios, when users preview images in the gallery application of mobile phones, they will zoom in or out on the images. When the mobile phone responds to the user's zoom in or out operation on the image, it usually processes the image using traditional image processing methods, resulting in stripes (or aliasing) on the zoomed in or out image, making the details of the zoomed in or out image unclear and unable to reflect the true contour information of the image, which seriously affects the quality of the zoomed in or out image.
[0105] The following describes an application scenario of enlarging a certain image in a gallery application in conjunction with the accompanying drawings.
[0106] See also Figure 1 , Figure 1 It is a schematic diagram of a magnified image scene shown in an exemplary embodiment of the present application.
[0107] In the embodiments of the present application, the electronic device is described as a mobile phone. Figure 1 As shown, Figure 1 (a) shows the main interface of the mobile phone. For example, when the user clicks the gallery icon 101 in the main interface of the mobile phone, the mobile phone displays an interface from Figure 1 The main interface shown in (a) jumps to Figure 1 The gallery display interface shown in (b) above shows multiple thumbnails. Figure 1 The mobile phone displays an image 102 in the multiple thumbnails shown in (b). Figure 1 The gallery display interface shown in (b) jumps to Figure 1 It should be understood that when viewing an image in the gallery, if the image is not a full-screen image, the non-image content area is displayed as a white background area or a black background area in the image display interface, such as Figure 1The white background area 103 shown in (c) is shown in the figure. The image 102 in the thumbnail is displayed as the original image in the image display interface, such as Figure 1 Image 104 shown in (c).
[0108] The non-full screen ratio image refers to an image displayed on the display interface whose display ratio is not full screen.
[0109] like Figure 1 As shown in (c), when the user wants to enlarge the partial image 105 in the image 104, the user can touch the area where the partial image 105 is located with two fingers at the same time and slide outward to enlarge the partial image 105. Figure 1 As shown in (d), the image 106 is the image after the partial image 105 is enlarged.
[0110] When the mobile phone responds to the user's zoom operation on the local image 105, if the traditional image processing method is used to process the local image 105, it will cause stripes (or jagged edges) to appear in the enlarged image 106, resulting in unclear details of the enlarged image 106, unable to reflect the true contour information of the image, and seriously affecting the quality of the enlarged image.
[0111] For easier understanding, see Figure 2 , Figure 2 It is a schematic diagram of another magnified image scene shown in an exemplary embodiment of the present application.
[0112] like Figure 2 As shown, if an image in the thumbnail is displayed as the original image, it will be displayed as Figure 2 In the image 201 shown in (a), when the user wants to zoom in on a partial image 202 in the image 201, the user can simultaneously touch the area where the partial image 202 is located with two fingers and slide outwards to zoom in on the partial image 202. Figure 2 As shown in (b), the image 203 is the image after the partial image 202 is enlarged.
[0113] When the local image 202 is enlarged by using the traditional image processing method, the edge characteristics of the local image 202 are not taken into consideration, so the enlarged image 203 obtained has stripes (or jagged edges), which makes the details of the enlarged image 203 unclear and unable to reflect the true contour information of the image, seriously affecting the quality of the enlarged image.
[0114] The above describes the application scenario of using traditional image processing methods to enlarge images. It can be seen that when using traditional image processing methods to enlarge images, the enlarged images obtained have stripes (or aliasing), which makes the details of the enlarged images unclear and cannot reflect the true contour information of the images, seriously affecting the quality of the enlarged images. It should be understood that when using traditional image processing methods to reduce images, the details of the reduced images will also be unclear and cannot reflect the true contour information of the images, seriously affecting the quality of the reduced images.
[0115] This is because the edge characteristics of the image are not taken into account during the image scaling process using traditional image processing methods (such as nearest neighbor interpolation, bilinear interpolation, bicubic interpolation, and multi-phase interpolation).
[0116] In view of this, an embodiment of the present application provides an image processing method. In the process of processing an image, the image processing method first determines the edge direction and grayscale value of each pixel to be interpolated according to the gradient direction of each pixel to be interpolated corresponding to the original image. Since the gradient direction and the edge direction are perpendicular to each other, when the grayscale value in the gradient direction changes fastest, the grayscale value in the corresponding edge direction changes slowest. Based on this, in the edge direction of each pixel to be interpolated, interpolation is performed based on the grayscale value of the pixel to be interpolated, which can improve the continuity of the grayscale value change of the interpolated image, reduce the interpolation loss, and thus make the target image obtained after processing retain more image details.
[0117] Since the present application takes into account the edge characteristics of the image, the image processing method provided by the present application, in the process of processing the image, combines the edge characteristics of the image, can make the target image obtained after processing reflect the real contour information of the image, improve the quality of the processed image, and ensure that the processed image is clear and lossless. Compared with the related art, it effectively avoids the stripe (or aliasing) phenomenon in the processed image.
[0118] The hardware structure of the electronic device involved in the embodiments of the present application will be briefly introduced below with reference to the accompanying drawings.
[0119] In some embodiments of the present application, the electronic device may be a mobile phone, a tablet computer, a wearable device, a television, a vehicle-mounted device, an augmented reality (AR) / virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), etc. It may also be various camera devices such as a SLR camera and a compact camera, or it may be other devices or apparatuses capable of performing image processing. The embodiments of the present application do not impose any restrictions on the specific type of the electronic device.
[0120] Please refer to Figure 3 , Figure 3 FIG. 1 is a schematic diagram of the hardware structure of an electronic device according to an exemplary embodiment of the present application. Figure 3 As shown, the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.
[0121] It is to be understood that the structure shown in the embodiment of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include Figure 3 More or fewer components than those shown, or the electronic device 100 may include Figure 3 Combinations of some of the components shown, or the electronic device 100 may include Figure 3 Subassemblies of some of the components shown. Figure 3 The components shown may be implemented in hardware, software, or a combination of software and hardware.
[0122] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.
[0123] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.
[0124] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.
[0125] In an embodiment of the present application, the processor 110 may execute the steps of determining the edge direction of each pixel to be interpolated according to the gradient direction of each pixel to be interpolated, determining the gray value of each pixel to be interpolated, and interpolating based on the gray value of the pixel to be interpolated in the edge direction of each pixel to be interpolated to obtain a target image. For example, the processor 110 may run the software code of the image processing method provided in an embodiment of the present application to perform scaling processing on the image.
[0126] In an embodiment of the present application, the processor 110 may execute the steps of determining a zoom ratio of the grayscale image in response to a user's zoom operation, and determining each pixel to be interpolated corresponding to the grayscale image according to the zoom ratio and an interpolation algorithm.
[0127] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI) interface, a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.
[0128] It is understandable that the interface connection relationship between the modules shown in this embodiment is only a schematic illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.
[0129] The charging management module 140 is used to receive charging input from a charger. While the charging management module 140 is charging the battery 142 , it can also supply power to the electronic device 100 through the power management module 141 .
[0130] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the external memory, the display screen 194, the camera 193, and the wireless communication module 160. In some other embodiments, the power management module 141 may also be provided in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 may also be provided in the same device.
[0131] Figure 3 The connection relationship between the modules shown is only a schematic illustration and does not constitute a limitation on the connection relationship between the modules of the electronic device 100. Optionally, the modules of the electronic device 100 may also adopt a combination of multiple connection modes in the above embodiments.
[0132] The wireless communication function of the electronic device 100 can be implemented through components such as the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor, and the baseband processor.
[0133] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.
[0134] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 2G / 5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.
[0135] The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a receiver 170B, etc.), or displays an image or video through a display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.
[0136] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.
[0137] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology, etc. GNSS may include global positioning system (GPS), global navigation satellite system (GLONASS), Beidou navigation satellite system (BDS), quasi-zenith satellite system (QZSS) and / or satellite based augmentation system (SBAS). It is understood that in the embodiments of the present application, the hardware module in the positioning or navigation system may be referred to as a positioning sensor.
[0138] The electronic device 100 can realize the display function through the GPU, the display screen 194 and the application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU can also be used to perform mathematical and posture calculations for graphics rendering, etc. The processor 110 may include one or more GPUs, which can generate or change display information by executing program instructions.
[0139] The display screen 194 can be used to display images or videos, and can also display a series of graphical user interfaces (GUIs), which are the main screens of the electronic device 100. Generally speaking, the size of the display screen 194 of the electronic device 100 is fixed, and only limited controls can be displayed on the display screen 194 of the electronic device 100. A control is a GUI element, which is a software component included in an application program, and controls all data processed by the application program and interactive operations on the data. Users can interact with the control through direct manipulation to read or edit relevant information of the application program. Generally speaking, a control can include visual interface elements such as icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, etc.
[0140] In the embodiment of the present application, the display screen 194 can be used to display each image before, during and after processing.
[0141] The display screen 194 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N may be a positive integer greater than 1.
[0142] The display screen 194 in the embodiment of the present application may be a touch screen. A touch sensor 180K may be integrated in the display screen 194. The touch sensor 180K may also be referred to as a "touch panel". In other words, the display screen 194 may include a display panel and a touch panel, and the touch sensor 180K and the display screen 194 form a touch screen, also known as a "touch screen". The touch sensor 180K is used to detect touch operations acting on or near it. After the touch operation detected by the touch sensor 180K, it can be passed to the upper layer by the driver of the kernel layer (such as the TP driver) to determine the type of touch event. Visual output related to the touch operation can be provided by the display screen 194. In other embodiments, the touch sensor 180K may also be arranged on the surface of the electronic device 100, which is different from the position of the display screen 194.
[0143] In the embodiment of the present application, the touch sensor 180K detects the user's touch operation. For example, when the user is viewing an image in the gallery application, two fingers touch the display screen 194 at the same time and slide outward or pinch inward. At this time, the touch sensor 180K detects the user's touch operation, which is passed to the upper layer by the kernel layer driver (such as TP driver) to determine the type of touch event, such as image zoom event, image zoom event, etc. The processor 110 responds to the user's touch operation and finally provides visual output related to the touch operation through the display screen 194, such as the display screen 194 displays an enlarged image, a reduced image, etc.
[0144] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function, such as storing music, video and other files in the external memory card.
[0145] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 can include a program storage area and a data storage area. The program storage area can store an operating system (such as Figure 2 The storage data area can store data (such as audio data, phone book, etc.) created during the use of the electronic device 100.
[0146] In addition, the internal memory 121 may include a high-speed random access memory, such as Figure 2The memory in the hardware system shown; the internal memory 121 may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (Universal Flash Storage, UFS), etc.
[0147] The pressure sensor 180A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be set on the display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. A capacitive pressure sensor can be a parallel plate including at least two conductive materials. When a force acts on the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 100 determines the intensity of the pressure based on the change in capacitance. When a touch operation acts on the display screen 194, the electronic device 100 detects the intensity of the touch operation based on the pressure sensor 180A. The electronic device 100 can also calculate the position of the touch based on the detection signal of the pressure sensor 180A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities can correspond to different operation instructions.
[0148] The acceleration sensor 180E can detect the magnitude of the acceleration of the electronic device 100 in various directions (generally the x-axis, y-axis and z-axis). When the electronic device 100 is stationary, the magnitude and direction of gravity can be detected. The acceleration sensor 180E can also be used to identify the posture of the electronic device 100 as an input parameter for applications such as horizontal and vertical screen switching and pedometers.
[0149] The ambient light sensor 180L is used to sense the ambient light brightness. The electronic device 100 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the electronic device 100 is in a pocket to prevent accidental touches.
[0150] The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to realize functions such as unlocking, accessing application locks, taking photos, and answering calls.
[0151] The key 190 includes a power key, a volume key, etc. The key 190 may be a mechanical key or a touch key. The electronic device 100 may receive key input and generate key signal input related to user settings and function control of the electronic device 100.
[0152] Motor 191 can generate vibration prompts. Motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback.
[0153] The indicator 192 may be an indicator light, which may be used to indicate the charging status, power change, message, missed call, notification, etc. The SIM card interface 195 is used to connect the SIM card. The SIM card may be connected to or disconnected from the electronic device 100 by inserting the SIM card interface 195 or pulling the SIM card interface 195 out. The electronic device 100 may support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 may support Nano SIM card, Micro SIM card, SIM card, etc.
[0154] In addition, various types of operating systems are running on the above components, such as Android system, IOS operating system, Symbian operating system, BlackBerry operating system, Linux operating system, Windows operating system, etc. This is only an exemplary description and is not limited to this. Different applications can be installed and run on these operating systems, such as any application that can display and / or process images.
[0155] The image processing method provided in the embodiment of the present application can be implemented in the electronic device 100 having the above hardware structure.
[0156] The above briefly introduces the software structure of the electronic device involved in the embodiments of the present application. The following embodiments of the present application will take the electronic device as an example, and combine the accompanying drawings and application scenarios to specifically explain the image processing method provided in the embodiments of the present application.
[0157] Before introducing the image processing method, it is hereby stated that the above scenario of zooming in or out of an image when previewing an image in a gallery application is an example of an application scenario and does not impose any limitation on the application scenario of the present application. The image processing method provided in the embodiment of the present application can be applied to but not limited to the following scenarios:
[0158] Zooming images when previewing images in various applications (such as zooming images when previewing images in gallery applications, zooming images when viewing images in social applications, zooming avatars when switching avatars in applications, etc.), zooming videos in video calls, zooming videos in video conferencing applications, zooming videos in long and short video applications, zooming videos in live video applications, zooming videos in video online class applications, zooming images in smart camera application scenarios, zooming videos when recording videos with the system camera recording function, zooming photos when taking photos with the system camera shooting function, zooming images or videos in video surveillance, and zooming images or videos in smart cat's eyes, etc.
[0159] The following is an example of an application scenario in which an image is zoomed in or out when previewing the image in a gallery application.
[0160] See also Figure 4 , Figure 4 : is a flow chart of an image processing method provided in an embodiment of the present application. The method comprises:
[0161] S101: Determine the scaling ratio of the original image.
[0162] Detecting an operation for scaling, or receiving a scaling instruction for the original image, and determining a scaling ratio corresponding to the original image. In the implementation of the present application, the original image may be an RGB image, a YUV image, a grayscale image, etc., which is subject to actual conditions and is not limited thereto.
[0163] It is worth noting that when the original image is an RGB image, the subsequent scaling process of the RGB image is to scale the images of the R channel, G channel and B channel respectively, and then merge the processed images of the R channel, G channel and B channel to obtain the processed RGB image. Similarly, when the original image is a YUV image, the subsequent scaling process of the YUV image is to scale the images of the Y channel, U channel and V channel respectively, and then merge the processed images of the Y channel, U channel and V channel to obtain the processed YUV image. When the original image is a grayscale image, since the grayscale image is a single-channel image, the grayscale image can be directly scaled to obtain the processed grayscale image.
[0164] The scaling ratio may be any multiple of the resolution of the original image, for example, 0.2, 0.5, 1.5, 2, 2.5, etc., of the resolution of the original image.
[0165] The scaling instruction may be an image processing instruction, which is used to scale the original image to any multiple. The scaling instruction may include a zoom-in instruction and a zoom-out instruction.
[0166] In one possible implementation, the zoom instruction can be triggered by the user, and the zoom ratio can be determined by detecting the position where the user's finger touches. For example, when the user is viewing an image in a gallery application, two fingers simultaneously touch an area of the image displayed on the display screen and slide outward, triggering a zoom-in instruction. For another example, when the user is viewing an image in a gallery application, two fingers simultaneously touch an area of the image displayed on the display screen and pinch inward, triggering a zoom-out instruction. For another example, when editing an image, the user clicks on the zoom-in option (or zoom-out option) in the display interface of the electronic device to trigger a zoom-in instruction (or zoom-out instruction).
[0167] In another possible implementation, the zoom instruction may also be automatically triggered by the electronic device, and the zoom ratio may be determined by the screen resolution of the electronic device. For example, the screen provided by the electronic device for displaying images has a specified screen resolution. When the resolution of the original image to be displayed is different from the screen resolution, when the original image is opened, the electronic device will automatically trigger a zoom-in instruction or a zoom-out instruction to adapt the resolution of the original image to be displayed to the screen resolution. For example, if the resolution of the original image to be displayed is lower than the screen resolution, the electronic device needs to enlarge the original image, that is, trigger the zoom-in instruction to convert the original image into an image with a resolution equal to the screen resolution. Similarly, if the resolution of the original image to be displayed is higher than the screen resolution, the electronic device needs to reduce the original image, that is, trigger the zoom-out instruction to convert the original image into an image with a resolution equal to the screen resolution.
[0168] It should be understood that the electronic device can also display images or videos of different resolutions according to the actual needs of the user, and this is only an exemplary description and is not limited to this. Similarly, the triggering conditions of the zoom instruction are not limited in the embodiment of the present application.
[0169] S102: Determine each pixel to be interpolated corresponding to the original image according to the scaling ratio and the interpolation algorithm.
[0170] The interpolation algorithm may include nearest neighbor interpolation, bilinear interpolation, bicubic interpolation, and multi-phase interpolation, etc. In the embodiment of the present application, the bilinear interpolation method is used to process the original image as a grayscale image as an example for description.
[0171] For example, the position of each pixel in the zoomed image in the original image can be determined according to the zoom instruction. Assuming that the zoom instruction is a zoom-in instruction and the zoom ratio is 2, the position of each pixel in the zoomed image in the original image can be determined according to the zoom instruction.
[0172] The original image (assuming the resolution is a×b) is upsampled by 2 times (also called interpolation or magnification) using bilinear interpolation to obtain an enlarged image (resolution is 2a×2b). Upsampling the original image by 2 times using bilinear interpolation means that multiple pixels to be interpolated are inserted into the original image, where the pixel value of each pixel to be interpolated can be calculated by substituting the pixel values of multiple original pixels into the interpolation function.
[0173] It should be noted that determining each pixel point to be interpolated corresponding to the original image is equivalent to determining the grayscale value and coordinates of each pixel point to be interpolated.
[0174] For easier understanding, see Figure 5 , Figure 5It is a schematic diagram of interpolation of an amplified image provided in an embodiment of the present application.
[0175] Figure 5 (a) shows the unscaled grayscale image. Figure 5 (b) shows an enlarged image, i.e. Figure 5 The grayscale image shown in (a) is a grayscale image after it is magnified 2 times. Figure 5 It can be seen that Figure 5 Each pixel in the grayscale image shown in (a) is Figure 5 There are corresponding ones in the enlarged image shown in (b), that is, Figure 5 The black pixels in the grayscale image shown in (a) are Figure 5 The black pixels in the enlarged image shown in (b) correspond one to one. Figure 5 The gray pixels in the enlarged image shown in (b) are the newly inserted pixels to be interpolated. The pixel value of each pixel to be interpolated is obtained by dividing the original pixels (i.e. Figure 5 The pixel value of the black pixel in the grayscale image shown in (a) is substituted into the interpolation function to obtain the result.
[0176] The following describes how to determine each pixel to be interpolated corresponding to the original image.
[0177] For easier understanding, see Figure 6 , Figure 6 It is a schematic diagram of a bilinear interpolation algorithm provided in an embodiment of the present application.
[0178] like Figure 6 As shown, for each pixel to be interpolated, in this example, the pixel to be interpolated is point O, the four adjacent (or nearest) pixels of the pixel to be interpolated in the grayscale image that is not scaled are determined according to the scaling ratio, which are P A , P B , P C , P D The coordinates of these four adjacent pixels are P A (x,y),P B (x+1,y),P C (x,y+1),P D (x+1,y+1).
[0179] like Figure 6 As shown, P A , P B , P C , P D Four adjacent pixels form a square. A horizontal line is drawn through point O. The horizontal line intersects with the square to form two intersection points, which are intersection points P and P. row_1 , intersection point Prow_2 Similarly, draw a vertical line through point O, which intersects the square to form two intersection points, namely intersection point P col_1 , intersection point P col_2 .
[0180] like Figure 6 As shown, P B With the intersection point P row_2 The distance between them is q, and the intersection point P row_2 With P D The distance between them is 1-q, P C With the intersection point P col_2 The distance between them is p, and the intersection point is P col_2 With P D The distance between them is 1-p.
[0181] Get the pixel value of each adjacent pixel point, and then calculate the pixel point to be interpolated (such as Figure 6 The pixel value of point O shown in the figure, the interpolation function is as follows:
[0182] P(x,y)=Pcol_1+(Pcol_2-Pcol_1)×q=(1-p)(1-q)P([x],[y])+p(1-q)P([x]+1,[y])+(1-p)qP([x],[y]+1)+pqP([x]+1,[y]+1)
[0183] In the above interpolation function, P(x,y) represents the pixel value of the pixel to be interpolated, P col_1 Indicates intersection col_1 The pixel value, P col_2 Indicates the intersection point P col_2 The pixel value of P B With the intersection point P row_2 The distance between them, 1-q represents the intersection point P row_2 With P D The distance between them, p represents P C With the intersection point P col_2 The distance between them, 1-p represents the intersection point P col_2 With P D The distance between, P([x],[y]) represents P A The pixel value of P([x]+1,[y]) represents P B The pixel value of P([x],[y]+1) represents P C The pixel value of P([x]+1,[y]+1) represents P D The pixel value of .
[0184] It should be understood that for a grayscale image, the pixel value of its pixel is the grayscale value. Therefore, the pixel value of the pixel to be interpolated calculated by the interpolation function is the grayscale value of the pixel to be interpolated. It is worth noting that in the embodiment of the present application, the grayscale value of the pixel to be interpolated determined by the traditional interpolation algorithm is called the initial grayscale value.
[0185] In this implementation, each pixel to be interpolated corresponding to the original image is first determined by a traditional interpolation algorithm, and the initial grayscale value of each pixel to be interpolated is determined, which provides a basis for the subsequent rapid and accurate determination of the gradient direction, edge direction and grayscale value of the pixel to be interpolated, thereby ensuring that a high-quality image is obtained after interpolation based on the gradient direction and grayscale value of the pixel to be interpolated.
[0186] It is worth noting that the bilinear interpolation algorithm does not consider the edge characteristics of the image during the interpolation process, and only interpolates based on the position of each pixel. The resulting image will have stripes (or jagged edges), which makes the details of the enlarged image unclear and cannot reflect the true contour information of the image, seriously affecting the quality of the enlarged image.
[0187] In view of this, the image processing method provided in the embodiment of the present application further adds S201 to S203 after the above step S102 to improve the image quality. Figure 7 , Figure 7 is a flow chart of an image processing method provided in an embodiment of the present application. Figure 7 As shown, the image processing method includes S201 to S203.
[0188] S201: Determine the edge direction of each pixel to be interpolated according to the gradient direction of each pixel to be interpolated corresponding to the original image.
[0189] Exemplarily, the original image corresponds to a plurality of pixels to be interpolated, and these pixels to be interpolated are determined by the above steps S101 and S102.
[0190] For each pixel to be interpolated, determine the gradient direction of the pixel to be interpolated, and determine the edge direction of the pixel to be interpolated according to the gradient direction of the pixel to be interpolated, wherein the edge direction of the pixel to be interpolated is perpendicular to the gradient direction of the pixel to be interpolated.
[0191] The gradient direction of the pixel to be interpolated indicates the direction of change of the grayscale value of the pixel to be interpolated. For the pixel to be interpolated, the grayscale value changes fastest along the gradient direction, and the grayscale value changes slowest along the edge direction. Therefore, if interpolation is performed in the edge direction, the higher the continuity of the interpolated image, the smaller the interpolation loss, thereby retaining more image details, making the final target image clear, lossless and high-quality.
[0192] S202: Determine the grayscale value of each pixel to be interpolated.
[0193] It should be noted that the grayscale value of each pixel to be interpolated is also calculated in step S102 , but the grayscale value of each pixel to be interpolated in step S102 is different from the grayscale value of each pixel to be interpolated determined in step S202 .
[0194] In step S102, the grayscale value of each pixel to be interpolated is calculated by a traditional interpolation algorithm. The grayscale value of the pixel to be interpolated in step S102 is an initial grayscale value, which is not used for the final interpolation. The grayscale value of the pixel to be interpolated determined in step S202 is calculated based on the initial grayscale value of each pixel to be interpolated, the edge direction of each pixel to be interpolated, etc., and is the final grayscale value used for interpolation.
[0195] S203: performing interpolation based on the grayscale value of each pixel to be interpolated in the edge direction of the pixel to be interpolated, to obtain a target image corresponding to the original image.
[0196] It should be understood that when interpolation is performed based on the grayscale value of each pixel to be interpolated in the edge direction of each pixel to be interpolated, different interpolation methods correspond to different scaling instructions.
[0197] For example, when the scaling instruction is a zoom-in instruction, up-interpolation is performed based on the grayscale value of each pixel to be interpolated in the edge direction of each pixel to be interpolated to obtain a target image corresponding to the original image.
[0198] It can be generally understood that when the scaling instruction is a zoom instruction, multiple pixels to be interpolated are inserted into the original image. The method of inserting multiple pixels to be interpolated is to insert the pixels to be interpolated based on the grayscale value of the pixels to be interpolated in the edge direction of each pixel to be interpolated. After the multiple pixels to be interpolated are inserted, the resulting image is the target image.
[0199] For another example, when the scaling instruction is a reduction instruction, downward interpolation is performed based on the grayscale value of each pixel to be interpolated in the edge direction of each pixel to be interpolated to obtain a target image corresponding to the original image.
[0200] It can be generally understood that when the zoom instruction is a reduction instruction, an image is reassembled according to a plurality of pixels to be interpolated, and the assembled image is the target image. In the edge direction of each pixel to be interpolated, the pixel to be interpolated is inserted into the new image based on the grayscale value of the pixel to be interpolated to obtain the target image. That is, when the zoom instruction is a reduction instruction, the target image is composed of a plurality of pixels to be interpolated, each pixel to be interpolated is in its corresponding edge direction, and the grayscale value of each pixel to be interpolated is the grayscale value determined in step S202.
[0201] The image processing method provided by the embodiment of the present application, in the process of processing the image, first determines the edge direction and grayscale value of each pixel to be interpolated according to the gradient direction of each pixel to be interpolated corresponding to the original image. Since the gradient direction and the edge direction are perpendicular to each other, when the grayscale value in the gradient direction changes fastest, the grayscale value in the corresponding edge direction changes slowest. Based on this, in the edge direction of each pixel to be interpolated, interpolation is performed based on the grayscale value of the pixel to be interpolated, which can improve the continuity of the grayscale value change of the interpolated image, reduce the interpolation loss, and thus make the target image obtained after processing retain more image details.
[0202] Since the present application takes into account the edge characteristics of the image, the image processing method provided by the present application, in the process of processing the image, combines the edge characteristics of the image, can make the target image obtained after processing reflect the real contour information of the image, improve the quality of the processed image, and ensure that the processed image is clear and lossless. Compared with the related art, it effectively avoids the stripe (or aliasing) phenomenon in the processed image.
[0203] See also Figure 8 , Figure 8 FIG. 1 is a flow chart of another image processing method provided in an embodiment of the present application. Figure 8 As shown, the image processing method may include S301 to S304.
[0204] S301: Determine the gradient direction of each pixel to be interpolated corresponding to the original image.
[0205] S302: Determine the edge direction of each pixel to be interpolated according to the gradient direction of each pixel to be interpolated corresponding to the original image.
[0206] S303: Determine the grayscale value of each pixel to be interpolated.
[0207] S304: performing interpolation based on the grayscale value of each pixel to be interpolated in the edge direction of the pixel to be interpolated to obtain a target image corresponding to the original image.
[0208] It is worth noting that steps S302 to S304 in this embodiment are similar to Figure 7 Steps S201 to S203 in the corresponding embodiment are consistent, and can be referred to above Figure 7 The description of steps S201 to S203 in the corresponding embodiment will not be repeated here about steps S302 to S304. Step S301 will be described in detail below.
[0209] Determining the gradient direction of each pixel to be interpolated corresponding to the original image is conducive to the subsequent rapid and accurate determination of the edge direction based on the gradient direction. For the pixel to be interpolated, the grayscale value along the gradient direction changes the fastest, and the grayscale value along the edge direction perpendicular to the gradient direction changes the slowest. Therefore, interpolation is performed based on the edge direction determined by the gradient direction, so that the continuity of the interpolated image is higher and the interpolation loss is smaller, thereby retaining more image details, effectively avoiding the appearance of stripes (or aliasing) in the processed image, and making the final target image clear, lossless and high-quality.
[0210] In a possible implementation, determining the gradient direction of each pixel to be interpolated corresponding to the original image includes: determining the grayscale gradient component of each pixel to be interpolated, performing vector synthesis on multiple grayscale gradient components of each pixel to be interpolated, and obtaining the grayscale gradient of each pixel to be interpolated.
[0211] The grayscale gradient component indicates the grayscale gradient of the pixel to be interpolated in a first direction, where the first direction is the direction from the neighboring pixel points of the pixel to be interpolated to the pixel to be interpolated.
[0212] For each pixel to be interpolated, the position of the pixel to be interpolated is mapped to the original image, and the pixels adjacent to the pixel to be interpolated in the original image are the adjacent pixels of the pixel to be interpolated.
[0213] For example, in different implementations, the number of adjacent pixels of each pixel to be interpolated is not limited and can be set and adjusted by the user according to actual conditions. The number of grayscale gradient components of each pixel to be interpolated corresponds to the number of adjacent pixels of each pixel to be interpolated.
[0214] For example, when the number of adjacent pixels of a pixel to be interpolated is 4, the number of grayscale gradient components of the pixel to be interpolated is 4. For another example, when the number of adjacent pixels of a pixel to be interpolated is 6, the number of grayscale gradient components of the pixel to be interpolated is 6. For another example, when the number of adjacent pixels of a pixel to be interpolated is 8, the number of grayscale gradient components of the pixel to be interpolated is 8. This is only an exemplary description and is not limited to this.
[0215] Exemplarily, a line is drawn between the pixel to be interpolated and each adjacent pixel, and the opposite direction of the line is the first direction corresponding to the adjacent pixel. The grayscale gradient of the pixel to be interpolated in the first direction is the grayscale gradient component of the pixel to be interpolated in the first direction. For each pixel to be interpolated, multiple grayscale gradient components corresponding to the pixel to be interpolated are obtained in this way, and vector synthesis of the multiple grayscale gradient components is performed to obtain the grayscale gradient of the pixel to be interpolated.
[0216] In this implementation, the grayscale gradient of the pixel to be interpolated is obtained by vector synthesis of multiple grayscale gradient components of the pixel to be interpolated. Since adjacent pixels are distributed in different directions, the method of vector synthesis through multiple grayscale gradient components can take into account the grayscale change characteristics of the pixel to be interpolated in different directions. The gradient direction of the pixel to be interpolated thus determined is more accurate, which is conducive to the subsequent accurate determination of the edge direction of the pixel to be interpolated based on the gradient direction.
[0217] Exemplarily, the grayscale gradient component of each pixel to be interpolated can be determined by, for each pixel to be interpolated, determining multiple adjacent pixels adjacent to the pixel to be interpolated in the original image, and the grayscale value of each adjacent pixel; and calculating multiple grayscale gradient components of the pixel to be interpolated based on the initial grayscale value of the pixel to be interpolated and the grayscale values of multiple adjacent pixels.
[0218] In the embodiment of the present application, the number of adjacent pixels is 4 as an example for description.
[0219] For easier understanding, see Fig. 9 , Fig. 9 It is a schematic diagram of grayscale gradient components provided in an embodiment of the present application.
[0220] Assume that point O is any pixel to be interpolated corresponding to the original image, such as Figure 8 As shown, the pixel to be interpolated is mapped to the original image, and the four adjacent pixels (or the nearest) to the pixel to be interpolated are A, B, C, and D. The coordinates of these four adjacent pixels are A(x, y), B(x+1, y), C(x, y+1), and D(x+1, y+1). It should be understood that the four adjacent pixels A, B, C, and D are the pixels in the original image. The four adjacent pixels A, B, C, and D form a square, and the pixel to be interpolated falls into the square.
[0221] like Fig. 9 As shown, a line is drawn between the pixel to be interpolated (point O) and each adjacent pixel, and the opposite direction of the line is the first direction corresponding to the adjacent pixel. The grayscale gradient of the pixel to be interpolated in the first direction is the grayscale gradient component of the pixel to be interpolated in the first direction.
[0222] like Fig. 9 As shown, the lines connecting the pixel to be interpolated (point O) and the four adjacent pixel points A, B, C, and D are drawn. A Represents the grayscale gradient component of the pixel to be interpolated (point O) in the AO direction, G B G represents the gray gradient component of the pixel to be interpolated (point O) in the BO direction, C represents the gray gradient component of the pixel to be interpolated (point O) in the CO direction, G D Represents the grayscale gradient component of the pixel to be interpolated (point O) in the DO direction.
[0223] Based on the initial grayscale value of the pixel to be interpolated, the grayscale value of each adjacent pixel and the first preset formula, multiple grayscale gradient components of the pixel to be interpolated are calculated. Since the adjacent pixel points are pixel points in the original image, the grayscale value of each adjacent pixel point can be directly obtained, and the initial grayscale value of the pixel to be interpolated can be determined by the above step S102.
[0224] Among them, the first preset formula is as follows:
[0225]
[0226] In the above formula (1), i represents each adjacent pixel point, and the value of i is A, B, C, D. represents the gray gradient component of the pixel to be interpolated (point O) in the first direction corresponding to different adjacent pixels, Represents the derivative of gray value and distance, represents a unit length vector, whose direction is from the adjacent pixel point to the pixel to be interpolated (point O), P o Represents the gray value of the pixel to be interpolated (point O), P i Represents the gray value of each adjacent pixel, r i Represents the distance between the i-th adjacent pixel and the pixel to be interpolated (point O).
[0227] For example, by substituting i=A into the above formula (1), the gray gradient component of the pixel to be interpolated (point O) in the AO direction can be obtained as:
[0228]
[0229] For another example, by substituting i=B into the above formula (1), the gray gradient component of the pixel to be interpolated (point O) in the BO direction can be obtained as:
[0230]
[0231] For another example, by substituting i=C into the above formula (1), the gray gradient component of the pixel to be interpolated (point O) in the CO direction can be obtained as:
[0232]
[0233] For another example, by substituting i=D into the above formula (1), the gray gradient component of the pixel to be interpolated (point O) in the DO direction can be obtained as:
[0234]
[0235] In this implementation, the grayscale gradient component of the pixel to be interpolated is determined based on multiple adjacent pixels adjacent to the pixel to be interpolated and the grayscale value of each adjacent pixel. In this process, the grayscale change characteristics of the adjacent pixels of the pixel to be interpolated in different directions are taken into account, so that the determined grayscale gradient component is more accurate, which is conducive to the subsequent accurate synthesis of the gradient direction of the pixel to be interpolated based on the grayscale gradient component.
[0236] After calculating multiple grayscale gradient components of the pixel to be interpolated, vector synthesis is performed on the multiple grayscale gradient components of the pixel to be interpolated to obtain the grayscale gradient of the pixel to be interpolated, and the grayscale gradient includes a gradient direction.
[0237] For easier understanding, see Fig.10 , Fig.10 It is a vector synthesis schematic diagram provided in an embodiment of the present application.
[0238] right Perform vector synthesis and obtain Fig.10 The gray gradient G of the pixel to be interpolated (point O) is shown.
[0239] like Fig.10 As shown, a coordinate system is established with the pixel point to be interpolated (point O) as the origin, and the angle formed by the x-axis and the straight line where the gradient direction of the pixel point to be interpolated (point O) is located is α, which is the gradient direction angle of the pixel point to be interpolated (point O).
[0240] Exemplarily, the specific vector synthesis process can be implemented by a second preset formula, and the second preset formula is as follows:
[0241]
[0242]
[0243]
[0244] In the above formula (2), the value of i is A, B, C, D, G xi Represents G iThe component on the x-axis, G yi Represents G i The component on the y-axis, α, represents the gradient direction angle of the pixel to be interpolated (point O).
[0245] In this implementation, the gradient direction of the pixel to be interpolated is obtained by vector synthesis of multiple grayscale gradient components of the pixel to be interpolated. In this process, the grayscale gradient components of the pixel to be interpolated in different directions are taken into account, so that the gradient direction obtained by vector synthesis of multiple grayscale gradient components is more accurate.
[0246] In another possible implementation, the gradient direction of each pixel to be interpolated corresponding to the original image is determined, including: determining a weight value between each pixel to be interpolated and each adjacent pixel, determining a grayscale gradient component of each pixel to be interpolated according to each weight value, and performing vector synthesis on multiple grayscale gradient components of each pixel to be interpolated to obtain a grayscale gradient of each pixel to be interpolated.
[0247] Exemplarily, the weight value between each pixel to be interpolated and each adjacent pixel is determined by the distance between each pixel to be interpolated and each adjacent pixel. The closer the distance between the pixel to be interpolated and the adjacent pixel, the greater the weight value corresponding to the adjacent pixel, and the farther the distance between the pixel to be interpolated and the adjacent pixel, the smaller the weight value corresponding to the adjacent pixel.
[0248] In an embodiment of the present application, different distance values may be pre-set to correspond to different weight values. For example, when the distance value between the pixel to be interpolated and the adjacent pixel is within a first distance range, the weight value corresponding to the adjacent pixel is set to W1; when the distance value between the pixel to be interpolated and the adjacent pixel is within a second distance range, the weight value corresponding to the adjacent pixel is set to W2. For another example, when the distance value between the pixel to be interpolated and the adjacent pixel is less than or equal to a preset distance threshold, the weight value corresponding to the adjacent pixel is set to W1; when the distance value between the pixel to be interpolated and the adjacent pixel is greater than the preset distance threshold, the weight value corresponding to the adjacent pixel is set to W2. This is only an exemplary description and is not limited to this.
[0249] Exemplarily, for each pixel to be interpolated, when the pixel to be interpolated is determined in the above S102, the coordinates of the pixel to be interpolated can be obtained. At the same time, since the adjacent pixel points are pixel points in the original image, the coordinates of each adjacent pixel point can be directly obtained. According to the coordinates of the pixel to be interpolated and the coordinates of each adjacent pixel point, the distance value between the pixel to be interpolated and each adjacent pixel point can be calculated. Combined with the correspondence between different preset distance values and different weight values, the weight value corresponding to each adjacent pixel point is determined.
[0250] Based on the initial grayscale value of the pixel to be interpolated, the weight value between each pixel to be interpolated and each adjacent pixel, the grayscale value of each adjacent pixel, and the second preset formula, multiple grayscale gradient components of the pixel to be interpolated are calculated. Since the adjacent pixel is a pixel in the original image, the grayscale value of each adjacent pixel can be directly obtained, and the initial grayscale value of the pixel to be interpolated can be determined by the above step S102.
[0251] Among them, the second preset formula is as follows:
[0252]
[0253] In the above formula (3), i represents each adjacent pixel point, and the value of i is A, B, C, D. represents the gray gradient component of the pixel to be interpolated (point O) in the first direction corresponding to different adjacent pixels, Represents the derivative of gray value and distance, represents a unit length vector, whose direction is from the adjacent pixel point to the pixel to be interpolated (point O), P o Represents the gray value of the pixel to be interpolated (point O), P i Represents the gray value of each adjacent pixel, W i Represents the weight value corresponding to each adjacent pixel, r i Represents the distance between the i-th adjacent pixel and the pixel to be interpolated (point O).
[0254] For example, by substituting i=A into the above formula (3), the grayscale gradient component of the pixel to be interpolated (point O) in the AO direction can be obtained as:
[0255]
[0256] For another example, by substituting i=B into the above formula (3), the gray gradient component of the pixel to be interpolated (point O) in the BO direction can be obtained as:
[0257]
[0258] For another example, by substituting i=C into the above formula (3), the gray gradient component of the pixel to be interpolated (point O) in the CO direction can be obtained as:
[0259]
[0260] For another example, by substituting i=D into the above formula (3), the gray gradient component of the pixel to be interpolated (point O) in the DO direction can be obtained as:
[0261]
[0262] After calculating the multiple grayscale gradient components of the pixel to be interpolated, the multiple grayscale gradient components of the pixel to be interpolated are vector synthesized to obtain the grayscale gradient of the pixel to be interpolated. The specific vector synthesis process can be implemented by the second preset formula above, which will not be repeated here.
[0263] In this implementation, when determining the grayscale gradient of the pixel to be interpolated, not only the grayscale change characteristics of the adjacent pixels of the pixel to be interpolated in different directions are taken into account, but also the grayscale values of the adjacent pixels are adjusted based on the weight values corresponding to different adjacent pixels, so that the determined grayscale gradient component is more accurate, and thus the gradient direction synthesized based on the grayscale gradient component is also more accurate.
[0264] See also Fig.11 , Fig.11 Schematic diagram of the process of calculating the grayscale value of the pixel to be interpolated provided by the embodiment of the present application. Fig.11 As shown, the above S202 may include S2021 and S2022.
[0265] S2021: Determine the edge direction angle of each pixel to be interpolated.
[0266] The edge direction angle refers to the angle formed by the x-axis and the straight line where the edge direction of the pixel to be interpolated is located, which is determined by using the x-axis direction as the standard direction.
[0267] In one possible implementation, the edge direction angle can be used to determine the slope of the straight line where the edge direction lies. Therefore, any angle formed by the x-axis and the straight line where the edge direction of the pixel to be interpolated lies can be used as the edge direction angle of the pixel to be interpolated as long as the slope of the straight line where the edge direction lies can be determined. The specific angle shall be based on the actual situation and is not limited to this.
[0268] In another possible implementation, the edge direction angle of each pixel to be interpolated may be determined by determining the gradient direction angle of each pixel to be interpolated, and determining the edge direction angle of each pixel to be interpolated based on the gradient direction angle of each pixel to be interpolated.
[0269] The gradient direction angle is the angle between the pixel to be interpolated in the horizontal direction and the gradient direction.
[0270] For easier understanding, see Fig.12 , Fig.12 This is a schematic diagram of an edge direction angle provided in an embodiment of the present application.
[0271] like Fig.12As shown in the figure, a straight line is drawn through the pixel to be interpolated (point O) and is perpendicular to the gradient direction of the pixel to be interpolated (point O). The direction of the straight line is the edge direction of the pixel to be interpolated (point O). The straight line intersects the square formed by the four adjacent pixel points A, B, C, and D at points M and N.
[0272] like Fig.12 As shown, the angle formed by the x-axis and the straight line where the gradient direction of the pixel point to be interpolated (point O) is located is α, and α is the gradient direction angle of the pixel point to be interpolated (point O).
[0273] like Fig.12 As shown, the x-axis and the straight line where the edge direction of the pixel point to be interpolated (point O) is located form an apparent angle θ, and θ is the edge direction angle of the pixel point to be interpolated (point O).
[0274] It can be understood that the straight line where the gradient direction of the pixel to be interpolated (point O) is located is perpendicular to the straight line where the edge direction of the pixel to be interpolated (point O) is located, and the angle formed by the straight line OM and the straight line where the gradient direction of the pixel to be interpolated (point O) is Then the edge direction angle θ can be obtained by the gradient direction angle and OK. For example,
[0275] In this implementation, the edge direction angle of each pixel to be interpolated is determined according to the gradient direction angle of each pixel to be interpolated, which is conducive to the subsequent accurate calculation of the grayscale value of the pixel to be interpolated according to the edge direction angle. Therefore, when interpolation is performed based on the grayscale value, the continuity of the interpolated image can be higher, the interpolation loss can be smaller, more image details can be retained, and the final target image can be clear, lossless and of high quality.
[0276] S2022: Calculate the grayscale value of each pixel to be interpolated according to the grayscale value of each adjacent pixel, the edge direction of each pixel to be interpolated, and the edge direction angle of each pixel to be interpolated.
[0277] It should be understood that the adjacent pixel points are the pixel points in the original image, so the grayscale value of each adjacent pixel point can be directly obtained.
[0278] Exemplarily, for each pixel to be interpolated, determine the intersection point generated by the intersection of the straight line where the edge direction of the pixel to be interpolated is located and the square (the square surrounded by the four adjacent pixels of the pixel to be interpolated). Determine the distance between the pixel to be interpolated and the intersection point. Based on the grayscale value of each adjacent pixel point, calculate the grayscale value of the intersection point. According to the distance between the pixel to be interpolated and the intersection point, and the grayscale value of the intersection point, calculate the grayscale value of the pixel to be interpolated.
[0279] In this implementation, the gray value of each pixel to be interpolated is accurately calculated based on the gray value of each adjacent pixel, the edge direction of each pixel to be interpolated, and the edge direction angle of each pixel to be interpolated. Therefore, when interpolation is performed based on the gray value, the continuity of the interpolated image can be higher, the interpolation loss can be smaller, more image details can be retained, and the final target image can be clear, lossless, and of high quality.
[0280] In the embodiment of the present application, the number of adjacent pixel points is 4 as an example for description. Therefore, the plurality of adjacent pixel points may include a first adjacent pixel point, a second adjacent pixel point, a third adjacent pixel point, and a fourth adjacent pixel point.
[0281] As an example of the present application, the specific implementation of the above S2022 may include S20221 to S20226:
[0282] S20221: For each pixel to be interpolated, determine a first intersection point and a second intersection point according to an edge direction of the pixel to be interpolated and a plurality of adjacent pixels.
[0283] The first intersection point, the second intersection point, and the pixel point to be interpolated are collinear, or in other words, the first intersection point, the second intersection point, and the pixel point to be interpolated are on a straight line where the edge direction is located.
[0284] In a possible implementation, for each pixel to be interpolated, a first intersection is determined according to the edge direction of the pixel to be interpolated, the first adjacent pixel, and the third adjacent pixel. The first intersection and the pixel to be interpolated are on a straight line where the edge direction is located.
[0285] For easier understanding, see Fig.13 , Fig.13 It is a schematic diagram of an intersection provided in an embodiment of the present application.
[0286] like Fig.13 As shown in the figure, A represents the first adjacent pixel point, B represents the second adjacent pixel point, C represents the third adjacent pixel point, and D represents the fourth adjacent pixel point. The straight line where the edge direction of the pixel point to be interpolated (point O) is located intersects with the straight lines where the first adjacent pixel point and the third adjacent pixel point are located to form a first intersection point. That is, MN intersects with AC to form a first intersection point M.
[0287] The second intersection point is determined according to the edge direction of the pixel to be interpolated, the second adjacent pixel point and the fourth adjacent pixel point. The second intersection point and the pixel to be interpolated are on a straight line where the edge direction is located.
[0288] like Fig.13As shown, the straight line where the edge direction of the pixel to be interpolated (point O) is located intersects with the straight lines where the second adjacent pixel and the fourth adjacent pixel are located to form a second intersection point. That is, MN intersects with BD to form a second intersection point N.
[0289] In another possible implementation, for each pixel to be interpolated, a first intersection is determined according to the edge direction of the pixel to be interpolated, the first adjacent pixel, and the second adjacent pixel. The first intersection and the pixel to be interpolated are on a straight line where the edge direction is located.
[0290] For easier understanding, see Fig.14 , Fig.14 This is another intersection schematic diagram provided in an embodiment of the present application.
[0291] like Fig.14 As shown in the figure, A represents the first adjacent pixel point, B represents the second adjacent pixel point, C represents the third adjacent pixel point, and D represents the fourth adjacent pixel point. The straight line where the edge direction of the pixel point to be interpolated (point O) is located intersects with the straight lines where the first adjacent pixel point and the second adjacent pixel point are located to form a first intersection point. That is, MN intersects with AB to form a first intersection point M.
[0292] like Fig.14 As shown, the straight line where the edge direction of the pixel to be interpolated (point O) is located intersects with the straight lines where the second adjacent pixel and the fourth adjacent pixel are located to form a second intersection point. That is, MN intersects with BD to form a second intersection point N.
[0293] In another possible implementation, for each pixel to be interpolated, a first intersection is determined according to the edge direction of the pixel to be interpolated, the first adjacent pixel, and the third adjacent pixel. The first intersection and the pixel to be interpolated are on a straight line where the edge direction is located.
[0294] For easier understanding, see Fig.15 , Fig.15 This is another intersection schematic diagram provided in an embodiment of the present application.
[0295] like Fig.15 As shown in the figure, A represents the first adjacent pixel point, B represents the second adjacent pixel point, C represents the third adjacent pixel point, and D represents the fourth adjacent pixel point. The straight line where the edge direction of the pixel point to be interpolated (point O) is located intersects with the straight lines where the first adjacent pixel point and the third adjacent pixel point are located to form a first intersection point. That is, MN intersects with AC to form a first intersection point M.
[0296] like Fig.15 As shown, the straight line where the edge direction of the pixel to be interpolated (point O) is located intersects with the straight lines where the third adjacent pixel point and the fourth adjacent pixel point are located to form a second intersection point. That is, MN intersects with CD to form a second intersection point N.
[0297] In another possible implementation, for each pixel to be interpolated, a first intersection is determined according to the edge direction of the pixel to be interpolated, the first adjacent pixel, and the third adjacent pixel. The first intersection and the pixel to be interpolated are on a straight line where the edge direction is located.
[0298] For easier understanding, see Fig.16 , Fig.16 This is another intersection schematic diagram provided in an embodiment of the present application.
[0299] like Fig.16 As shown in the figure, A represents the first adjacent pixel point, B represents the second adjacent pixel point, C represents the third adjacent pixel point, and D represents the fourth adjacent pixel point. The straight line where the edge direction of the pixel point to be interpolated (point O) is located intersects with the straight lines where the first adjacent pixel point and the third adjacent pixel point are located to form a first intersection point. That is, MN intersects with AC to form a first intersection point M.
[0300] like Fig.16 As shown, the straight line where the edge direction of the pixel to be interpolated (point O) is located intersects with the straight lines where the first adjacent pixel and the second adjacent pixel are located to form a second intersection point. That is, MN intersects with AB to form a second intersection point N.
[0301] In another possible implementation, for each pixel to be interpolated, a first intersection is determined according to the edge direction of the pixel to be interpolated, the third adjacent pixel, and the fourth adjacent pixel. The first intersection and the pixel to be interpolated are on a straight line where the edge direction is located.
[0302] For easier understanding, see Fig.17 , Fig.17 This is another intersection schematic diagram provided in an embodiment of the present application.
[0303] like Fig.17 As shown in the figure, A represents the first adjacent pixel point, B represents the second adjacent pixel point, C represents the third adjacent pixel point, and D represents the fourth adjacent pixel point. The straight line where the edge direction of the pixel point to be interpolated (point O) is located intersects with the straight lines where the third adjacent pixel point and the fourth adjacent pixel point are located to form a first intersection point. That is, MN intersects with CD to form a first intersection point M.
[0304] like Fig.17 As shown, the straight line where the edge direction of the pixel to be interpolated (point O) is located intersects with the straight lines where the second adjacent pixel and the fourth adjacent pixel are located to form a second intersection point. That is, MN intersects with BD to form a second intersection point N.
[0305] In another possible implementation, for each pixel to be interpolated, a first intersection is determined according to the edge direction of the pixel to be interpolated, the first adjacent pixel, and the second adjacent pixel. The first intersection and the pixel to be interpolated are on a straight line where the edge direction is located.
[0306] For easier understanding, see Fig.18 , Fig.18 This is another intersection schematic diagram provided in an embodiment of the present application.
[0307] like Fig.18 As shown in the figure, A represents the first adjacent pixel point, B represents the second adjacent pixel point, C represents the third adjacent pixel point, and D represents the fourth adjacent pixel point. The straight line where the edge direction of the pixel point to be interpolated (point O) is located intersects with the straight lines where the first adjacent pixel point and the second adjacent pixel point are located to form a first intersection point. That is, MN intersects with AB to form a first intersection point M.
[0308] like Fig.18 As shown, the straight line where the edge direction of the pixel to be interpolated (point O) is located intersects with the straight lines where the third adjacent pixel point and the fourth adjacent pixel point are located to form a second intersection point. That is, MN intersects with CD to form a second intersection point N.
[0309] It should be understood that different implementation scenarios form intersections in different ways, and the above is only an exemplary description, which is subject to the actual situation and is not limited to this.
[0310] S20222: Determine a first distance and a second distance according to an edge direction angle of the pixel to be interpolated.
[0311] The first distance represents the distance between the pixel point to be interpolated and the first intersection point, and the second distance represents the distance between the pixel point to be interpolated and the second intersection point.
[0312] like Fig.13 As shown, it is assumed that a coordinate system is established with the pixel point to be interpolated (point O) as the coordinate origin, the horizontal distance between the pixel point to be interpolated (point O) and the adjacent pixel point in the upper left corner (i.e., the first adjacent pixel point) is p, and the vertical distance between the pixel point to be interpolated (point O) and the adjacent pixel point in the upper left corner (i.e., the first adjacent pixel point) is q.
[0313] Assume that the vertical distance between the first adjacent pixel point and the third adjacent pixel point is 1, the horizontal distance between the first adjacent pixel point and the second adjacent pixel point is 1, the vertical distance between the second adjacent pixel point and the fourth adjacent pixel point is 1, and the horizontal distance between the third adjacent pixel point and the fourth adjacent pixel point is 1.
[0314] Based on the above-mentioned set distances, it can be determined that the coordinates of the first adjacent pixel point are A(-p, -q), the coordinates of the second adjacent pixel point are B(1-p, -q), the coordinates of the third adjacent pixel point are C(-p, 1-q), and the coordinates of the third adjacent pixel point are D(1-p, 1-q).
[0315] Assuming that the slope of the straight line where the edge direction of the pixel to be interpolated (point O) lies is k, the value of k is calculated by the following formula (3).
[0316]
[0317] In the above formula (4), k represents the slope of the straight line where the edge direction of the pixel point to be interpolated (point O) is located, and θ is the edge direction angle of the pixel point to be interpolated (point O).
[0318] According to mathematical knowledge, we can calculate that the coordinates of the first intersection point M are (-p, -p*tanθ), and the coordinates of the second intersection point N are (1-p, (1-p)*tanθ).
[0319] like Fig.13 As shown, the first distance between the pixel point to be interpolated (point O) and the first intersection point M is represented by r1, and the second distance between the pixel point to be interpolated (point O) and the second intersection point N is represented by r2.
[0320] The values of r1 and r2 are calculated by the following formula (5):
[0321]
[0322] In the above formula (5), r1 represents the first distance between the pixel point to be interpolated (point O) and the first intersection point M, and r2 represents the second distance between the pixel point to be interpolated (point O) and the second intersection point N.
[0323] S20223: Based on the grayscale value of each adjacent pixel point, calculate the grayscale value of the first intersection point and the grayscale value of the second intersection point.
[0324] For example, the grayscale value of the first intersection is determined according to the grayscale value of the first adjacent pixel and the grayscale value of the third adjacent pixel.
[0325] Exemplarily, linear interpolation is performed between the grayscale value of the first adjacent pixel point and the grayscale value of the third adjacent pixel point to obtain the grayscale value of the first intersection point.
[0326] For example, the grayscale value of the first intersection point can be calculated by a first linear interpolation polynomial, and the first linear interpolation polynomial is as follows:
[0327] P M =P A +(P C -PA )q1, (6)
[0328] In the above formula (6), P M represents the gray value of the first intersection point M, P A Represents the gray value of the first adjacent pixel A, P C represents the gray value of the third adjacent pixel point C, and q1 represents the distance between the first intersection point M and the first adjacent pixel point A.
[0329] Wherein, q1 can be calculated by the coordinates of the first intersection point M and the coordinates of the first adjacent pixel point A. For example, q1=qp tanθ.
[0330] The grayscale value of the second intersection is determined according to the grayscale value of the second adjacent pixel and the grayscale value of the fourth adjacent pixel.
[0331] Exemplarily, linear interpolation is performed between the grayscale value of the second adjacent pixel point and the grayscale value of the fourth adjacent pixel point to obtain the grayscale value of the second intersection point.
[0332] For example, the grayscale value of the second intersection point can be calculated by a second linear interpolation polynomial, and the second linear interpolation polynomial is as follows:
[0333] P N =P B +(P D -P B )q2, (7)
[0334] In the above formula (7), P N Represents the gray value of the second intersection point N, P B represents the gray value of the second adjacent pixel B, P D represents the gray value of the fourth adjacent pixel point D, and q2 represents the distance between the second intersection point N and the second adjacent pixel point B.
[0335] Wherein, q2 can be calculated by the coordinates of the second intersection point N and the coordinates of the second adjacent pixel point B. For example, q2=q+(1-p)tanθ.
[0336] S20224: Calculate the grayscale value of the pixel to be interpolated according to the first distance, the second distance, the grayscale value of the first intersection, and the grayscale value of the second intersection.
[0337] Exemplarily, linear interpolation is performed between the grayscale value of the first intersection point and the grayscale value of the second intersection point to obtain the grayscale value of the pixel to be interpolated.
[0338] For example, the grayscale value of the pixel to be interpolated can be calculated by the third linear interpolation polynomial, and the third linear interpolation polynomial is as follows:
[0339] In the above formula (8), P O represents the gray value of the pixel to be interpolated (point O), r2 represents the second distance between the pixel to be interpolated (point O) and the second intersection point N, P M represents the gray value of the first intersection point M, r1 represents the first distance between the pixel point to be interpolated (point O) and the first intersection point M, P N Represents the grayscale value of the second intersection point N.
[0340] In the edge direction of each pixel to be interpolated, interpolation is performed based on the calculated grayscale value of the pixel to be interpolated to obtain a target image.
[0341] In this implementation, the gray value of each pixel to be interpolated is accurately calculated based on the gray value of each adjacent pixel, the edge direction of each pixel to be interpolated, and the edge direction angle of each pixel to be interpolated. Therefore, when interpolation is performed based on the gray value, the continuity of the interpolated image can be higher, the interpolation loss can be smaller, more image details can be retained, and the final target image can be clear, lossless, and of high quality.
[0342] It is worth noting that the initial grayscale value of the pixel to be interpolated (i.e., determined in step S102) is determined by a traditional interpolation algorithm. In order to obtain more accurate gradient directions, edge directions, and grayscale values of the pixel to be interpolated, the grayscale values of the pixel to be interpolated can be continuously iterated and calculated until the edge direction angle θ converges. Then, interpolation is performed based on the grayscale values of the pixel to be interpolated obtained when the edge direction angle θ converges to obtain the target image. Due to continuous iteration, the determined gradient direction, edge direction, and grayscale value of the pixel to be interpolated are more accurate, so when interpolation is performed based on the edge direction and grayscale value after the iteration, the interpolated image can retain more details and be more accurate, further improving the quality of the target image.
[0343] In view of this, the image processing method provided in the embodiment of the present application may further include S401 to S407. It is worth noting that S401 to S407 may be performed after the above step S202.
[0344] See also Fig.19 , Fig.19 is a flow chart of an image processing method provided in an embodiment of the present application. Fig.19 As shown, the image processing method includes S401 to S407.
[0345] S401: For each pixel to be interpolated, determine the grayscale value of the pixel to be interpolated as the initial grayscale value of the pixel to be interpolated.
[0346] Taking one round of execution as an example, the grayscale value of the pixel to be interpolated determined in step S202 is used as the initial grayscale value of the pixel to be interpolated. Based on the current initial grayscale value of the pixel to be interpolated, the process of calculating the grayscale value of the pixel to be interpolated is performed again, thereby calculating a new grayscale value of the pixel to be interpolated.
[0347] S402: Calculate multiple grayscale gradient components of the pixel to be interpolated based on the initial grayscale value and the grayscale values of multiple adjacent pixels.
[0348] Based on the re-determined initial grayscale value of the pixel to be interpolated and the grayscale values of multiple adjacent pixels, multiple grayscale gradient components of the pixel to be interpolated are calculated. The specific calculation process can refer to the description in the above step S301, which will not be repeated here.
[0349] S403: Determine the grayscale gradient of the pixel to be interpolated based on multiple grayscale gradient components.
[0350] Exemplarily, multiple grayscale gradient components of the pixel to be interpolated are vector-synthesized to obtain the grayscale gradient of the pixel to be interpolated. The specific implementation process can refer to the description in the above step S301, which will not be repeated here.
[0351] S404: Determine the edge direction of the pixel to be interpolated according to the grayscale gradient.
[0352] Exemplarily, the grayscale gradient determined in step S403 includes a gradient direction, which is an updated gradient direction. Based on the updated gradient direction, a new edge direction of the pixel to be interpolated is determined. The new edge direction is perpendicular to the updated gradient direction.
[0353] S405: Determine the edge direction angle of the pixel to be interpolated.
[0354] Exemplarily, the gradient direction angle of the pixel to be interpolated is determined, and the edge direction angle of the pixel to be interpolated is determined according to the gradient direction angle of the pixel to be interpolated. The specific implementation process can refer to the description in the above step S2021, which will not be repeated here.
[0355] S406: Calculate the grayscale value of the pixel to be interpolated according to the grayscale value of each adjacent pixel, the edge direction of the pixel to be interpolated, and the edge direction angle of the pixel to be interpolated.
[0356] The specific implementation process can refer to the description in the above step S2022. It is worth noting that the difference between step S406 and step S2022 is that the edge direction of the pixel to be interpolated and the edge direction angle of the pixel to be interpolated in step S406 are determined according to the re-determined initial grayscale value of the pixel to be interpolated, that is, the edge direction of the pixel to be interpolated and the edge direction angle of the pixel to be interpolated in step S406 are updated.
[0357] S407: Repeat the above steps. When the edge direction angle converges, interpolation is performed in the edge direction of each pixel to be interpolated based on the grayscale value of the pixel to be interpolated to obtain a target image, including: in the edge direction of each pixel to be interpolated, interpolation is performed based on the grayscale value of the pixel to be interpolated obtained when the edge direction angle converges to obtain a target image.
[0358] Exemplarily, the edge direction angle convergence refers to the edge direction angle tending to a certain preset angle.
[0359] In a possible implementation, if the edge direction angle has not converged, the above steps S401 to S406 are repeatedly executed, that is, the grayscale value of the pixel to be interpolated calculated in step S406 is continuously determined as the new initial grayscale value of the pixel to be interpolated. The gradient direction, edge direction, and grayscale value of the pixel to be interpolated are updated based on the new initial grayscale value of the pixel to be interpolated that is determined again until the edge direction angle converges.
[0360] In another possible implementation, the edge direction angles converge. For ease of understanding, see Fig. 20 , Fig. 20 It is a schematic diagram of edge direction angle convergence provided in an embodiment of the present application.
[0361] like Fig. 20 As shown, the horizontal axis represents the number of iterations, the vertical axis represents the angle of the edge direction angle, the dotted line represents the preset angle corresponding to the convergence of the edge direction angle, and the curve represents the actual angle of the edge direction angle changing at different stages. It can be seen that with the increase in the number of iterations, the edge direction angle gradually converges.
[0362] When the edge direction angle converges, the grayscale value of the pixel to be interpolated obtained when the edge direction angle converges is used as the final grayscale value, that is, the grayscale value finally used for interpolation. In the edge direction of each pixel to be interpolated, interpolation is performed according to the grayscale value of the pixel to be interpolated obtained when the edge direction angle converges to obtain the target image.
[0363] For example, when the scaling instruction is a zoom-in instruction, up-interpolation is performed in the edge direction of each pixel to be interpolated according to the grayscale value of the pixel to be interpolated obtained when the edge direction angle converges to obtain the target image.
[0364] For another example, when the scaling instruction is a reduction instruction, in the edge direction of each pixel to be interpolated, downward interpolation is performed according to the grayscale value of the pixel to be interpolated obtained when the edge direction angle converges to obtain the target image.
[0365] In this implementation, due to continuous iteration, the determined gradient direction, edge direction and grayscale value of the pixel to be interpolated are more accurate. Therefore, when interpolation is performed based on the edge direction and grayscale value after the iteration, the interpolated image can retain more details and be more accurate, further improving the quality of the target image.
[0366] For easier understanding, see Fig.21 , Fig.21 It is a schematic diagram of a scene in which an image is enlarged using the image processing method provided by the present application.
[0367] like Fig.21 As shown, if an image in the thumbnail is displayed as the original image, it will be displayed as Fig.21 In the image 301 shown in (a), when the user wants to zoom in on a partial image 302 in the image 301, the user can simultaneously touch the area where the partial image 302 is located with two fingers and slide outwards to zoom in on the partial image 302. Fig.21 As shown in (b), image 303 is the image after the partial image 302 is enlarged.
[0368] Since the present application takes into account the edge characteristics of the image, the image processing method provided by the present application, in the process of processing the image, combines the edge characteristics of the image, can make the target image obtained after processing reflect the real contour information of the image, improve the quality of the processed image, and ensure that the processed image is clear and lossless. Compared with the related art, it effectively avoids the stripe (or aliasing) phenomenon in the processed image.
[0369] The above describes in detail an example of an image processing method provided by an embodiment of the present application. It is understandable that, in order to implement the above functions, the electronic device includes hardware and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in combination with the embodiments, but such implementation should not be considered to exceed the scope of the present application.
[0370] The embodiment of the present application can divide the functional modules of the electronic device according to the above method example. For example, each function can be divided into various functional modules, such as a first determination unit, a second determination unit, and an interpolation unit, etc., or two or more functions can be integrated into one module. The above integrated module can be implemented in the form of hardware or in the form of a software functional module. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.
[0371] It should be noted that all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module and will not be repeated here.
[0372] The electronic device provided in this embodiment is used to execute the above-mentioned image processing method, and thus can achieve the same effect as the above-mentioned implementation method.
[0373] In the case of an integrated unit, the electronic device may further include a processing module, a storage module and a communication module. The processing module may be used to control and manage the actions of the electronic device. The storage module may be used to support the electronic device to execute stored program codes and data, etc. The communication module may be used to support the electronic device to communicate with other devices.
[0374] Among them, the processing module can be a processor or a controller. It can implement or execute various exemplary logic boxes, modules and circuits described in conjunction with the disclosure of this application. The processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, etc. The storage module can be a memory. The communication module can specifically be a device that interacts with other electronic devices, such as a radio frequency circuit, a Bluetooth chip, a WiFi chip, etc.
[0375] In one embodiment, when the processing module is a processor and the storage module is a memory, the electronic device involved in this embodiment may be a Figure 3 Device of the structure shown.
[0376] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the processor executes the image processing method of any of the above embodiments.
[0377] The embodiment of the present application also provides a computer program product. When the computer program product is run on a computer, the computer is enabled to execute the above-mentioned related steps to implement the image processing method in the above-mentioned embodiment.
[0378] The present application also provides a chip. Fig. 22 , Fig. 22 It is a schematic diagram of the structure of a chip provided in an embodiment of the present application. Fig. 22 The chip shown can be a general-purpose processor or a dedicated processor. The chip includes a processor 410. The processor 410 is used to execute the image processing method of any of the above embodiments.
[0379] Optionally, the chip further includes a transceiver 420, which is used to accept control of the processor and to support the communication device in executing the technical solution shown above.
[0380] Optional, Fig. 22 The chip shown may also include: a storage medium 430 .
[0381] It should be noted that Fig. 22 The chip shown can be implemented using the following circuits or devices: one or more field programmable gate arrays (FPGAs), programmable logic devices (PLDs), controllers, state machines, gate logic, discrete hardware components, any other suitable circuits, or any combination of circuits that can perform the various functions described throughout this application.
[0382] Among them, the electronic device, computer-readable storage medium, computer program product or chip provided in this embodiment are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above and will not be repeated here.
[0383] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.
[0384] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0385] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.
[0386] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0387] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.
[0388] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. An image processing method, characterized in that: include: Determine the edge direction of each pixel to be interpolated according to the gradient direction of each pixel to be interpolated corresponding to the original image, wherein the edge direction is perpendicular to the gradient direction; Determine the grayscale value of each pixel to be interpolated; the determining the grayscale value of each pixel to be interpolated includes: determining the edge direction angle of each pixel to be interpolated; for each pixel to be interpolated, determine a first intersection point and a second intersection point according to the edge direction of the pixel to be interpolated and a plurality of adjacent pixels; Determine a first distance and a second distance according to the edge direction angle of the pixel to be interpolated; calculate the grayscale values of the first intersection and the second intersection based on the grayscale value of each adjacent pixel; calculate the grayscale value of the pixel to be interpolated according to the first distance, the second distance, the grayscale value of the first intersection, and the grayscale value of the second intersection; In the edge direction of each pixel to be interpolated, interpolation is performed based on the grayscale value of the pixel to be interpolated to obtain a target image corresponding to the original image.
2. The image processing method according to claim 1, characterized in that: The method further comprises: Determine the gradient direction of each pixel to be interpolated corresponding to the original image.
3. The image processing method according to claim 2, characterized in that: The step of determining the gradient direction of each pixel to be interpolated corresponding to the original image includes: Determine a grayscale gradient component of each pixel to be interpolated, where the grayscale gradient component indicates a grayscale gradient of the pixel to be interpolated in a first direction, where the first direction is a direction from a pixel adjacent to the pixel to be interpolated to the pixel to be interpolated; Vector synthesis is performed on multiple grayscale gradient components of each pixel to be interpolated to obtain a grayscale gradient of each pixel to be interpolated, wherein the grayscale gradient includes a gradient direction.
4. The image processing method according to claim 3, characterized in that: The step of determining the grayscale gradient component of each pixel to be interpolated includes: For each pixel to be interpolated, determine a plurality of adjacent pixels in the original image that are adjacent to the pixel to be interpolated, and a grayscale value of each adjacent pixel; Based on the initial grayscale value of the pixel to be interpolated and the grayscale values of multiple adjacent pixels, multiple grayscale gradient components of the pixel to be interpolated are calculated.
5. The image processing method according to claim 1, wherein: The first intersection point, the second intersection point and the pixel point to be interpolated are on the same line.
6. The image processing method according to claim 1, characterized in that: The first distance represents the distance between the pixel point to be interpolated and the first intersection point, and the second distance represents the distance between the pixel point to be interpolated and the second intersection point.
7. The image processing method according to claim 1, characterized in that: The step of determining the edge direction angle of each pixel to be interpolated comprises: Determine the gradient direction angle of each pixel to be interpolated, where the gradient direction angle is the angle between the pixel to be interpolated in the horizontal direction and the gradient direction; The edge direction angle of each pixel to be interpolated is determined according to the gradient direction angle of each pixel to be interpolated.
8. The image processing method according to any one of claims 1 to 7, characterized in that: The method further comprises: For each pixel to be interpolated, determining the grayscale value of the pixel to be interpolated as the initial grayscale value of the pixel to be interpolated; Calculating multiple grayscale gradient components of the pixel to be interpolated based on the initial grayscale value and the grayscale values of multiple adjacent pixels; Based on the multiple grayscale gradient components, determining the grayscale gradient of the pixel to be interpolated; Determining the edge direction of the pixel to be interpolated according to the grayscale gradient; Determine the edge direction angle of the pixel to be interpolated; Calculate the grayscale value of the pixel to be interpolated according to the grayscale value of each adjacent pixel, the edge direction of the pixel to be interpolated, and the edge direction angle of the pixel to be interpolated; Repeating the above steps, when the edge direction angle converges, interpolating based on the grayscale value of each pixel to be interpolated in the edge direction of the pixel to be interpolated, to obtain a target image corresponding to the original image, including: In the edge direction of each pixel to be interpolated, interpolation is performed according to the grayscale value of the pixel to be interpolated obtained when the edge direction angle converges to obtain the target image.
9. The image processing method according to any one of claims 1 to 7, characterized in that: Before determining the edge direction of each pixel to be interpolated according to the gradient direction of each pixel to be interpolated corresponding to the original image, the method further includes: Detecting operations for zooming; In response to the operation, determining a scaling ratio of the original image; According to the scaling ratio and the interpolation algorithm, each pixel point to be interpolated corresponding to the original image is determined.
10. An electronic device, characterized in that: include: one or more processors; one or more memories; The memory stores one or more programs, and when the one or more programs are executed by the processor, the electronic device executes the method according to any one of claims 1 to 9.
11. A chip, characterized in that: include: A processor, used to call and run a computer program from a memory, so that an electronic device equipped with the chip executes a method as claimed in any one of claims 1 to 9.
12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor is caused to execute the method according to any one of claims 1 to 9.
