Image processing method and related apparatus

Abstract

The application discloses an image processing method and related device, the method comprises the following steps: carrying out scaling processing on a first image of a first resolution to obtain an intermediate image of a second resolution, the second resolution is smaller than the first resolution; based on sharpening intensity, the intermediate image is sharpened to obtain a target image of the second resolution; the sharpening intensity is determined according to the first scaling ratio of the target image relative to the first image, and the first scaling ratio is determined according to the second resolution and the first resolution. In this way, the image scaling effect can be effectively improved for various scaling ratios, the definition of the scaled image is effectively improved, and the user experience is improved.

Description

Technical Field

[0001] This application relates to the field of electronic technology, and in particular to image processing methods and related apparatus. Background Technology

[0002] Image scaling refers to enlarging or reducing an image by increasing or decreasing the number of pixels. Image scaling algorithms, as fundamental image processing algorithms, are widely used in terminal devices. For example, in screen mirroring scenarios, a high-resolution main screen can be projected onto a low-resolution secondary screen; in icon scaling scenarios, an application icon needs to be adaptively scaled down to the required display size for different screen sizes on electronic devices such as PCs, mobile phones, and watches; in gallery image scaling scenarios, original images need to be reduced to thumbnails for quick viewing; and in wallpaper scaling scenarios, wallpapers from a shared theme library across different devices are adapted to the different screen sizes of various electronic devices. The image scaling effect directly impacts the user experience. Currently, scaled images are not clear enough, and the image scaling effect needs further improvement. Summary of the Invention

[0003] This application provides an image processing method and related apparatus that can effectively improve the image scaling effect for various scaling ratios, effectively improve the clarity of the image after scaling, and enhance the user experience.

[0004] In a first aspect, this application provides an image processing method applied to an electronic device, the method comprising: scaling a first image at a first resolution to obtain an intermediate image at a second resolution, the second resolution being smaller than the first resolution; sharpening the intermediate image based on a sharpening intensity to obtain a target image at the second resolution; the sharpening intensity being determined based on a first scaling ratio of the target image relative to the first image, the first scaling ratio being determined based on the second resolution and the first resolution.

[0005] In implementing the embodiments of this application, the original image is first scaled using an image scaling algorithm to obtain an intermediate image with the same size as the target image; then, the intermediate image is sharpened based on the sharpening intensity to obtain the target image. The sharpening intensity is dynamically adjusted according to the scaling ratio; thus, for different scaling ratios, while achieving image scaling, the clarity of the scaled target image can be effectively improved, enhancing the user experience.

[0006] In one implementation, when the first scaling factor decreases, the sharpening intensity does not decrease; that is, it remains unchanged or increases.

[0007] In one implementation, when the first scaling ratio is a first scale, the sharpening intensity is a first value; when the first scaling ratio is a second scale, the sharpening intensity is a second value; the second scale is smaller than the first scale, and the second value is greater than the first value. Implementing the embodiments of this application, the smaller the first scaling ratio, the more high-frequency information is lost in the scaled target image, resulting in a more blurred image. Therefore, when the first scaling ratio decreases, increasing the sharpening intensity can adaptively improve the clarity of the sharpened target image; thus, the clarity of the target image at each scaling ratio is guaranteed.

[0008] In one implementation, when the first scaling ratio is the third scaling ratio, the sharpening intensity is the first value; the third scaling ratio is less than the first scaling ratio. By implementing the embodiments of this application, multiple scaling ratios within a preset range correspond to a single sharpening intensity, and visually, the target images corresponding to these multiple scaling ratios have similar clarity. Thus, associating multiple scaling ratios within a preset range with a single sharpening intensity also facilitates setting and managing the correspondence between scaling ratios and sharpening intensities.

[0009] In one implementation, the first scaling factor is determined based on a horizontal scaling factor and a vertical scaling factor; the horizontal scaling factor is equal to the width of the second resolution divided by the width of the first resolution, and the vertical scaling factor is equal to the height of the second resolution divided by the height of the first resolution. In another implementation, the first scaling factor is one of the following: the horizontal scaling factor, the vertical scaling factor, or the average of the horizontal and vertical scaling factors.

[0010] In one implementation, scaling a first image at a first resolution to obtain an intermediate image at a second resolution includes: scaling the first image at the first resolution based on the kernel radius and the kernel weight matrix to obtain the intermediate image at the second resolution. The kernel radius and / or weight matrix are determined based on the horizontal scaling ratio and the vertical scaling ratio. By implementing the embodiments of this application, the kernel radius and / or weight matrix can be dynamically adjusted according to the scaling ratio and the vertical scaling ratio; thus, the image scaling effect can be effectively improved for various scaling ratios.

[0011] In one implementation, the width of the convolution kernel radius increases as the horizontal scaling ratio decreases, while the height of the convolution kernel radius increases as the vertical scaling ratio decreases. This increase in the convolution kernel radius as the scaling ratio decreases can reduce frequency domain aliasing during image scaling, i.e., reduce the jaggedness of the scaled intermediate image, making the edges of graphics in both the intermediate and target images smoother, thus effectively improving the image scaling effect.

[0012] In one implementation, the width of the convolution kernel radius is proportional to the inverse of the horizontal scaling ratio, and the height of the convolution kernel radius is proportional to the inverse of the vertical scaling ratio. This reduces the jagged edges of the scaled intermediate image, making the edges of graphics in both the intermediate and target images smoother and effectively improving the image scaling effect.

[0013] In one implementation, the low-pass bandwidth corresponding to the weight matrix of the convolution kernel decreases as the first scaling ratio decreases. Implementing this embodiment, the weight matrix of the convolution kernel is adjusted as the scaling ratio decreases, thereby reducing the low-pass bandwidth corresponding to the weight matrix. This can alleviate frequency domain aliasing during image scaling, i.e., reduce the jaggedness of the scaled intermediate image, making the edges of graphics in the intermediate and target images smoother, effectively improving the image scaling effect.

[0014] In one implementation, the weight matrix of the convolution kernel is associated with the bandwidth adjustment parameter beta; the low-pass bandwidth corresponding to the weight matrix of the convolution kernel decreases as beta increases. Thus, decreasing the bandwidth adjustment parameter beta of the weight matrix as the scaling ratio decreases can mitigate frequency domain aliasing during convolution operations during image scaling, i.e., reduce the jaggedness of the scaled intermediate image, making the edges of graphics in the intermediate and target images smoother, effectively improving the image scaling effect.

[0015] In one implementation, the first resolution is sw*sh, and the second resolution is dw*dh. Based on the kernel radius and the kernel weight matrix, the first image at the first resolution is scaled to obtain an intermediate image at the second resolution. This includes determining the pixel P of the intermediate image. ij 'The convolution center C in the original image ij coordinates (C) i C j ), P ij ' represents the pixel in the i-th row and j-th column of the intermediate image, where i ranges from [0, dh-1] and j ranges from [0, dw-1]; based on the convolution center C ij And the kernel radius, determine the convolution center C ij The pixel matrix S corresponding to the first convolution window ij Based on the scaling ratio and bandwidth adjustment parameter beta, determine the weight matrix K corresponding to the first convolution window. ij For pixel matrix S ij With weight matrix K ij Perform convolution operation to obtain the pixel P of the intermediate image. ij The pixel value of '. Implementing the embodiments of this application, the weight matrix K corresponding to the first convolutional window can be dynamically adjusted based on the scaling ratio and the bandwidth adjustment parameter beta. ij；This effectively improves image scaling for various scaling ratios.

[0016] In one implementation, pixel S mn Let m be the pixel in the m-th row and n-th column within the first convolution window, where m can take values ​​ranging from 1 to 2. The range of values ​​for n is symbol This indicates rounding down by *. Gw is the width of the convolution kernel radius, Gh is the height of the convolution kernel radius, rw is the horizontal scaling ratio, and rh is the vertical scaling ratio. Based on the scaling ratio and the bandwidth adjustment parameter beta, the weight matrix K corresponding to the first convolution window is determined. ij This includes: determining the pixel S within the first convolutional window. mn The corresponding horizontal weight L n L n It is based on X n Definitely, X n =rw*(n–C j / beta; Determines the pixel S within the first convolution window. mn The corresponding vertical weight L m L m It is based on X m Definitely, X m =rh*(m–C i / beta; Determines the pixel S within the first convolution window. mn The corresponding weight K mn =L n *L m Implementing embodiments of this application can be based on the horizontal scaling ratio rw and the horizontal distance between the pixel and the convolution center (i.e., (n–C)). j The horizontal weight L is dynamically adjusted using the bandwidth adjustment parameter beta. n Based on the vertical scaling ratio rw and the vertical distance between the pixel and the convolution center (i.e., (m–C) i The vertical weight L is dynamically adjusted using the bandwidth adjustment parameter beta. m Thus, as the scaling factor decreases, the dynamically adjusted weight matrix K... ij The corresponding reduction in low-pass bandwidth can reduce the jaggedness of the scaled intermediate image, making the edges of the graphics in the intermediate and target images smoother and effectively improving the image scaling effect.

[0017] In one implementation, L n Equal to the number of pixels S within the first convolution window mn L(X) n The function is the normalized value of L(*) relative to the L(*) function of all pixels in the same pixel row; where L(*) is the Lanczos kernel function, L(X) is the normalized value of L(X) relative to the L(*) function of all pixels in the same pixel row.n ) = sinc(X n )*sinc(X n / Gh);L m Equal to the number of pixels S within the first convolution window mn L(X) m The function is the normalized value of the L(*) function relative to all pixels in its column; where L(X) is the normalized value of the L(*) function. m ) = sinc(X m )*sinc(X m / Gw). By implementing the embodiments of this application, the horizontal weights L can be dynamically adjusted based on the convolution radius. n and vertical weight L m Thus, as the scaling factor decreases, the dynamically adjusted weight matrix K... ij The corresponding reduction in low-pass bandwidth can reduce the jaggedness of the scaled intermediate image, making the edges of the graphics in the intermediate and target images smoother and effectively improving the image scaling effect.

[0018] In one implementation, the method further includes: if the electronic device has obtained a preset value for the first parameter, then determining that the first parameter is a preset value; if the electronic device has not obtained a preset value for the first parameter, then determining the first parameter based on a horizontal scaling ratio and / or a vertical scaling ratio; the first parameter is used to generate a target image corresponding to the first image. In different usage scenarios, the aforementioned preset value for the same parameter can be different. In this way, the electronic device, the user, or the developer can freely specify the aforementioned preset value, and the electronic device can also flexibly adjust the value of the first parameter based on whether a preset value for the first parameter has been set.

[0019] In one implementation, the method further includes: the electronic device acquiring a first parameter of the first image from another device, the first parameter being determined based on a horizontal scaling ratio and / or a vertical scaling ratio; the first parameter being used to generate a target image corresponding to the first image. Implementing embodiments of this application, the first parameter can also be pre-calculated by other devices based on the scaling ratio, saving the load and energy consumption of the electronic device in image scaling and reducing the latency of image scaling.

[0020] In one implementation, the first parameter is the sharpening intensity.

[0021] In one implementation, the first parameter is any one of the following: kernel radius, kernel weight matrix, and bandwidth adjustment parameter beta.

[0022] Secondly, embodiments of this application provide an electronic device, which includes a processor and a memory, the memory being coupled to the processor, the memory being used to store computer program code, the computer program code including computer instructions, and the processor reading the computer instructions from the memory to cause the electronic device to execute the image processing method described in the first aspect.

[0023] Thirdly, embodiments of this application provide a computer storage medium including computer instructions, which, when executed on an electronic device, cause the electronic device to perform the image processing method in any of the possible implementations of any of the above aspects.

[0024] Fourthly, embodiments of this application provide a computer program product that, when run on a computer, causes the computer to execute the image processing method in any of the possible implementations of any of the above aspects. Attached Figure Description

[0025] Figure 1 A pixel diagram of an original image and a target image provided for an embodiment of this application;

[0026] Figure 2 A schematic diagram illustrating the existing image scaling effect in the screen projection scenario provided in the embodiments of this application;

[0027] Figure 3 A schematic diagram illustrating the existing image scaling effect in the icon scaling scenario provided in the embodiments of this application;

[0028] Figure 4A and Figure 4B This is a schematic diagram illustrating another image scaling effect in a screen projection scenario provided in an embodiment of this application.

[0029] Figure 5A and Figure 5B This is a schematic diagram illustrating another image scaling effect in an icon scaling scenario provided in an embodiment of this application.

[0030] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0031] Figures 7A to 7C A schematic diagram of the software architecture provided for embodiments of this application;

[0032] Figure 8 A schematic flowchart of the image processing method provided in the embodiments of this application;

[0033] Figure 9 A schematic diagram illustrating the variation of serration intensity with convolution kernel radius, provided for embodiments of this application;

[0034] Figure 10 A schematic diagram illustrating the process of determining the weight matrix provided in an embodiment of this application;

[0035] Figure 11A and Figure 11B A schematic diagram of two types of convolutional windows provided in the embodiments of this application;

[0036] Figure 12 This is a schematic flowchart of the image processing method provided in an embodiment of this application. Detailed Implementation

[0037] The technical solutions in the embodiments of this application will be clearly and thoroughly described below with reference to the accompanying drawings. In the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B; the word "and / or" in the text is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, in the description of the embodiments of this application, "multiple" refers to two or more than two.

[0038] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0039] The term "user interface (UI)" used in the following embodiments of this application refers to the medium interface through which an application or operating system interacts and exchanges information with the user. It realizes the conversion between the internal form of information and the form that the user can accept. The user interface is source code written in a specific computer language such as Java or Extensible Markup Language (XML). The interface source code is parsed and rendered on the electronic device, ultimately presenting content that the user can recognize. A common form of user interface is the graphical user interface (GUI), which refers to a user interface related to computer operation displayed graphically. It can be visible interface elements such as text, icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, and widgets displayed on the screen of an electronic device.

[0040] Image scaling refers to obtaining a target image by increasing or decreasing the number of pixels in an original image. Common image scaling algorithms are usually based on convolution operations, such as bilinear interpolation, bicubic interpolation, Lanczos resampling, and Cubic Spline.

[0041] Image scaling algorithms based on convolution operations include: (1) using the width sw and height sh of the original image, and the width dw and height dh of the scaled target image, to calculate the horizontal scaling ratio and vertical scaling ratio of the 2D image respectively; where the horizontal scaling ratio rw = dw / sw, and the vertical scaling ratio rh = dh / sh. (2) determining the 2D coordinates C of the convolution center according to the scaling ratio. ij ; where the scaling ratio includes the horizontal scaling ratio and the vertical scaling ratio, i and j are positive integers, the value range of i is [0, dh-1], the value range of j is [0, dw-1], and the number of convolution centers is consistent with the number of pixels in the target image. (3) In the original image, with the convolution center C ij C is extracted based on the geometric center and the kernel radius. ij The surrounding pixels form a convolution window; using convolution operations, the pixel in the i-th row and j-th column of the target image (i.e., pixel P) is generated based on this convolution window. ij The convolution kernel radius can be represented as Gw*Gh, the width of the convolution window is twice Gw, and the height of the convolution window is twice Gh; the size of the convolution kernel can be represented as the width * height of the convolution window, i.e., 2Gw*2Gh.

[0042] For example, Figure 1 An exemplary pixel diagram of the original and target images is shown, where each grid represents a pixel. The pixel in the i-th row and j-th column of the target image can be represented as P. ij ;like Figure 1 As shown, the width and height of the convolution window in the original image are both 2. The electronic device 100 extracts the convolution center C. ij The corresponding 2x2 convolution window contains a matrix of pixels [p1, p2; p3, p4]; the target image has pixel P. ij The pixel value is equal to the sum of the weights of all pixels within the convolution window, i.e., P. ij =sum([p1,p2;p3,p4]*[k1,k2;k3,k4])=p1*k1+p2*k2+p3*k3+p4*k4, where the weight matrix [k1,k2;k3,k4] is the convolution kernel of the aforementioned convolution window. The multiplication (i.e., *) between the matrix formed by the pixels within the aforementioned convolution window and the convolution kernel is the Hadamard product, i.e., element-wise multiplication. The differences between different image scaling algorithms usually lie in the different calculation methods of the convolution kernel radius and the weights in the convolution kernel.

[0043] For example, taking bilinear interpolation as an image scaling algorithm, the convolution kernel size of bilinear interpolation is fixed at 2*2, that is, the convolution kernel radius is fixed at 1*1. If pixel P in the target image... ij The corresponding convolution center C ij The 2D coordinates of the convolution kernel are (x, y), and the coordinates of the pixels p1, p2, p3, and p4 within the convolution window are (x1, y1), (x1, y2), (x2, y1), and (x2, y2), respectively. Therefore, the weight matrix [k1, k2; k3, k4] corresponding to the convolution kernel is [(x2-x)*(y2-y), (x2-x)*(y-y1), (x-x1)*(y2-y), (x-x1)*(y-y1)]. This shows that in the bilinear interpolation algorithm, the weight matrix is ​​determined by the distance between the position of each pixel within the convolution window and the center of the convolution, and is independent of the horizontal and vertical scaling ratios. This is also true for other image scaling algorithms (such as bicubic interpolation, Lanczos resampling, and Cubic Spline).

[0044] Current image scaling algorithms typically suffer from the following four problems: Problem 1: When scaling down from the original image to the target image, the algorithm extracts low-frequency information from the original image using convolution operations to generate pixels in the target image. However, the high-frequency information lost during convolution operations often includes detailed information from the original image, such as edge information (e.g., black-and-white boundaries). This leads to a decrease in the sharpness of the scaled-down image, and the sharpness generally decreases as the scaling ratio decreases. Problem 2: The size of the convolution kernel in image scaling algorithms is usually fixed. For example, the kernel size in bilinear interpolation is 2*2. The weights in the kernel are determined only by the distance between the pixel and the center of the convolution. The size and weights of the kernel are independent of the scaling ratio. This means that the filtering bandwidth corresponding to the kernel does not change with the scaling ratio, making the scaled-down image prone to aliasing in the frequency domain, i.e., jagged edges. The jagged edges are usually more pronounced as the scaling ratio decreases. Problem 3: The size and weights of the convolution kernel are generally tied to the specific algorithm and cannot be freely specified, lacking flexibility. Question 4: The complexity of scaling algorithms comes from the size of the convolution kernel and the complexity of calculating the weights; the larger the size of the convolution kernel, the more complex the weight calculation formula, and the higher the algorithm load and time consumption; when electronic devices scale images, the weights of the convolution kernel are calculated in real time, which brings additional overhead and image scaling latency to the electronic devices.

[0045] The following examples, using screen mirroring and icon scaling scenarios, illustrate the image scaling effects of existing image scaling algorithms.

[0046] Taking screen mirroring as an example, electronic device 100 (e.g., computer) can mirror the display of a high-resolution screen 1 (e.g., user interface 11) to a low-resolution screen 2. Figure 2 This diagram illustrates the effect of using existing image scaling algorithms in a screen projection scenario. Screen 1 can be the main screen of electronic device 100, and screen 2 can be a separate extended screen or the display screen of another electronic device (e.g., a mobile phone, tablet, television, etc.). For example, Figure 2 The diagram shows the original image of user interface 11 displayed on screen 1 at resolution 1 of a computer. After the computer projects user interface 11 onto screen 2 at resolution 2, screen 2 displays the target image 1 of user interface 11; alternatively, after the computer projects user interface 11 onto screen 2 at resolution 3, screen 2 displays the target image 2 of user interface 11. Here, resolution 1 (e.g., 4K) is greater than resolution 2 (e.g., 2K), and resolution 2 is greater than resolution 3 (e.g., 1K). This is for ease of observation and comparison. Figure 2 The same icon (e.g., a contact icon) for the user interface 11 in the original image, target image 1, and target image 2 are also shown respectively. In this embodiment, the device type and resolution of the source device (e.g., a computer) and target device (e.g., a television) in the screen projection scenario are not specifically limited; the physical size of the screen 1 and screen 2 is not specifically limited, and the physical size of screen 2 can be equal to, less than, or greater than the physical size of screen 1.

[0047] Taking icon scaling as an example, for electronic devices with different resolutions such as PCs, mobile phones, and watches, it is necessary to shrink the original image of the same application icon (such as a contact icon) to a target image that fits the screen size of the device. Figure 3 This diagram illustrates the effect of using existing image scaling algorithms in an icon scaling scenario. It's understandable that the target image resolution of the scaled application icon will differ for electronic devices with different resolutions. For example, Figure 3 The original image of the contact icon at resolution 4 is shown, as well as the target image 3 of the contact icon displayed on a mobile phone at resolution 5 and the target image 4 of the contact icon displayed on a watch at resolution 6. Resolution 4 is greater than resolution 5, and resolution 5 is greater than resolution 6. In this embodiment, the image resolution can also be referred to as the image size.

[0048] like Figure 2 and Figure 3 As shown, the smaller the resolution of the scaled target image, that is, the smaller the scaling ratio, the blurrier the target image, and the more severe the jagged edges, and even undesirable visual effects such as black borders appear.

[0049] This application provides an image processing method that cascades an image scaling algorithm (e.g., bilinear interpolation) and an image sharpening algorithm (e.g., desharpening masking). First, the image scaling algorithm is used to reduce the size of the original image, obtaining an intermediate image of the same size as the target image. Then, the image sharpening algorithm is used to sharpen the intermediate image to improve its clarity, resulting in the target image. This method improves the clarity of the target image compared to existing image scaling algorithms.

[0050] However, while the image processing methods described above can improve upon problem 1, existing image sharpening algorithms typically only sharpen intermediate images according to a preset sharpening intensity alpha. This sharpening intensity alpha is unrelated to the scaling ratio and cannot perform adaptive sharpening for target images with different scaling ratios. Currently, intermediate images scaled by image scaling algorithms exhibit jagged edges, which are exacerbated by conventional sharpening processes. Furthermore, the image processing methods described above cannot solve problems 2, 3, and 4.

[0051] This application also provides an image processing method in which an improved image scaling algorithm 1 and an improved image sharpening algorithm 1 are cascaded. First, the image scaling algorithm 1 is used to reduce the size of the original image to obtain an intermediate image of the same size as the target image. Then, the image sharpening algorithm 2 is used to sharpen the intermediate image to obtain the target image. The improved image sharpening algorithm 1 can dynamically adjust the sharpening intensity according to the scaling ratio; thus, it can effectively improve the clarity of the target image for different scaling ratios, solving the aforementioned problem 1.

[0052] In some embodiments, the improved image scaling algorithm 1 can adaptively adjust the convolution kernel radius and weights according to the scaling ratio. In one implementation, the convolution kernel radius can increase as the scaling ratio decreases. Given a convolution kernel radius, the weights in the convolution kernel can fluctuate less as the scaling ratio decreases. Furthermore, the weights in the convolution kernel are adjusted by the bandwidth adjustment parameter beta, and the weights in the convolution kernel can fluctuate less as beta increases. The reduced fluctuation of the weights in the convolution kernel can decrease the bandwidth of the low-pass filter in the convolution operation, thereby mitigating the jagged edges. Thus, for different scaling ratios, the convolution operation for image scaling can be adaptively performed, mitigating the frequency domain aliasing phenomenon that increases as the scaling ratio decreases, thereby effectively reducing the jagged edges of graphics in the intermediate image, making the graphics edges smoother, and solving the aforementioned problem 2.

[0053] In some embodiments, if the electronic device 100 obtains a preset value for parameter 1, the electronic device 100 can determine that the value of parameter 1 is the aforementioned preset value, and then perform image scaling and sharpening on the original image according to the preset value to obtain the target image corresponding to the original image; if the electronic device 100 does not obtain a preset value for parameter 1, it dynamically adjusts parameter 1 according to the scaling ratio, and then performs image scaling and sharpening image processing methods on the original image according to the dynamically adjusted parameter 1 to obtain the target image corresponding to the original image. Here, parameter 1 can be any one of the convolution kernel radius Gw*Gh, the weight matrix of the convolution kernel, and the sharpening intensity alpha. In different usage scenarios, the aforementioned preset value of the same parameter can be different; the aforementioned preset value of parameter 1 can be specified by the electronic device, by the user, or by the developer of the application using the image processing method provided in this application. This embodiment of the application does not specifically limit the method of obtaining the preset values ​​of each parameter. In this way, the electronic device 100, the user, or the developer can freely specify the aforementioned preset values, and the electronic device 100 can also flexibly adjust the values ​​of each parameter according to whether or not the preset values ​​of each parameter are obtained, thus solving the aforementioned problem 3.

[0054] In some embodiments, before the electronic device 100 performs image scaling, other devices or the electronic device 100 can pre-calculate parameter 1 based on the scaling ratio. This way, the electronic device 100 does not need to calculate parameter 1 in real time during image scaling, reducing the latency of image scaling. In one implementation, other devices pre-calculate parameter 1 based on the scaling ratio, and the electronic device 100 also obtains the corresponding parameter 1 of the original image when acquiring the original image. This saves the load and energy consumption of the electronic device 100 during image scaling. In summary, the image processing method provided by the embodiments of this application can solve the above-mentioned problem 4.

[0055] For example, comparison Figure 2 , Figure 4A A schematic diagram illustrating the image scaling effect of the image processing method provided in this application in a screen projection scenario is shown. Figure 4B This diagram illustrates a comparison of the effects of existing image scaling algorithms and the image processing method provided in this application in this scenario. For example, the comparison... Figure 3 , Figure 5A This diagram illustrates the effect of using the image processing method provided in this application in an icon scaling scenario. Figure 4B A schematic diagram comparing the effects of existing image scaling algorithms and the image processing method provided in this application is shown in this scenario.

[0056] like Figure 4A , Figure 4B , Figure 5A and Figure 5BIt is understood that implementing the image processing method provided in the embodiments of this application can significantly improve the clarity of the target image and reduce the jaggedness of the target image, thereby effectively improving the visual effect of image scaling.

[0057] The image processing method provided in this application is applicable to any scenario involving image scaling, including but not limited to: screen mirroring scenarios, application icon scaling, image scaling scenarios in the gallery, wallpaper scaling scenarios, and image scaling in third-party applications.

[0058] The specific implementation of the image processing method provided in the embodiments of this application will be described in detail below.

[0059] First, the electronic device 100 to which the image processing method provided in this application embodiment is applied is introduced. The electronic device 100 can be a mobile phone, tablet computer, desktop computer, laptop computer, handheld computer, notebook computer, ultra-mobile personal computer (UMPC), netbook, or a cellular phone, personal digital assistant (PDA), augmented reality (AR) device, virtual reality (VR) device, artificial intelligence (AI) device, wearable device (e.g., smart bracelet), in-vehicle device, smart home device (e.g., smart TV, smart screen, large-screen device, etc.), and / or smart city device. This application embodiment does not impose any special limitations on the specific type of electronic device 100.

[0060] For example, Figure 6 A schematic diagram of an electronic device 100 is shown.

[0061] Electronic device 100 may include processor 110, external memory interface 120, internal memory 121, universal serial bus (USB) interface 130, charging management module 140, power management module 141, battery 142, antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, sensor module 180, button 190, motor 191, indicator 192, camera 193, display screen 194, and subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0062] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0063] Processor 110 may include one or more processing units, such as application processors (APs), modem processors, graphics processing units (GPUs), image signal processors (ISPs), controllers, video codecs, digital signal processors (DSPs), baseband processors, and / or neural network processing units (NPUs). These different processing units may be independent devices or integrated into one or more processors.

[0064] The controller can generate operation control signals based on the instruction opcode and timing signals to complete the control of instruction fetching and execution.

[0065] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0066] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0067] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple I2C buses. The processor 110 can couple to the touch sensor 180K, charger, flash, camera 193, etc., through different I2C bus interfaces. For example, the processor 110 can couple to the touch sensor 180K through the I2C interface, enabling the processor 110 and the touch sensor 180K to communicate through the I2C bus interface, thereby realizing the touch function of the electronic device 100.

[0068] The I2S interface can be used for audio communication. In some embodiments, the processor 110 may include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to enable communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the I2S interface to enable the function of answering phone calls through a Bluetooth headset.

[0069] The PCM interface can also be used for audio communication, sampling, quantizing, and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled via the PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface, enabling the function of answering phone calls through a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0070] The UART interface is a universal serial data bus used for asynchronous communication. This bus can be a bidirectional communication bus. It converts the data to be transmitted between serial and parallel communication. In some embodiments, the UART interface is typically used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 via the UART interface to implement Bluetooth functionality. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the UART interface to enable music playback through Bluetooth headphones.

[0071] The MIPI interface can be used to connect the processor 110 to peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI) and a display serial interface (DSI). In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to enable the electronic device 100 to capture images. The processor 110 and the display screen 194 communicate via the DSI interface to enable the electronic device 100 to display images.

[0072] The GPIO interface can be configured via software. It can be configured as a control signal or a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 to a camera 193, a display screen 194, a wireless communication module 160, an audio module 170, a sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0073] USB port 130 is a USB standard compliant interface, specifically a Mini USB port, Micro USB port, USB Type-C port, etc. USB port 130 can be used to connect a charger to charge electronic device 100, and can also be used for data transfer between electronic device 100 and peripheral devices. It can also be used to connect headphones for audio playback. This interface can also be used to connect other electronic devices, such as AR devices.

[0074] It is understood that the interface connection relationships between the modules illustrated in the embodiments of this application are merely illustrative and do not constitute a structural limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0075] The charging management module 140 receives charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 receives charging input from the wired charger via the USB interface 130. In some wireless charging embodiments, the charging management module 140 receives wireless charging input via the wireless charging coil of the electronic device 100. While charging the battery 142, the charging management module 140 can also supply power to the electronic device via the power management module 141.

[0076] The power management module 141 connects the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, providing power to the processor 110, internal memory 121, display screen 194, camera 193, and wireless communication module 160, etc. The power management module 141 can also monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage current, impedance). In some other embodiments, the power management module 141 may also be located within the processor 110. In other embodiments, the power management module 141 and the charging management module 140 may be located in the same device.

[0077] The wireless communication function of electronic device 100 can be realized through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor, etc.

[0078] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with tuning switches.

[0079] The mobile communication module 150 can provide solutions for wireless communication, including 2G / 3G / 4G / 5G, applied to the electronic device 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1. In some embodiments, at least some functional modules of the mobile communication module 150 may be housed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 may be housed in the same device.

[0080] The modem processor may include a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs sound signals through an audio device (not limited to speaker 170A, receiver 170B, etc.) or displays images or videos through the display screen 194. In some embodiments, the modem processor may be a separate device. In other embodiments, the modem processor may be independent of the processor 110 and may be housed in the same device as the mobile communication module 150 or other functional modules.

[0081] The wireless communication module 160 can provide solutions for wireless communication applications on the electronic device 100, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, demodulates and filters the electromagnetic wave signals, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, frequency modulate and amplify them, and then convert them into electromagnetic waves for radiation via antenna 2.

[0082] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, enabling electronic device 100 to communicate with networks and other devices via wireless communication technology. The wireless communication technology may include Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technologies, etc. The GNSS may include the Global Positioning System (GPS), the Global Navigation Satellite System (GLONASS), the BeiDou Navigation Satellite System (BDS), the Quasi-Zenith Satellite System (QZSS), and / or satellite-based augmentation systems (SBAS).

[0083] Electronic device 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0084] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a miniature LED, a microLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, electronic device 100 may include one or N displays 194, where N is a positive integer greater than 1.

[0085] Electronic device 100 can perform shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.

[0086] The ISP (Image Signal Processor) is used to process data fed back from the camera 193. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, converting it into an image visible to the naked eye. The ISP can also perform algorithmic optimization on image noise and brightness. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 193.

[0087] Camera 193 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments, the electronic device 100 may include one or N cameras 193, where N is a positive integer greater than 1.

[0088] Digital signal processors (DSPs) are used to process digital signals. Besides digital image signals, they can also process other digital signals. For example, when electronic device 100 selects a frequency, the DSP can perform Fourier transforms on the frequency energy.

[0089] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. Thus, electronic device 100 can play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0090] An NPU (Neural Processing Unit) is a computational processor for neural networks (NNs). By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs enable intelligent cognitive applications in electronic devices, such as image recognition, facial recognition, speech recognition, and text understanding.

[0091] Internal memory 121 may include one or more random access memory (RAM) and one or more non-volatile memory (NVM).

[0092] Random access memory can include static random-access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), and double data rate synchronous dynamic random access memory (DDR SDRAM, such as fifth-generation DDR SDRAM, which is generally called DDR5 SDRAM). Non-volatile memory can include disk storage devices and flash memory.

[0093] Flash memory can be classified according to its operating principle, including NOR FLASH, NAND FLASH, 3D NAND FLASH, etc.; according to the level of the storage cell, including single-level cell (SLC), multi-level cell (MLC), triple-level cell (TLC), quad-level cell (QLC), etc.; and according to the storage specification, including universal flash storage (UFS) and embedded multimedia card (eMMC), etc.

[0094] The random access memory can be directly read and written by the processor 110. It can be used to store executable programs (such as machine instructions) of the operating system or other running programs, as well as user and application data.

[0095] Non-volatile memory can also store executable programs and user and application data, and can be pre-loaded into random access memory for direct reading and writing by the processor 110.

[0096] The external memory interface 120 can be used to connect to external non-volatile memory, thereby expanding the storage capacity of the electronic device 100. The external non-volatile memory communicates with the processor 110 through the external memory interface 120 to perform data storage functions. For example, music, video, and other files can be stored in the external non-volatile memory.

[0097] Electronic device 100 can implement audio functions, such as music playback and recording, through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0098] The audio module 170 is used to convert digital audio information into analog audio signals for output, and also to convert analog audio input into digital audio signals. The audio module 170 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 170 may be located in the processor 110, or some functional modules of the audio module 170 may be located in the processor 110.

[0099] The speaker 170A, also known as a "loudspeaker," is used to convert audio electrical signals into sound signals. The electronic device 100 can listen to music or make hands-free calls through the speaker 170A.

[0100] The receiver 170B, also known as the "earpiece," is used to convert audio electrical signals into sound signals. When the electronic device 100 answers a telephone call or voice message, the receiver 170B can be brought close to the ear to listen to the voice.

[0101] Microphone 170C, also known as a "microphone" or "voice transducer," is used to convert sound signals into electrical signals. When making a phone call or sending a voice message, the user can speak by bringing their mouth close to microphone 170C, inputting the sound signal into microphone 170C. Electronic device 100 may have at least one microphone 170C. In some embodiments, electronic device 100 may have two microphones 170C, which, in addition to collecting sound signals, can also perform noise reduction. In other embodiments, electronic device 100 may also have three, four, or more microphones 170C, which can collect sound signals, reduce noise, identify the sound source, and perform directional recording, etc.

[0102] The 170D headphone jack is used to connect wired headphones. The 170D headphone jack can be a USB 130 interface or a 3.5mm Open Mobile Terminal Platform (OMTP) standard interface, a CTIA (Cellular Telecommunications Industry Association of the USA) standard interface.

[0103] The pressure sensor 180A is used to sense pressure signals and can convert the pressure signals into electrical signals. In some embodiments, the pressure sensor 180A can be disposed on the display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc.

[0104] The gyroscope sensor 180B can be used to determine the motion attitude of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 about three axes (i.e., the x, y, and z axes) can be determined by the gyroscope sensor 180B. The gyroscope sensor 180B can be used for image stabilization.

[0105] The barometric pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates altitude using the air pressure value measured by the barometric pressure sensor 180C to assist in positioning and navigation.

[0106] The magnetic sensor 180D includes a Hall sensor. The electronic device 100 can use the magnetic sensor 180D to detect the opening and closing of the flip cover.

[0107] The accelerometer 180E can detect the magnitude of acceleration of electronic device 100 in various directions (generally three axes).

[0108] Distance sensor 180F is used to measure distance. Electronic device 100 can measure distance via infrared or laser.

[0109] The proximity light sensor 180G may include, for example, a light-emitting diode (LED) and a light detector, such as a photodiode. The LED may be an infrared LED.

[0110] The 180L ambient light sensor is used to detect ambient light intensity.

[0111] The fingerprint sensor 180H is used to collect fingerprints.

[0112] The 180J temperature sensor is used to detect temperature.

[0113] Touch sensor 180K, also known as a "touch device," can be located on display screen 194. The touch sensor 180K and display screen 194 together form a touchscreen, also known as a "touchscreen." Touch sensor 180K detects touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 194. In other embodiments, touch sensor 180K may also be located on the surface of electronic device 100, in a different position than display screen 194.

[0114] The bone conduction sensor 180M can acquire vibration signals.

[0115] Buttons 190 include a power button, volume buttons, etc. Buttons 190 can be mechanical buttons or touch buttons. Electronic device 100 can receive button input and generate key signal inputs related to user settings and function control of electronic device 100.

[0116] Motor 191 can generate vibration alerts. Motor 191 can be used for incoming call vibration alerts or for touch vibration feedback.

[0117] Indicator 192 can be an indicator light, which can be used to indicate charging status, power changes, messages, missed calls, notifications, etc.

[0118] The SIM card interface 195 is used to connect the SIM card.

[0119] The electronic device 100 provided in this application embodiment can run an operating system (OS). This operating system can be various operating systems used in industry, such as operating systems developed based on OpenHarmony, like HarmonyOS; or other operating systems such as Android™, iOS mobile operating systems; it can also be various open-source operating systems or their derivatives, such as Linux OS, and other embedded operating systems; or it can be a future new operating system, such as an AI operating system based on artificial intelligence. An operating system is a set of interconnected system software programs that manage and control the operation of the electronic device 100, utilize and run hardware and software resources, and provide public services to organize user interaction. In the electronic device 100, the operating system connects downwards to the physical devices at the hardware layer and provides a runtime environment for application software upwards.

[0120] An operating system typically includes a kernel layer, a middleware layer, and an application layer. The application layer includes applications, which can include system applications and third-party applications. The middleware layer includes a suite of software providing various services to application developers, or frameworks providing services such as databases, multimedia, and graphics, or capabilities such as distributed scheduling and system scaling. For example, the middleware layer may include a framework layer and / or a system service layer. The framework layer provides application programming interfaces (APIs) and programming frameworks for applications in the application layer. The system service layer includes the system's core capabilities, providing services to applications through the framework layer. The kernel layer is the layer between hardware and software. The kernel layer may include hardware drivers and the operating system kernel. In addition to providing hardware drivers, the kernel layer also supports functions such as memory management and system process management.

[0121] The electronic devices we use in our daily lives come in various types and forms, and are applied in a wide range of scenarios. Therefore, based on the different forms and functions of electronic devices, different application scenarios, and different user needs, the operating systems used in these devices may also differ. The basic functions implemented by the electronic device 100 provided in this application can be implemented using a general-purpose operating system or a dedicated operating system. To more clearly illustrate the implementation of the embodiments of this application under a specific operating system, the architecture of HarmonyOS is shown below. Those skilled in the art can deduce the implementation of the embodiments of this application under other specific operating systems, such as Android™.

[0122] For example, such as Figure 7AAs shown, the software architecture of electronic device 100 can be divided into several layers. In some embodiments, from bottom to top, these layers are: kernel layer, system service layer, framework layer, and application layer. Layers communicate with each other through software interfaces. System functions can be tailored, added, or combined at the subsystem granularity in different device deployment scenarios, and each subsystem can also be tailored, added, or combined at the functional granularity.

[0123] The kernel layer can include: Kernel Abstract Layer (KAL), kernel subsystem, driver subsystem, etc.

[0124] Kernel Abstraction Layer: By shielding the differences between multiple kernels, it provides basic kernel capabilities to the upper layers, including but not limited to process / thread management, memory management, file system, network management, and peripheral device management.

[0125] Kernel Subsystem: Supports the selection of a suitable OS kernel for different resource-constrained devices, including but not limited to Linux kernel, HarmonyOS kernel, LiteOS (Lite Operating System), etc.

[0126] Driver Subsystem: The driver framework is the foundation for the open system hardware ecosystem, providing unified peripheral access capabilities and a framework for driver development and management. The driver framework includes: display drivers, camera drivers, audio drivers, Bluetooth drivers, sensor drivers, etc.

[0127] The system service layer comprises the core capabilities of the system, providing services to applications through the framework layer. This layer includes, but is not limited to, the following subsystems:

[0128] The system's basic capability subsystem set provides fundamental capabilities for the operation, scheduling, and migration of distributed applications across multiple devices. This set may include distributed soft bus, distributed data management, distributed task scheduling, and Ark multi-language runtime; it may also include multi-modal input subsystem, graphics subsystem, security subsystem, and AI business subsystem.

[0129] The basic software service subsystem set provides common and general software services. This set may include an event notification subsystem, a telephone service subsystem, a multimedia subsystem, an audio framework system, etc. The multimedia subsystem, in particular, provides a simple and easy-to-understand interface, allowing applications to access and use the system's media resources.

[0130] Enhanced software service subsystem suite: Provides differentiated enhanced software services for different devices; the enhanced software service subsystem suite may include smart screen proprietary business subsystem, wearable proprietary business subsystem, IoT proprietary business subsystem, etc.

[0131] Hardware service subsystem set: Provides hardware services; the hardware service subsystem set may include location service subsystem, user IAM (Identity and Access Management) subsystem, wearable proprietary hardware service subsystem, biometric identification, IoT proprietary hardware service subsystem, etc.

[0132] Distributed task scheduling enables distributed service management (discovery, synchronization, registration, and invocation), supporting remote startup, remote invocation, remote connection, and migration of applications across devices.

[0133] Distributed data management enables data synchronization, data storage, data sharing, and data access across all scenarios and devices.

[0134] The distributed soft bus provides communication-related capabilities for seamless interconnection between multiple devices, including: WLAN service capabilities, Bluetooth service capabilities, soft bus, inter-process communication RPC (Remote Procedure Call), and StarFlash communication capabilities.

[0135] Ark Multilingual Runtime is a unified compilation runtime platform designed to support the joint compilation and execution of multiple programming languages ​​and multiple chip platforms.

[0136] The framework layer provides application programming interfaces (APIs) and programming frameworks for applications in the application layer. The framework layer includes: the ArkUI framework (which provides a complete infrastructure for UI development of system applications, including UI functions such as components, layouts, animations, and interactive events, as well as a real-time interface preview tool), the user application framework, and the Ability framework (an Ability is a lightweight application; the Ability framework schedules and manages the operation and lifecycle of Abilities). Different devices may have different operating systems, and therefore support different APIs.

[0137] The HarmonyOS API is a series of open capabilities provided to support HarmonyOS application development. The HarmonyOS API can be set at the framework layer or independently of the framework layer. The HarmonyOS API includes the Audio API (audio service), Push API (push service), Account API (account service), and Smart Screen Recognition service.

[0138] Applications can include system apps and extended / third-party apps. System apps can include the desktop, control bar, settings, contacts, phone, camera, etc., while extended / third-party apps can include social apps, travel apps, etc.

[0139] In some embodiments, the software architecture of the electronic device may also refer to, for example, Figure 7B and Figure 7C As shown. Figure 7B and Figure 7C As shown, the system architecture, from top to bottom, consists of: application layer, system service layer, and kernel layer. In one implementation, the application layer, system service layer, and kernel layer can be defined as follows: Figure 7A Related descriptions.

[0140] The application layer may include a series of application packages, such as a desktop; the desktop includes SceneBoard, which is used to manage desktop processes. In this embodiment, the application layer includes application 1, which can initiate a scaling request for the original image through SceneBoard. The scaling request is used to obtain the target image corresponding to the original image.

[0141] Application 1 can be a system application (e.g., a desktop or other system application) or a third-party application. The image processing method provided in this application embodiment can scale 2D images and can also adaptively extend to 3D scenes to scale 3D images; therefore, Application 1 can also be a 2D application, a 3D application, or an extended reality (XR) application.

[0142] The system service layer includes a graphics subsystem; this subsystem may include UI components, layout, animation, fonts, input events, window management, rendering, and other modules. The related services of the graphics subsystem can provide graphics rendering and display output functions, and through the rational utilization of system hardware resources, provide a smooth and efficient display experience for the system.

[0143] like Figure 7B and Figure 7CAs shown, the graphics subsystem includes some or all of the following: a rendering service (RenderService), a rendering engine, a shader library, and a media library. The RenderService receives drawing instructions for self-rendering layers and controls from other system service processes (e.g., desktop processes) and user processes, performs unified combination and rendering control; its rendering actions can be performed using general-purpose computing devices such as CPUs / GPUs. In some embodiments, the rendering service can be used to control the rendering of a target image.

[0144] The rendering engine provides 2D (and / or 3D) graphics for drawing the underlying API, supporting low-level capabilities for both graphics and text drawing. In some embodiments, the rendering engine can be used to render a target image.

[0145] Shader libraries and / or media libraries can be used to store SLR algorithms, which are used to implement the image processing methods provided in this application. The SLR algorithms in the shader library are implemented as GPU shader code, while the SLR algorithms in the media library are implemented as CPU code.

[0146] The kernel layer is the layer between hardware and software. It can include hardware drivers and the operating system kernel, supporting functions such as memory management and system process management. For example, the kernel layer contains at least a GPU driver and a display driver. The GPU driver is used to drive the GPU to render the target image; the display driver is used to drive the display screen to display the target image.

[0147] In some use cases (such as screen mirroring), see [link to relevant documentation]. Figure 7B Based on the image scaling request from Application 1, SceneBoard calls RenderService to scale the original image; RenderService calls the SLR algorithm through the shader library to scale the original image and outputs the target image corresponding to the original image for projection; the target image output by the SLR algorithm is sent to the rendering engine, which then renders the target image; the rendering engine calls the GPU to obtain the rendered target image. Electronic device 100 projects the rendered target image onto the target screen.

[0148] In some use cases (such as icon scaling), see [link to relevant documentation]. Figure 7CBased on the image scaling request from Application 1, SceneBoard uses the media library to call the SLR algorithm to scale the original image of the icon to the target size, and then calls RenderService to render the target image of the icon. RenderService calls the rendering engine to render the target image of the icon. The rendering engine calls the GPU to obtain the rendered target image of the icon. Electronic device 100 displays the rendered target image of the icon on the screen, for example, on the main interface.

[0149] This application provides an image processing method that cascades an image scaling algorithm and an image sharpening algorithm. First, the image scaling algorithm is used to reduce the size of the original image, obtaining an intermediate image of the same size as the target image. Then, the image sharpening algorithm is used to sharpen the intermediate image to improve its clarity, resulting in the target image. This effectively improves the jagged edges and clarity of the scaled image, enhancing the user experience. For example, Figure 8 A flowchart of the image processing method is shown, which includes, but is not limited to, steps S101 to S106.

[0150] S101. Based on the resolution of the original image and the resolution of the target image, the electronic device 100 determines the scaling ratio, which includes the horizontal scaling ratio and the vertical scaling ratio.

[0151] The original image has a resolution of sw*sh, meaning the horizontal pixel count is sw and the vertical pixel count is sh. The target image has a resolution of dw*dh, meaning the horizontal pixel count is dw and the vertical pixel count is dh. In some embodiments, the scaling ratio of the original and target images includes a horizontal scaling ratio and a vertical scaling ratio. The horizontal scaling ratio rw is equal to the ratio of the horizontal pixel count of the target image to that of the original image, i.e., rw = dw / sw. The vertical scaling ratio rh is equal to the ratio of the vertical pixel count of the target image to that of the original image, i.e., rh = dh / sh.

[0152] In this embodiment, the electronic device 100 can determine parameter 1 according to the scaling ratio; wherein, parameter 1 is used to obtain the target image after scaling the original image, and parameter 1 can be any one of the following: convolution kernel radius Gw*Gh, convolution kernel weight matrix K, and sharpening intensity alpha. In some embodiments, if the electronic device 100 obtains a preset value for parameter 1, the electronic device 100 can determine that the value of parameter 1 is the aforementioned preset value; if the electronic device 100 does not obtain a preset value for parameter 1, then parameter 1 is dynamically adjusted according to the scaling ratio.

[0153] S102. Based on the scaling ratio, the electronic device 100 determines the convolution kernel radius Gw*Gh.

[0154] In some embodiments, if the electronic device 100 obtains a preset value 1 for the kernel radius (e.g., gw1*gh1), then the kernel radius is determined to be the preset value 1, that is, the width of the kernel radius Gw = gw1 and the height of the kernel radius Gh = gh1; if the electronic device 100 does not obtain a preset value for the kernel radius, then the kernel radius is determined according to the scaling ratio.

[0155] In some embodiments, the width of the convolution kernel radius increases as the horizontal scaling ratio decreases, and the height of the convolution kernel radius increases as the vertical scaling ratio decreases. In one implementation, when rw = rw2, Gw = gw2; when rw = rw3, Gw = gw3; where rw3<rw2,gw3> When gw2;rh=rh2, Gh=gh2; when rh=rh3, Gh=gh3; where rh3<rh2,gh3> gh2.

[0156] In some embodiments, the height of the convolution kernel radius is proportional to the reciprocal of the vertical scaling ratio, and the width of the convolution kernel radius is proportional to the reciprocal of the vertical scaling ratio. In one implementation, the height of the convolution kernel radius is equal to the reciprocal of the vertical scaling ratio, i.e., Gh = [1 / rh], and the width of the convolution kernel radius is equal to the reciprocal of the vertical scaling ratio, i.e., Gw = [1 / rw]. In one implementation, the symbol [*] indicates rounding down. In another implementation, the symbol [*] indicates rounding to the nearest integer. For example, taking the reduction of a 1024*1024 original image to a target image of 176*176 as an example, in this example, the original image is square, the horizontal scaling ratio rw and the vertical scaling ratio rh are both equal to 176 / 1024, i.e., 11 / 64, and the width Gw and height Gh of the convolution kernel are also equal, i.e., Gh = Gw = G; Figure 9 An exemplary schematic diagram of the discrete curve of the aliasing intensity in the target image as a function of the convolution kernel radius G is shown; as can be seen from the figure, the aliasing intensity is lowest when G = 6 (i.e. [64 / 11]). Figure 9 The aliasing intensity shown is the gradient value obtained by calculating the second-order gradient of the edge points of the image. The larger the gradient value, the more severe the aliasing. This application does not specifically limit the calculation method of aliasing intensity.

[0157] Step S102 is optional. In some embodiments, other electronic devices (e.g., electronic device 200) execute steps S101 and S102 to obtain the convolution kernel radius; electronic device 100 directly obtains the convolution kernel radius from electronic device 200. This application embodiment does not specifically limit the device type of electronic device 200, nor does it specifically limit the communication method between electronic device 100 and electronic device 200.

[0158] S103. Based on the scaling ratio, the electronic device 100 determines the weight matrix K of the convolution kernel; the weight matrix K includes the convolution center C.ij The corresponding weight matrix K ij i and j are positive integers, with i ranging from [0, dh-1] and j ranging from [0, dw-1].

[0159] In some embodiments, if the electronic device 100 obtains the weight matrix K of the convolution kernel... ij Given the preset matrix K1, the weight matrix K is then determined. ij The value is taken as the preset matrix K1. If the electronic device 100 does not obtain the preset matrix of the weight matrix K, the weight matrix K is determined according to the scaling ratio. ij .

[0160] In some embodiments, when the kernel radius is fixed, the weight matrix of the kernel fluctuates more smoothly as the scaling ratio decreases, meaning that the low-pass bandwidth of the convolution operation is smaller during image scaling.

[0161] In one implementation, the convolution center C in the original image... ij The coordinates are (C i C j Convolution window 1 is the original image with convolution center C. ij For a convolution window within the radius of the central convolution kernel, the pixel in the m-th row and n-th column of the original image can be represented as S. mn S in the original image mn The value of m ranges from [0, sh-1], and the value of n ranges from [0, sw-1]; in convolution window 1, S mn The range of values ​​for m is The range of values ​​for n is symbol This indicates rounding down by *. Figure 10 As shown, the weight matrix K corresponding to convolution window 1 is determined according to the scaling ratio. ij This includes steps S1 to S5.

[0162] S1. Determine the pixel P of the intermediate image based on the horizontal scaling ratio rw and the vertical scaling ratio rh. ij 'The convolution center C in the original image ij The 2D coordinates, i.e. (C i C j ), pixel P ij ' represents the pixel in the i-th row and j-th column of the intermediate image.

[0163] In some embodiments,

[0164] It should be noted that the embodiments of this application specify the convolution center C. ij The method for determining the coordinates is not specifically limited and is not limited to Formula 1 above; the convolution center C can also be calculated using other formulas.ij The coordinates.

[0165] S2, based on convolution center C ij Given the kernel radius Gw*Gh, determine the convolution center C. ij The corresponding pixel matrix S of convolution window 1 ij .

[0166] In this embodiment of the application, the convolution center C is located in the original image. ij Pixels within the radius of the convolution kernel are extracted to generate convolution window 1.

[0167] In some embodiments, when the convolution center C ij The lateral distance from the edge of the original image is greater than or equal to the width Gw of the convolution kernel radius, and the convolution center C ij When the vertical distance from the edge of the original image is greater than or equal to the height Gh of the convolution kernel radius, the convolution center C ij The corresponding convolution window 1 is within the original image range. Based on the pixel values ​​of the pixels within convolution window 1 in the original image, the pixel matrix S of convolution window 1 can be directly determined. ij Pixel matrix S ij It can be represented as in, m1 is the row number of the first row within convolution window 1. mg is the row number of the last row within convolution window 1. n1 is the row number of the first row within convolution window 1. mg is the row number of the last row within convolution window 1. Pixel S within convolution window 1 mn , refers to the pixel with row number m and column number n within convolution window 1.

[0168] For example, such as Figure 11A and Figure 11B As shown, taking "reducing the resolution of an 8*8 original image to a 4*4 target image" as an example, based on the resolutions of the original and target images, the horizontal scaling ratio rw and the vertical scaling ratio rh are both equal to 0.5, the convolution kernel radius is 2*2, and the convolution window size is 4*4. Figure 11A As shown, the pixel P of the intermediate image 11 For example, pixel P 11 'The convolution center C in the original image 11 The coordinates of ' are (2.5, 2.5). Convolution window 1 includes the pixels in columns 1-4 of rows 1-4 of the original image. Convolution window 1 does not exceed the range of the original image.

[0169] In some embodiments, the convolution center C ij The lateral distance from the edge of the original image is less than the width Gw of the convolution kernel radius, and / or the convolution center C. ijWhen the vertical distance from the edge of the original image is less than the height Gh of the convolution kernel radius, the range of convolution window 1 will exceed the original image. For pixels within convolution window 1 that exceed the original image, if the row of the pixel within convolution window 1 does not exceed the range of the original image, but the column of the pixel within convolution window 1 does, then the pixel can be set to the value of the original image. Figure 1 The pixel value of the nearest pixel in the inner horizontal direction; if the pixel column within the convolution window 1 does not exceed the range of the original image, but the pixel row exceeds the range of the original image, then the pixel value can be the original value. Figure 1 The pixel value is the pixel value of the closest horizontally distance within the convolution window 1. If the pixel row and column of the pixel in the convolution window 1 are outside the range of the original image, the pixel value can be the pixel value of the corner pixel of the original image within the convolution window 1. The corner pixel is the pixel corresponding to the four corners of the original image.

[0170] For example, such as Figure 11B As shown, the pixel P of the intermediate image 00 For example, pixel P 00 'The convolution center C in the original image 00 The coordinates are (0.5, 0.5). Convolution window 1 includes pixels in columns 0-2 of rows 0-2 of the original image. The first row and first column of pixels in convolution window 1 both extend beyond the range of the original image. Based on the aforementioned pixel matrix S... ij It can be seen that the row and column indices that extend beyond the original image within convolution window 1 can be assigned negative values; for example, Figure 11B In the example, The pixel values ​​of the last three columns of the first row of convolution window 1 are respectively taken as the pixel values ​​S of the original image. 00 S 01 and S 02 The pixel values ​​of the last three rows of pixels in the first column of convolution window 1 are respectively taken as the pixel values ​​S of the original image. 00 S 10 and S 20 The pixel value in the first row and first column of convolution window 1 is taken as the corner pixel S of the original image. 00 .

[0171] S3, Determine the pixel S within convolution window 1. mn The corresponding horizontal weight L n L n Equal to the number of pixels S within the convolution window 1 mn L(X) n The function is the normalized value of L(*) relative to the L(*) function of all pixels in the same pixel row; where L(*) is the Lanczos kernel function, L(X) is the normalized value of L(X) relative to the L(*) function of all pixels in the same pixel row. n ) = sinc(X n )*sinc(X n / Gw), X n For S mnThe adjustment function, X n =rw*(n–C j ) / beta_w, where beta_w is the bandwidth adjustment parameter.

[0172] Understandable, L n It is based on the horizontal scaling ratio rw, the horizontal distance between the pixels within the convolution window and the center of the convolution (i.e., (n–C)). j It is determined by )) and beta_w.

[0173] S4. Determine the pixel S within convolution window 1. mn The corresponding vertical weight L m L m Equal to the number of pixels S within the convolution window 1 mn L(X) m The function is the normalized value of the L(*) function relative to all pixels in its column; where L(X) is the normalized value of the L(*) function. m ) = sinc(X m )*sinc(X m / Gh), X m For pixel S mn The adjustment function, X m =rh*(m–C i ) / beta_h, where beta_h is the bandwidth adjustment parameter.

[0174] Understandable, L m It is based on the vertical scaling ratio rw and the vertical distance between the pixels within the convolution window and the center of the convolution (i.e., (m–C)). i It is determined by )) and beta_h.

[0175] In steps S3 and S4, beta_w and beta_h are used to finely adjust the bandwidth of the low-pass filter for image scaling. The larger beta_w and beta_h are, the smoother the fluctuation of the weights in the convolution kernel, and the closer they are to the average value. This results in lower jagged edges in the image, but at the same time, the image is also more blurred. Therefore, the values ​​of beta_w and beta_h should balance jagged edge intensity and sharpness. The electronic device 100 can set beta_w and beta_h according to actual needs.

[0176] In some embodiments, both beta_w and beta_h are equal to beta. When the scaling factor is greater than rw1 (e.g., 0.8), the optimal range of beta values ​​is [1, a1], and when the horizontal scaling factor is less than rw1 (e.g., 0.8), the optimal range of beta values ​​is [a2, 1]. In one implementation, based on extensive experiments, it is known that when rw1 equals 0.8, the optimal range of beta values ​​is [1, 1.2] when the scaling factor is greater than 0.8, and the optimal range of beta values ​​is [0.8, 1] when the scaling factor is less than 0.8. For example, when a 260*260 resolution icon is reduced to a 189*189 resolution icon, the scaling factor is less than 0.8. When beta is changed from 1 to 1.1, the aliasing intensity calculated based on the second-order gradient decreases by 13%.

[0177] In some embodiments, beta_w and beta_h are not equal. When the horizontal scaling ratio is greater than rw1 (e.g., 0.8), the optimal value of beta is [1, a1], and when the horizontal scaling ratio is less than rw1 (e.g., 0.8), the optimal value of beta_w is [a2, 1]. When the vertical scaling ratio is greater than rw2 (e.g., 0.8), the optimal value of beta_h is [1, a3], and when the vertical scaling ratio is less than rw2 (e.g., 0.8), the optimal value of beta_h is [a4, 1]. In one implementation, based on extensive experiments, it is known that rw1 = rw2 equals 0.8; when the horizontal scaling ratio is greater than 0.8, the optimal value of beta_w is [1, 1.2], and when the horizontal scaling ratio is less than 0.8, the optimal value of beta_w is [0.8, 1]; when the vertical scaling ratio is greater than 0.8, the optimal value of beta_h is [1, 1.2], and when the vertical scaling ratio is less than 0.8, the optimal value of beta_h is [0.8, 1].

[0178] It should be noted that the optimal value of beta (or beta_w and beta_h) may be different in different scenarios; electronic device 100 can obtain the optimal value of beta under the actual needs of different scenarios through a large number of experiments.

[0179] S5, Determine the convolution window size as 1 pixel. mn The corresponding weight K mn =L n *L m Then, the weight matrix K corresponding to convolution window 1 is obtained. ij .

[0180] The weight matrix K of the convolution kernel corresponding to convolution window 1 ij It can be represented as Wherein, m1, mg, n1 and ng can be referred to the relevant description in step S2.

[0181] Step S103 is optional. In some embodiments, other electronic devices (e.g., electronic device 200) perform steps S101 and S103 to obtain the weight matrix K; electronic device 100 directly obtains the aforementioned weight matrix K from electronic device 200.

[0182] S104. Based on the convolution kernel radius and the weight matrix K of the convolution kernel, the electronic device 100 performs image scaling on the original image to obtain the intermediate image corresponding to the original image.

[0183] In some embodiments, image scaling is performed on the original image based on the weight matrix K of the convolution kernel, including: scaling the convolution center C. ij The corresponding pixel matrix S of convolution window 1 ij With weight matrix K ij Perform convolution operation to obtain pixel P in the intermediate image. ij The pixel value of '. In this way, the pixel value of each pixel in the intermediate image can be obtained, thus obtaining the intermediate image. For example, P ij The formula for calculating ' is shown below.

[0184]

[0185] It is understandable that the weight matrix K of the convolution kernel... ij The weight matrix K can be adjusted by the parameter beta; the larger the beta, the larger the weight matrix K. ij The smaller the fluctuation of the weight values ​​in the formula, the closer they are to the average value.

[0186] S105. Based on the scaling ratio, the electronic device 100 determines the sharpening intensity alpha.

[0187] In this embodiment, a sharpening algorithm is used to extract high-frequency information from an intermediate image through a high-pass filter and superimpose it onto the intermediate image to obtain a sharpened target image. The intermediate image and the target object have the same resolution; compared to the intermediate image, the target image has improved image clarity. When superimposing high-frequency information with the intermediate image, the high-frequency information is multiplied by the sharpening intensity alpha. alpha controls the sharpening intensity; a value greater than 0 indicates a more pronounced sharpening effect and a clearer image, but also stronger jagged edges. Therefore, the value of alpha must balance jagged edges and image clarity. This embodiment does not specifically limit the sharpening algorithm used for sharpening. For example, the sharpening algorithm can be an anti-sharpening mask algorithm. In some embodiments, if the electronic device 100 obtains a preset value of 2 for the sharpening intensity alpha, then the value of alpha is determined to be the preset value 2; if the electronic device 100 does not obtain the preset value of alpha, then alpha is determined according to the scaling ratio.

[0188] In some embodiments, the sharpening intensity alpha does not decrease as the scaling ratio r decreases. In one implementation, the scaling ratio r is a horizontal scaling ratio, a vertical scaling ratio, or the average value of the horizontal scaling ratio and the vertical scaling ratio. In one implementation, when r=r1, alpha=alpha_1; when r=r2, alpha=alpha_2; where r2<r1, alpha_2>alpha_1. In one implementation, when r is equal to r1 or r3, alpha=alpha_1; where r3<r1.

[0189] In some embodiments, determining alpha according to the scaling ratio includes: when the scaling ratio r is greater than rw3 (for example, 0.8), the optimal value range of alpha is [0, b1]; when the scaling ratio r is between (rw4, rw3], the optimal value range of alpha is [b2, b3]; when the scaling ratio r is between [rw5, rw4], the optimal value range of alpha is [b4, b5]; when the scaling ratio r is below rw5 (for example, 0.5), the optimal value range of alpha is [b6, b7]. Wherein, rw5<rw4<rw3. In one implementation, 0<b2<b4<b6, b1<b3<b5<b7.

[0190] In one implementation, it can be known from a large number of experiments that when the scaling ratio r is greater than 0.8, the optimal value range of alpha is [0, 0.1]; when the scaling ratio r is between 0.6 and 0.8, the optimal value range of alpha is [0.06, 0.2]; when the scaling ratio r is between 0.5 and 0.6, the optimal value range of alpha is [0.1, 0.25]; when the scaling ratio r is below 0.5, the optimal value range of alpha is [0.15, 0.3].

[0191] It should be noted that the optimal value of alpha may vary in different scenarios; the electronic device 100 may obtain the optimal value of alpha under the scenario requirement through a large number of experiments according to the actual requirements of different scenarios.

[0192] Step S104 is optional. In some embodiments, other electronic devices (for example, the electronic device 200) perform steps S101 and S105 to obtain the sharpening intensity alpha; the electronic device 100 directly obtains the above-mentioned sharpening intensity alpha from the electronic device 200.

[0193] S106. According to the sharpening intensity alpha, the electronic device 100 uses a sharpening algorithm to perform sharpening processing on the intermediate image to obtain a target image.

[0194] In some embodiments, the pixel P in the i-th row and j-th column of the target image is determined. ij =P ij '+alpha*h ij h ij =(4*P ij '–P (i-1)j '–P (i+1)j '–P i(j-1) '–P i(j+1) '). Among them, h ij Instructed P ij The corresponding high-frequency information can be understood as pixel P. ij The gradient value at point ', P (i-1)j ', P (i+1)j ', P i(j-1) ', P i(j+1) 'Is pixel P' ij The pixels at the top, bottom, left, and right positions of the target image are used. Thus, based on the intermediate image and the alpha value, the pixel values ​​of each pixel in the target image can be obtained, i.e., the target image is obtained.

[0195] Based on the foregoing embodiments, this application provides an image processing method. For example, Figure 12 A flowchart of the image processing method is shown, which includes, but is not limited to, steps S201 to S202.

[0196] S201. Scale the first image at the first resolution to obtain an intermediate image at the second resolution, where the second resolution is smaller than the first resolution.

[0197] S202. Based on the sharpening intensity, the intermediate image is sharpened to obtain a target image with a second resolution; the sharpening intensity is determined according to the first scaling ratio of the target image relative to the first image, and the first scaling ratio is determined according to the second resolution and the first resolution.

[0198] The first image can be the original image from the aforementioned embodiments, the first resolution is the resolution of the original image, and the second resolution is the resolution of the target image after scaling up the original image.

[0199] In some embodiments, when the first scaling ratio is a first ratio, the sharpening intensity is a first value; when the first scaling ratio is a second ratio, the sharpening intensity is a second value; the second ratio is less than the first ratio, and the second value is greater than the first value. For example, referring to the relevant description of step S105, when the first scaling ratio is r, r = r1, alpha = alpha_1; when r = r2, alpha = alpha_2; where r2...<r1,alpha_2> alpha_1.

[0200] In some embodiments, when the first scaling ratio is the third scaling ratio, the sharpening intensity is the first value; the third scaling ratio is less than the first scaling ratio. For example, referring to the relevant description of step S105, when r equals r1 or r3, alpha = alpha_1; where r3... <r1。

[0201] In some embodiments, the first scaling factor is determined based on a horizontal scaling factor and a vertical scaling factor; the horizontal scaling factor is equal to the width of the second resolution divided by the width of the first resolution, and the vertical scaling factor is equal to the height of the second resolution divided by the height of the first resolution. In one implementation, the first scaling factor is one of the following: a horizontal scaling factor, a vertical scaling factor, or the average of the horizontal and vertical scaling factors.

[0202] In some embodiments, scaling a first image at a first resolution to obtain an intermediate image at a second resolution includes: scaling the first image at the first resolution based on the kernel radius and the kernel weight matrix to obtain an intermediate image at the second resolution, wherein the kernel radius and / or the weight matrix are determined based on the horizontal scaling ratio and the vertical scaling ratio.

[0203] In some embodiments, the width of the kernel radius increases as the horizontal scaling ratio decreases, and the height of the kernel radius increases as the vertical scaling ratio decreases. In some embodiments, the width of the kernel radius is proportional to the reciprocal of the horizontal scaling ratio, and the height of the kernel radius is proportional to the reciprocal of the vertical scaling ratio. In one implementation, referring to the relevant description of step S102, the height of the kernel radius is equal to the reciprocal of the vertical scaling ratio, i.e., Gh = [1 / rh], and the width of the kernel radius is equal to the reciprocal of the vertical scaling ratio, i.e., Gw = [1 / rw].

[0204] In some embodiments, the low-pass bandwidth corresponding to the weight matrix of the convolution kernel decreases as the first scaling factor decreases. Specifically, refer to the description of the low-pass bandwidth in step S103.

[0205] In some embodiments, the weight matrix of the convolution kernel is associated with the bandwidth adjustment parameter beta; the low-pass bandwidth corresponding to the weight matrix of the convolution kernel decreases as beta increases. For details, please refer to the description of beta in step S103.

[0206] In some embodiments, the first resolution is sw*sh, and the second resolution is dw*dh. Based on the kernel radius and the kernel weight matrix, the first image at the first resolution is scaled to obtain an intermediate image at the second resolution, including: determining the pixel P of the intermediate image. ij 'The convolution center C in the original image ij coordinates (C) i C j), P ij ' represents the pixel in the i-th row and j-th column of the intermediate image, where i ranges from [0, dh-1] and j ranges from [0, dw-1]; based on the convolution center C ij And the kernel radius, determine the convolution center C ij The pixel matrix S corresponding to the first convolution window ij Based on the scaling ratio and bandwidth adjustment parameter beta, determine the weight matrix K corresponding to the first convolution window. ij For pixel matrix S ij With weight matrix K ij Perform convolution operation to obtain the pixel P of the intermediate image. ij The pixel value of '. The first convolutional window can be the aforementioned convolutional window 1. Specifically, refer to the weight matrix K in step S103. ij Related descriptions.

[0207] In some embodiments, pixel S mn Let m be the pixel in the m-th row and n-th column within the first convolution window, where m can take values ​​ranging from 1 to 2. The range of values ​​for n is symbol This indicates rounding down by *. Gw is the width of the convolution kernel radius, Gh is the height of the convolution kernel radius, rw is the horizontal scaling ratio, and rh is the vertical scaling ratio. Based on the scaling ratio and the bandwidth adjustment parameter beta, the weight matrix K corresponding to the first convolution window is determined. ij This includes: determining the pixel S within the first convolutional window. mn The corresponding horizontal weight L n L n It is based on X n Definitely, X n =rw*(n–C j / beta; Determines the pixel S within the first convolution window. mn The corresponding vertical weight L m L m It is based on X m Definitely, X m =rh*(m–C i / beta; Determines the pixel S within the first convolution window. mn The corresponding weight K mn =L n *L m Specifically, please refer to the horizontal weight L in step S103. n Vertical weight L m and weight K mn Related descriptions.

[0208] In some embodiments, L nEqual to the number of pixels S within the first convolution window mn L(X) n The function is the normalized value of L(*) relative to the L(*) function of all pixels in the same pixel row; where L(*) is the Lanczos kernel function, L(X) is the normalized value of L(X) relative to the L(*) function of all pixels in the same pixel row. n ) = sinc(X n )*sinc(X n / Gh);L m Equal to the number of pixels S within the first convolution window mn L(X) m The function is the normalized value of the L(*) function relative to all pixels in its column; where L(X) is the normalized value of the L(*) function. m ) = sinc(X m )*sinc(X m / Gw). Specifically, please refer to the horizontal weight L in step S103. n and vertical weight L m Related descriptions.

[0209] In some embodiments, the method further includes: if the electronic device obtains a preset value for the first parameter, then determining that the first parameter is a preset value; if the electronic device does not obtain a preset value for the first parameter, then determining the first parameter according to the horizontal scaling ratio and / or the vertical scaling ratio; the first parameter is used to generate a target image corresponding to the first image. The first parameter may be parameter 1 in the aforementioned embodiments.

[0210] In some embodiments, the method further includes: the electronic device acquiring a first parameter of the first image from another device, the first parameter being determined based on a horizontal scaling ratio and / or a vertical scaling ratio; the first parameter being used to generate a target image corresponding to the first image.

[0211] In some embodiments, the first parameter is the sharpening intensity.

[0212] In some embodiments, the first parameter is any one of the following: kernel radius, kernel weight matrix, and bandwidth adjustment parameter beta.

[0213] The various embodiments of this application can be combined arbitrarily to achieve different technical effects.

[0214] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0215] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

[0216] In summary, the above description is merely an embodiment of the technical solution of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made based on the disclosure of this application should be included within the scope of protection of this application.

Claims

1. An image processing method applied to electronic devices, characterized in that, The method includes: A first image at a first resolution is scaled to obtain an intermediate image at a second resolution, where the second resolution is smaller than the first resolution. The intermediate image is sharpened based on the sharpening intensity to obtain a target image with the second resolution; the sharpening intensity is determined according to a first scaling ratio of the target image relative to the first image, and the first scaling ratio is determined based on the second resolution and the first resolution.

2. The method according to claim 1, characterized in that, When the first scaling ratio is a first ratio, the sharpening intensity is a first value; when the first scaling ratio is a second ratio, the sharpening intensity is a second value; the second ratio is less than the first ratio, and the second value is greater than the first value.

3. The method according to claim 2, characterized in that, When the first scaling ratio is the third scaling ratio, the sharpening intensity is the first value; the third scaling ratio is less than the first scaling ratio.

4. The method according to claim 1, characterized in that, The first scaling ratio is determined based on the horizontal scaling ratio and the vertical scaling ratio; the horizontal scaling ratio is equal to the width of the second resolution divided by the width of the first resolution, and the vertical scaling ratio is equal to the height of the second resolution divided by the height of the first resolution.

5. The method according to claim 4, characterized in that, The scaling process of the first image at the first resolution to obtain an intermediate image at the second resolution includes: Based on the kernel radius and the kernel weight matrix, the first image at the first resolution is scaled to obtain an intermediate image at the second resolution; the kernel radius and / or the weight matrix are determined based on the horizontal scaling ratio and the vertical scaling ratio.

6. The method according to claim 5, characterized in that, The width of the convolution kernel radius increases as the horizontal scaling ratio decreases, and the height of the convolution kernel radius increases as the vertical scaling ratio decreases.

7. The method according to claim 6, characterized in that, The width of the convolution kernel radius is proportional to the reciprocal of the horizontal scaling ratio, and the height of the convolution kernel radius is proportional to the reciprocal of the vertical scaling ratio.

8. The method according to claim 5, characterized in that, The low-pass bandwidth corresponding to the weight matrix of the convolution kernel decreases as the first scaling factor decreases.

9. The method according to claim 5, characterized in that, The weight matrix of the convolution kernel is associated with the bandwidth adjustment parameter beta; the low-pass bandwidth corresponding to the weight matrix of the convolution kernel decreases as beta increases.

10. The method according to claim 5, characterized in that, The first resolution is sw*sh, and the second resolution is dw*dh. Based on the kernel radius and the kernel weight matrix, the first image at the first resolution is scaled to obtain an intermediate image at the second resolution, including: Determine the pixel P of the intermediate image ij 'The convolution center C in the original image ij coordinates (C) i C j ), P ij ' represents the pixel in the i-th row and j-th column of the intermediate image, where the value of i ranges from [0, dh-1] and the value of j ranges from [0, dw-1]. Based on the convolution center C ij The convolution center C is determined by the convolution kernel radius. ij The pixel matrix S corresponding to the first convolution window ij ; Based on the scaling ratio and bandwidth adjustment parameter beta, the weight matrix K corresponding to the first convolutional window is determined. ij ; For the pixel matrix S ij With the weight matrix K ij Perform convolution operation to obtain the pixel P of the intermediate image. ij The pixel value of '.

11. The method according to claim 10, characterized in that, Pixel S mn Let m be the pixel in the m-th row and n-th column within the first convolutional window, where m can take values ​​ranging from 1 to 2. The range of values ​​for n is symbol This indicates rounding down by *, where Gw is the width of the convolution kernel radius, Gh is the height of the convolution kernel radius, rw is the horizontal scaling ratio, and rh is the vertical scaling ratio. The weight matrix K corresponding to the first convolutional window is determined based on the scaling ratio and the bandwidth adjustment parameter beta. ij ,include: Determine the pixel S within the first convolution window mn The corresponding horizontal weight L n The L n It is based on X n Definitely, X n =rw*(n–C j ) / beta; Determine the pixel S within the first convolution window mn The corresponding vertical weight L m The L m It is based on X m Definitely, X m =rh*(m–C i ) / beta; Determine the pixel S within the first convolution window mn The corresponding weight K mn =L n *L m .

12. The method according to claim 11, characterized in that, The L n Equal to the number of pixels S within the first convolution window mn L(X) n The function is the normalized value of L(*) relative to the L(*) function of all pixels in the same pixel row; where L(*) is the Lanczos kernel function, L(X) is the normalized value of L(X) relative to the L(*) function of all pixels in the same pixel row. n ) = sinc(X n )*sinc(X n / Gh); The L m Equal to the number of pixels S within the first convolution window mn L(X) m The function is the normalized value of L(*) relative to the L(*) function of all pixels in its column; where L(X) m ) = sinc(X m )*sinc(X m / Gw).

13. The method according to any one of claims 2 to 12, characterized in that, The method further includes: if the electronic device obtains a preset value of the first parameter, then determining that the value of the first parameter is the preset value; if the electronic device does not obtain the preset value of the first parameter, then determining the first parameter according to the horizontal scaling ratio and / or the vertical scaling ratio; the first parameter is used to generate the target image corresponding to the first image.

14. The method according to any one of claims 2 to 12, characterized in that, The method further includes: the electronic device acquiring a first parameter of the first image from another device, the first parameter being determined based on the horizontal scaling ratio and / or the vertical scaling ratio; the first parameter being used to generate the target image corresponding to the first image.

15. The method according to claim 13 or 14, characterized in that, The first parameter is the sharpening intensity.

16. The method according to claim 13 or 14, characterized in that, The first parameter is any one of the following: kernel radius, kernel weight matrix, and bandwidth adjustment parameter beta.

17. An electronic device, characterized in that, include: A processor and a memory, the memory being coupled to the processor, the memory being used to store computer program code, the computer program code including computer instructions, which, when the processor reads from the memory, cause the electronic device to perform the image processing method as described in any one of claims 1 to 16.

18. A computer-readable storage medium, characterized in that, Includes computer instructions that, when executed on an electronic device, cause the electronic device to perform the image processing method as described in any one of claims 1 to 16.

19. A computer program product, characterized in that, When the computer program product is run on a computer, the computer causes the computer to perform the image processing method as described in any one of claims 1 to 16.

Images