Image processing method and related apparatus

Abstract

An image processing method and a related apparatus. The method comprises: acquiring a first image, the first image comprising a center pixel point and an edge pixel point; and performing blurring processing on the first image to obtain a second image. The second image comprises a first pixel point corresponding to the center pixel point and a second pixel point corresponding to the edge pixel point. Pixel points of the first image are all in the sampling range of the center pixel point. A pixel value of the first pixel point is obtained on the basis of a pixel value of a pixel point of the first image. A first position point located outside of the first image is present in the sampling range of the edge pixel point. A pixel value of the second pixel point is obtained on the basis of a pixel value of a pixel point of the first image and a pixel value of the first position point. The pixel value of the first position point is equal to a pixel value of a third pixel point in the first image. The third pixel value and the first position point are symmetrical relative to the edge pixel point. The described method avoids the problem of edge darkening after image blurring, and does not require an additional edge supplementing operation before performing image blurring, thereby effectively reducing the load.

Description

Image processing method and related device

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of China on February 5, 2024, with application number 202410166178.7, and priority to the Chinese patent application entitled “Image Processing Method and Related Devices”, all contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of terminal and computer technology, and in particular to image processing methods and related devices. Background Art

[0003] Image blurring is a common operation in image processing. Blurring an image can remove fine details. This allows it to be applied to various end-user scenarios, such as creating blur filters, reducing image noise, and performing lossy image compression. Summary of the Invention

[0004] The present application provides an image processing method and related devices, which can accelerate the speed of image blurring.

[0005] In a first aspect, an embodiment of the present application provides an image processing method, which is applied to an electronic device, and the method includes: acquiring a first image; the first image includes intermediate pixel points and edge pixel points, the edge pixel points are located at the edge of the first image, and the edge pixel points surround the intermediate pixel points; blurring the first image to obtain a second image; the second image includes a first pixel point corresponding to the intermediate pixel point, and a second pixel point corresponding to the edge pixel point; wherein, the sampling range of the intermediate pixel point is all pixel points of the first image, and the pixel value of the first pixel point is obtained according to the pixel value of the pixel point of the first image; within the sampling range of the edge pixel point, there is a first position point outside the first image, and the pixel value of the second pixel point is obtained according to the pixel value of the pixel point of the first image and the pixel value of the first position point; the pixel value of the first position point is equal to the pixel value of the third pixel point in the first image, and the third pixel point and the first position point are symmetrical relative to the edge pixel point.

[0006] When implementing the method provided in the first aspect, the electronic device does not need to fill in the edges of the image during the image blurring process. Instead, it finds the pixel values ​​of the position points outside the image in a symmetrical manner to ensure that the image edges do not darken after the image is blurred. At the same time, since there is no need to fill in the edges of the image, the drawing process is reduced, the speed of image blurring is accelerated, the load is effectively reduced, and the performance of the device is improved.

[0007] In combination with the first aspect, in one implementation, the pixel value of the first pixel point is equal to the mean or weighted average of the pixel values ​​of the pixel points within the sampling range of the middle pixel point; the pixel value of the second pixel point is equal to the pixel value of the pixel points within the sampling range of the edge pixel point, and the mean or weighted average of the pixel values ​​of the first position point.

[0008] Among them, if the pixel value of the first pixel point or the second pixel point adopts the average of multiple pixel values, it means that the algorithm used when blurring the first image is the average blur algorithm; if the pixel value of the first pixel point or the second pixel point adopts the weighted average of multiple pixel values, it means that the algorithm used when blurring the first image is the Gaussian blur algorithm.

[0009] That is to say, the method provided in the embodiment of the present application can be applied to the average blur algorithm or the Gaussian blur algorithm to improve the effect and speed of electronic devices when blurring images using the average blur algorithm or the Gaussian blur algorithm.

[0010] In addition, it can be understood that the method provided in the embodiment of the present application can also be applied to other blur algorithms. The sampling ranges adopted by different blur algorithms and the calculation methods of the pixel values ​​after blurring of the pixels may be different, but the image processing method provided in the embodiment of the present application can be used to determine the pixel values ​​of the position points outside the image within the sampling range. The embodiment of the present application does not limit the blur algorithm.

[0011] In combination with the first aspect, in one implementation, blurring the first image to obtain the second image specifically includes: blurring the first image based on a blur radius to obtain the second image; wherein the sampling range of the pixel point refers to the range of the blur radius centered on the pixel point.

[0012] The blur radius is used to indicate the degree of blurring when blurring an image. That is, the electronic device can determine the sampling range of pixel points based on the blur radius.

[0013] In combination with the first aspect, in one implementation, blurring the first image to obtain the second image specifically includes: blurring the first image horizontally to obtain the third image; and blurring the third image vertically to obtain the second image.

[0014] For example, if the image processing method provided in the embodiment of the present application is applied to the Gaussian blur algorithm, the electronic device can implement Gaussian blur of the image by first blurring horizontally and then blurring vertically during the process of blurring the first image, thereby speeding up the blurring speed of the image.

[0015] In combination with the first aspect, in one implementation, the first position point is located in the extension direction of the row or column of the first image, and the third pixel point and the first position point are symmetrical with the column or row where the first pixel point is located as the symmetry axis.

[0016] In other words, electronic devices can use an axially symmetrical method to determine the pixel values ​​of points outside the image, restore the effect of blurred image edges as much as possible, reduce the difference in blurring effects between pixel points at the image edge and in the middle of the image, and make the overall visual effect of the blurred image more uniform and natural.

[0017] In combination with the first aspect, in one implementation, if the first position point is located in the extension direction of the row of the first image, the third pixel point and the first position point are symmetrical with the column where the edge pixel point is located as the symmetry axis; if the first position point is located in the extension direction of the column of the first image, the third pixel point and the first position point are symmetrical with the row where the edge pixel point is located as the symmetry axis.

[0018] That is to say, if the position point is located on the left or right side of the image, the pixel value of the position point can be found with a column in the image as the axis of symmetry. If the position point is located above or below the image, the pixel value of the position point can be found with a row of the image as the axis of symmetry.

[0019] In combination with the first aspect, in one implementation, the row or column of the third pixel point can be determined by the following formula: X=W-abs(abs(n)-W)

[0020] If the first position point is located in the extension direction of the row of the first image, n represents the column number of the first position point, W represents the maximum column number of the first image, X represents the column number of the third pixel point, and abs() represents the absolute value function; the row number of the third pixel point is equal to the row number of the first position point; if the first position point is located in the extension direction of the column of the first image, n represents the row number of the first position point, W represents the maximum row number of the first image, X represents the row number of the third pixel point, and abs() represents the absolute value function; the column number of the third pixel point is equal to the column number of the first position point.

[0021] That is to say, in the actual process of searching for the pixel value of the location point, the pixel value of the location point can be directly found according to the above formula, thereby speeding up the image processing.

[0022] In combination with the first aspect, in one implementation, the first position point is not located in the row and column extension directions of the first image, and the third pixel point and the first position point are symmetrical with the edge pixel point as the center point.

[0023] In other words, electronic devices can use a centrally symmetrical method to determine the pixel values ​​of points outside the image, restore the effect of blurred image edges as much as possible, reduce the difference in blurring effects between the image edges and the pixels in the middle of the image, and make the overall visual effect of the blurred image more uniform and natural.

[0024] In combination with the first aspect, in one implementation, the row and column of the third pixel can be determined by the following formula: X = W1-abs(abs(n)-W1), Y = W2-abs(abs(m)-W2)

[0025] Wherein, n represents the row number of the first position point, m represents the column number of the first position point, X represents the row number of the third pixel point, Y represents the column number of the third pixel point, W1 represents the maximum row number of the first image, W2 represents the maximum column number of the first image, and abs() represents the absolute value function.

[0026] That is to say, in the actual process of searching for the pixel value of the location point, the pixel value of the location point can be directly found according to the above formula, thereby speeding up the image processing.

[0027] In combination with the first aspect, in one implementation, obtaining a first image specifically includes: downsampling the third image to obtain the first image; blurring the first image based on the blur radius to obtain the second image, and the method also includes: upsampling the second image to obtain a fourth image, and the fourth image has the same resolution as the third image.

[0028] It can be seen that in the overall process of image blurring, in addition to blurring the image, it can also include upsampling and downsampling of the image. By upsampling and downsampling, the amount of calculation in the image blurring process is reduced and the speed of image blurring is accelerated.

[0029] In combination with the first aspect, in one implementation, the method further includes: displaying a fourth image.

[0030] That is, after processing the image, the electronic device can display the final blurred image to the user so that the user can understand the effect of the blurred image.

[0031] In combination with the first aspect, in one implementation, blurring the first image to obtain the second image specifically includes: blurring the first image using a graphics processor (GPU) to obtain the second image.

[0032] That is to say, the method provided in the embodiment of the present application can also be applied in the GPU. By making corresponding improvements to the GPU, a better image blurring effect can be achieved without filling the edges. There is no need to use other devices, such as the CPU, to perform image filling, thereby reducing the image drawing process and speeding up the image blurring.

[0033] In a second aspect, an embodiment of the present application provides an electronic device comprising a memory, one or more processors, and one or more programs; when the one or more processors execute the one or more programs, the electronic device implements the method described in the first aspect or any one of the implementation methods of the first aspect.

[0034] In a third aspect, an embodiment of the present application provides a computer-readable storage medium comprising instructions, which, when executed on an electronic device, causes the electronic device to execute the method described in the first aspect or any one of the implementations of the first aspect.

[0035] In a fourth aspect, an embodiment of the present application provides a chip, which is applied to an electronic device. The chip includes one or more processors, and the processor is used to call computer instructions to enable the electronic device to execute the method described in the above first aspect or any one of the implementation methods of the first aspect.

[0036] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when executed on a computer, enables the computer to execute the method described in the first aspect or any one of the implementations of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] FIG1 is a schematic diagram of the principle of blurring an image according to an embodiment of the present application;

[0038] FIG2 is a schematic diagram showing the principle of using an edge-filling fuzzy algorithm to fill in the edges of an image;

[0039] FIG3 is an overall flow chart of the edge filling fuzzy algorithm;

[0040] FIG4 is an overall flow chart of the image processing method provided in an embodiment of the present application;

[0041] Figures 5A-5B and 6A-6C are some user interfaces provided in embodiments of the present application;

[0042] FIG7 is a schematic diagram of a flow chart of an image processing method provided in an embodiment of the present application;

[0043] FIG8 is a schematic diagram of an original image provided in an embodiment of the present application;

[0044] FIG9 is a schematic diagram of another original image provided in an embodiment of the present application;

[0045] FIG10 is a schematic diagram showing the blurring principle of a pixel point in the middle of an image during Gaussian blurring of an original image according to an embodiment of the present application;

[0046] FIG11 is a function waveform diagram of a two-dimensional Gaussian function.

[0047] FIG12 is a schematic diagram showing the blurring principle of pixel points at the edge of an image during Gaussian blurring of an original image according to an embodiment of the present application;

[0048] FIG13 is a comparison diagram of the image blurring effects of the image processing method provided by an embodiment of the present application and the native Gaussian blurring method of SKIA;

[0049] FIG14 is a schematic diagram of the hardware structure of an electronic device 100 provided in an embodiment of the present application;

[0050] FIG15 is a block diagram of the software structure of the electronic device 100 provided in an embodiment of the present application. DETAILED DESCRIPTION

[0051] The following is a clear and detailed description of the technical solutions in the embodiments of the present application in conjunction with the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in the text is only a description of the association relationship between related objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0052] In the following, the terms "first" and "second" are used for descriptive purposes only and should not be understood to imply or suggest relative importance or implicitly indicate the number of the technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of this application, unless otherwise specified, "plurality" means two or more.

[0053] The term "user interface (UI)" in the following embodiments of this application refers to a medium interface for interaction and information exchange between an application or operating system and a user, which realizes the conversion between the internal form of information and the form acceptable to the user. The user interface is a source code written in a specific computer language such as Java and extensible markup language (XML). The interface source code is parsed and rendered on an electronic device and finally presented as content that the user can recognize. The commonly used form of user interface is graphical user interface (GUI), which refers to a user interface related to computer operations that is displayed in a graphical manner. It can be a visual interface element such as text, icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, etc. displayed on the display screen of an electronic device.

[0054] The principle of image blurring is: for each pixel in the image, you can sample the pixel, take the pixel values ​​of the pixels around the pixel, use these pixel values ​​to calculate the new pixel value of the pixel, and use the new pixel value as the pixel value after blurring.

[0055] In the embodiment of the present application, for the sake of convenience of description, the process of taking the pixel values ​​of the pixels surrounding the pixel point and using these pixel values ​​to calculate the new pixel value is called sampling of the pixel point, and the range of the selected surrounding pixels is the sampling range.

[0056] For example, the average blur algorithm calculates a new pixel value by taking the average of the surrounding pixel values. Another example is the Gaussian blur algorithm, which calculates a new pixel value by taking a weighted average of the surrounding pixel values.

[0057] In image blurring, no matter which blurring algorithm is used, the blurring effect of the image is achieved by reducing the gap between pixels.

[0058] However, if a pixel is located at the edge of an image, when sampling that pixel, the surrounding pixel values ​​taken will include the pixel values ​​of locations outside the image. Due to the lack of pixels outside the image, the electronic device will default the pixel value of that location to 0, that is, the color of that location will be black. As a result, when using the sampled surrounding pixel values ​​to calculate the pixel value after the image edge is blurred, the pixel value of 0 will cause the final calculated pixel value to be closer to 0, which will make the color of the image edge appear darker after blurring, resulting in a poor visual effect after the image is blurred.

[0059] For a better understanding, FIG1 shows a schematic diagram of the principle of blurring an image.

[0060] Assume that the image blur algorithm used is the average blur algorithm. As shown in FIG1 , the image 1 includes: pixel A and pixel B. Pixel A is located in the middle of the image 1, and pixel B is located at the edge of the image 1.

[0061] In Image 1, pixel A originally has a value of 2, and the values ​​of the pixels surrounding pixel A are all 1. After blurring Image 1, the blurred pixel value of pixel A is the average of the pixel values ​​of the pixels surrounding pixel A, which is (1+1+1+1+1+1+1+1) / 8=1.

[0062] In Image 1, pixel B originally has a value of 2, and the values ​​of the pixels surrounding pixel B are all 1. After blurring Image 1, the blurred pixel value of pixel B is the average of the pixel values ​​of the pixels surrounding pixel B, which is (1+1+1+1+1+0+0+0) / 8=0.625.

[0063] Comparing pixel A and pixel B, it can be seen that even though the pixel values ​​around pixel A and pixel B are the same, because pixel B is at the edge of image 1, the pixel value of pixel B after blurring is smaller, making the edge of image 1 appear darker after blurring.

[0064] It should be noted that, when sampling pixel points A and pixel point B shown in FIG1 , pixel values ​​within a range of no more than 1 pixel point away from the pixel points are selected for example, and the sampling range of the pixel points can also be a larger range.

[0065] To address the darkening of edges after image blurring, one edge-padding blurring algorithm adds two Drawcalls. These are the calls made by the central processor unit (CPU) to the graphics application programming interface (API). Each Drawcall can contain one or more rendering instructions, which the CPU processes and submits to the graphics processing unit (GPU). The GPU then performs rendering operations based on the rendering instructions. Before blurring the image, the CPU first uses one Drawcall to pad the left and right edges of the image, and then uses another Drawcall to pad the top and bottom edges of the image. The resulting images are then passed to the GPU for blurring.

[0066] FIG2 is a schematic diagram showing the principle of the edge-filling blurring algorithm for image edge-filling.

[0067] As shown in Figure 2, for an image with an image size of N*N (N represents the number of pixels), if the width and height of the image need to be increased by a length of M, when the left and right edges of the image are padded, M*N pixels need to be added to both the left and right sides of the image. Among them, the M*N pixels added on the left side of the image are the pixels obtained by stretching the M*N pixels on the left edge of the image by M pixels to the left. Similarly, the M*N pixels added on the right side of the image are the pixels obtained by stretching the M*N pixels on the right edge of the image by M pixels to the right. When filling the upper and lower edges of an image, it is necessary to add (N+2M)*M pixels on both the upper and lower sides of the image. The (N+2M)*M pixels added at the upper end of the image are obtained by stretching the (N+2M)*M pixels at the upper edge of the image upward by M pixels. Similarly, the (N+2M)*M pixels added at the lower end of the image are obtained by stretching the (N+2M)*M pixels at the lower edge of the image downward by M pixels.

[0068] As can be seen from Figure 2, the original pixels in the image are located in the white area, and the newly added pixels are located in the gray area.

[0069] In this way, when sampling pixels originally located at the edge of the image, the pixels sampled include the pixels in the image and the pixels obtained by stretching the pixels in the image, thus avoiding the problem of blurring the image using pixel values ​​outside the image.

[0070] However, this has certain defects: adding two new Drawcalls will increase the overall blurring duration, resulting in a decrease in performance. When multiple blurring is involved, the overall power consumption will be significantly increased. Moreover, the pixel values ​​of the blurred pixels at the edge of the image are not completely calculated based on the pixel values ​​around the pixel in the image. After stretching, the pixel's own weight increases, the blurring effect is poor, and the visual effect optimization is not obvious.

[0071] An embodiment of the present application provides an image processing method, in which an electronic device can obtain an original image and then blur the original image to obtain a target image, wherein the original image includes edge pixel points located at the edge of the image and intermediate pixel points located in the middle of the image, and the target image includes a first pixel point corresponding to the intermediate pixel point and a second pixel point corresponding to the edge pixel point.

[0072] Among them, the pixel value of the first pixel point can be obtained according to the pixel value of the pixel point of the first image; the pixel value of the second pixel point can be obtained according to the pixel value of the pixel point of the first image, and the pixel value of the first position point within the sampling range of the edge pixel point, the pixel value of the first position point is equal to the pixel value of the third pixel point in the original image, and the third pixel point and the first position point are symmetrical relative to the first pixel point.

[0073] Among them, the middle pixel point located in the middle of the image means that the sampling range of the pixel point is all the pixel points in the original image. Correspondingly, the edge pixel point located at the edge of the image means that there is a position point outside the original image within the sampling range of the pixel point, such as the first position point.

[0074] It can be seen that the image processing method provided by the embodiment of the present application, in the process of blurring the image, adopts different pixel value selection methods according to whether the sampled pixel point is a pixel point at the edge of the image. If the sampled pixel point is a pixel point in the middle of the image, the pixel value selected for the pixel point includes the pixel values ​​around it. If the sampled pixel point is a pixel point at the edge of the image, then when selecting the pixel values ​​around the pixel point, for the pixel values ​​of the position points outside the image included in the sampling range, the pixel values ​​of the pixel points in the image that are symmetrical to the position point are used, avoiding the use of pixel values ​​outside the image that are 0 to calculate the pixel values ​​after blurring, thereby solving the problem of darkening the edges of the image after blurring and improving the blurring effect of the image. In addition, compared with the above-mentioned edge-filling blurring algorithm, the image processing method provided by the embodiment of the present application does not need to add a Drawcall to fill the edge of the image, reducing the drawing process. It only needs to adjust the pixel values ​​selected during the sampling process, effectively reducing the load and improving the performance of the device.

[0075] In order to better understand the difference between the image processing method provided in the embodiment of the present application and the above-mentioned edge-filling blur algorithm, Figures 3 and 4 show the difference between the image processing method provided in the embodiment of the present application and the edge-filling blur algorithm.

[0076] FIG3 shows the overall flow chart of the edge filling fuzzy algorithm.

[0077] As shown in Figure 3, blurring an image using the edge-padding algorithm requires five steps: downsampling, left and right padding, top and bottom padding, blurring, and upsampling to obtain the final blurred image. The blurring step in the edge-padding algorithm involves sampling the pixels in the padded image. The sampling rules for pixels in the center of the image are the same as those for pixels at the edges: the values ​​of the pixels surrounding the pixel are selected.

[0078] FIG4 shows an overall flow chart of the image processing method provided in an embodiment of the present application.

[0079] As shown in Figure 4, when blurring an image using the image processing method provided in an embodiment of the present application, it is necessary to go through the following three steps: downsampling, blurring, and upsampling, and finally obtain a blurred image. Among them, the blurring step in the image processing method includes sampling the pixel points in the image, and the sampling rules for the pixel points in the middle of the image are different from those for the pixel points at the edge of the image. When sampling the pixel points in the middle of the image, the pixel values ​​selected are the pixel values ​​around the pixel points, and when sampling the pixel points at the edge of the image, the pixel values ​​selected include: the pixel values ​​around the pixel point in the image, and the pixel values ​​of the position points outside the image and within the sampling range mapped to the pixel point after the image is symmetrically mapped with the pixel point as the reference point.

[0080] By comparing FIG3 and FIG4 , it can be seen that the image processing method provided in the embodiment of the present application does not include the left and right edge padding and top and bottom edge padding steps in the edge padding blur algorithm, and the blurring step is different from the steps in the edge padding blur algorithm.

[0081] In some embodiments, if the blur algorithm is a Gaussian blur algorithm, the blur steps shown in Figures 3 and 4 above may include two sub-steps: horizontal blur and vertical blur. The horizontal blur is used to blur each pixel in the image horizontally, and the vertical blur is used to blur each pixel in the image vertically. For example, the horizontal blur can be performed on the image first, and then the vertical blur can be performed on the image after the horizontal blur. Alternatively, the vertical blur can be performed on the image first, and then the horizontal blur can be performed on the image after the vertical blur. The embodiment of the present application does not limit the order in which these two sub-steps are performed.

[0082] Among them, both horizontal blur and vertical blur include sampling pixel points in the image. The image processing method provided in the embodiment of the present application can use a symmetrical mapping method to determine the pixel value of a position point outside the image when sampling pixel points.

[0083] It's important to note that the upsampling and downsampling shown in Figures 3 and 4 correspond to each other. Downsampling, also known as downsampling, is used to reduce the image size and lower its resolution, while upsampling, also known as upsampling, is used to enlarge the image and increase its resolution. By using both upsampling and downsampling, the computational effort in the blurring step can be reduced, speeding up the image blurring process.

[0084] The following describes the application scenarios of the image processing method provided in the embodiments of the present application.

[0085] 1) Provide users with image blur function

[0086] When users edit pictures, there are usually multiple picture editing requirements. The image processing method provided by the embodiment of the present application can provide users with an image blurring function to meet the user's needs for blurring pictures.

[0087] 5A-5B show relevant user interfaces of the electronic device 100 for blurring an image based on user operations.

[0088] Figure 5A illustrates an image editing interface 10 provided in an embodiment of the present application. As shown in Figure 5A , the image editing interface 10 may include an image display area 101 and a menu bar 102. The image display area 101 is used to display images, and the menu bar 102 may include options corresponding to one or more image editing functions. The menu bar 102 may include a blur option 102A, which can be used to blur an image.

[0089] When the electronic device 100 detects a user operation on the blur option 102A, in response to the operation, the electronic device 100 uses the image processing method provided in the embodiment of the present application to blur the image displayed in the image display area 101 and displays the image editing interface 10 shown in Figure 5B.

[0090] As shown in FIG5B , the picture displayed in the picture display area 101 is a blurred picture.

[0091] 2) Blur the currently displayed UI when opening the Notification Center

[0092] In order to create a more beautiful and vivid visual effect when opening the notification center, the current display screen can be gradually blurred by the image processing method provided in the embodiment of the present application as the screen corresponding to the notification center gradually cuts into the screen, so that the user's line of sight gradually transitions to the notification center as the user opens the notification center, avoiding abrupt screen switching.

[0093] 6A-6C illustrate user interfaces related to opening a notification center on the electronic device 100 .

[0094] Figure 6A shows a user interface 20 for the settings application. As shown in Figure 6A, if the electronic device 100 detects a user operation, such as a downward swipe from the top of the display screen, while displaying the user interface 20, the notification center is opened in response to the operation and the user interfaces shown in Figures 6B-6C are displayed.

[0095] As shown in Figures 6B-6C, the electronic device 100 displays a gradually blurred user interface 20 following the user's operation, and displays a user interface 30 where the notification center is located, which gradually appears following the user's operation. The gradually blurred user interface 20 can be an image obtained by blurring the user interface 20 using the image processing algorithm provided in the embodiment of the present application by the electronic device 100.

[0096] It is understandable that the image processing method provided in the embodiment of the present application may also have other application scenarios, such as blurring the background when opening a multi-tasking window or performing a global search, etc., which are not listed here one by one.

[0097] It can be seen from the above application scenarios that the image processing algorithm provided in the embodiments of the present application can be applied to various aspects of image processing on terminal electronic devices, providing users with richer visual effects and enabling users to view natural and uniform image blur effects.

[0098] FIG7 is a flow chart of an image processing method provided in an embodiment of the present application.

[0099] As shown in FIG7 , the image processing method provided in the embodiment of the present application may include the following steps:

[0100] S101. The electronic device 100 obtains an original image.

[0101] The original image may be a screenshot captured by the electronic device 100 or a picture selected by the user. The embodiment of the present application does not limit the source of the original image.

[0102] Exemplarily, the original image may be the image displayed in the image display area 101 shown in FIG. 5A , or may be a screenshot of the user interface 20 shown in FIG. 6A .

[0103] In some embodiments, the electronic device 100 may trigger blurring of the image upon detecting a user operation, i.e., starting steps S101-S107. For example, the user operation may be a user operation on the blur option 102A as shown in FIG5A, or a sliding operation as shown in FIG6A.

[0104] S102. The electronic device 100 traverses each pixel in the original image.

[0105] The principle of image blurring is to select the pixels around each pixel in the image to calculate its blurred pixel value, thereby blurring the image.

[0106] Therefore, when the electronic device 100 blurs the original image, it can traverse each pixel point in the original image, calculate the blurred pixel value point by point, and use these blurred pixel values ​​to obtain the blurred target image.

[0107] S103. The electronic device 100 determines whether the target pixel is located at the edge of the original image.

[0108] The target pixel can be any pixel in the original image.

[0109] In order to avoid the pixel values ​​of the pixels at the edge of the image from becoming darker after being blurred, the electronic device 100 can distinguish between the pixels located in the middle of the image and the pixels located at the edge of the image during the process of traversing the pixels, and use different sampling rules to determine the pixel values ​​selected during the sampling process.

[0110] During the sampling process, the pixel values ​​selected for pixels located in the center of the original image are all pixel values ​​of the pixels in the image, while the pixel values ​​selected for pixels located at the edge of the original image may include pixel values ​​of locations outside the image. The electronic device 100 may determine the pixel values ​​located at the edge of the original image based on the blur radius of the image.

[0111] The blur radius, also known as the blur range, is a parameter defined when blurring an image. It indicates the size of the sampling range when selecting pixel values ​​around a pixel. This can be the distance between the sampled pixel and the farthest pixel within the sampling range. A larger blur radius results in a more pronounced image blur, while a smaller blur radius results in a less pronounced image blur.

[0112] For example, if the blur radius is 1, then when selecting pixel values ​​around a pixel, the sampling range forms a rectangular area centered on the pixel and extending outward by one pixel. The rectangular area is centered on the pixel, and both its width and height are 3. For example, see the sampling ranges of pixel A and pixel B shown in Figure 1. If the blur radius is 3, then when selecting pixel values ​​around a pixel, the sampling range forms a rectangular area centered on the pixel and extending outward by three pixels. The rectangular area is centered on the pixel, and both its width and height are 5.

[0113] The blur radius may be a parameter set by the electronic device 100 or a parameter set by the user. The embodiment of the present application does not limit the source of the blur radius.

[0114] Among them, if the target pixel point is a pixel point in the middle of the original image, then the sampling range of the target pixel point is all the pixel points in the original image. In other words, the blur radius centered on the target pixel point is all the pixel points in the original image. If the target pixel point is a pixel point at the edge of the original image, then the sampling range of the target pixel point contains position points outside the original image. In other words, the blur radius centered on the target pixel point contains position points outside the original image.

[0115] It can also be seen that the pixels at the edge of the original image can be the pixels in the outermost r rows and r columns of the original image, where r is the blur radius. For example, if the blur radius is 1, the pixels at the edge of the original image can be the pixels in the outermost row and column of the original image. For another example, if the blur radius is 3, the pixels at the edge of the original image can be the pixels in the outermost 3 rows and 3 columns of the original image.

[0116] That is, the electronic device 100 can determine whether the target pixel point is located at the edge of the original image by determining whether the target pixel point belongs to the outermost r rows and r columns of pixels of the original image.

[0117] If the target pixel point does not belong to the outermost r rows and r columns of the original image, then the target pixel point is not located at the edge of the original image, and step S104 is executed. If the target pixel point belongs to the outermost r rows and r columns of the original image, then the target pixel point is located at the edge of the original image, and step S105 is executed.

[0118] In each embodiment, r rows and r columns of pixels are used as examples of pixel points at the edge of the original image. It should be understood that the r rows and r columns of pixels do not constitute a limitation to the embodiments of the present application.

[0119] It can be understood that step S102 and step S103 are optional steps. In the actual process of image blurring, the electronic device 100 does not need to traverse every pixel in the image, but directly finds the pixel points belonging to the edge of the image and the pixel points belonging to the middle of the image in the original image, and determines the sampling input values ​​of the pixel points at the edge of the image and the sampling input values ​​of the pixel points in the middle of the image respectively, and then calculates the pixel values ​​of the pixel points at the edge of the image after blurring, and the pixel values ​​of the pixel points in the middle of the image after blurring, to obtain the target image.

[0120] S104. The electronic device 100 determines that the sampled input value of the target pixel point includes: the pixel values ​​of the pixel points within the sampling range of the target pixel point in the original image.

[0121] If the target pixel is not located at the edge of the original image, the target pixel can be sampled directly according to the sampling range corresponding to the blur radius, and the pixel values ​​around the target pixel are selected to determine the sampling input value of the target pixel.

[0122] The pixels within the sampling range of the target pixel in the original image may be pixels within a blur radius centered on the target pixel, including pixels that are no more than r pixels away from the target pixel, where r is the blur radius.

[0123] That is to say, if the target pixel point is the middle pixel point of the original image, the sampling input value of the target pixel point is the pixel value of the pixel point in the original image, and the electronic device 100 can determine the blurred pixel value of the target pixel point based on the pixel value of the pixel point in the original image.

[0124] In some implementations, the pixel points within the sampling range of the target pixel point in the original image may further include: the target pixel point.

[0125] S105. The electronic device 100 determines that the sampling input value of the target pixel point includes: the pixel points within the sampling range of the target pixel point and the pixel value of the position point, and the pixel value of the position point is equal to the pixel value of the pixel point in the original image that is symmetrical to the position point relative to the target pixel point.

[0126] If the target pixel is located at the edge of the original image, the pixel value within the sampling range cannot be directly selected as the sampling input value of the target pixel.

[0127] When the target pixel is located at the edge of the original image, the sampling range of the target pixel, that is, the blur radius centered on the target pixel, contains two pixel values: the pixel value of the pixel inside the original image, and the pixel value of the position outside the original image. Therefore, for the pixel values ​​inside and outside the original image, the sampling input values ​​can have the following two situations:

[0128] 1) Pixels in the sampling range that are within the original image

[0129] For the pixel points in the sampling range that are located in the original image, the pixel values ​​of these pixel points can be directly used as the sampling input values ​​corresponding to the pixel points in the original image.

[0130] Among them, the pixel points located in the original image in the sampling range can be the pixel points within the sampling range of the target pixel point in the original image, that is, the pixel points within the blur radius centered on the target pixel point, including: pixel points that are no more than r pixels away from the target pixel point, where r is the blur radius.

[0131] In some implementations, the pixel points within the sampling range of the target pixel point in the original image may further include: the target pixel point.

[0132] 2) Points outside the original image in the sampling range

[0133] The position point may be a position point outside the original image that exists within a blur radius centered on the target pixel point.

[0134] If the sampling range includes a position point outside the original image, the pixel values ​​in the original image can be symmetrically mapped to the target pixel point as the reference point in a mirror sampling manner, so that the sampling input value corresponding to the position point outside the original image in the sampling range no longer uses the pixel value corresponding to black (i.e., 0), but uses the pixel value in the original image padded by the mirror sampling method.

[0135] Among them, there are two situations for the position points outside the original image:

[0136] a) The position point is located in the extension direction of the row or column of the original image

[0137] The position point is located in the extension direction of the row or column of the original image, which means that although the position point is located outside the original image, the position point is located on the left, right, above or below the original image.

[0138] FIG8 is a schematic diagram of an original image provided by an embodiment of the present application. The original image may be composed of edge pixels and intermediate pixels, where the edge pixels are located at the edge of the original image and the intermediate pixels are located in the center of the original image. As can be seen from FIG8 , edge pixels are pixels in the gray area of ​​the original image, and intermediate pixels are pixels in the white area of ​​the original image, where the edge pixels surround the intermediate pixels.

[0139] As shown in FIG. 8 , the position point located in the extension direction of the row or column of the original image may be a position point located in region_1.

[0140] If the position point is located in the extension direction of the row or column of the original image, the pixel point symmetrical to the position point in the original image may be the pixel point symmetrical to the position point with the row or column where the target pixel point is located as the central axis.

[0141] Specifically, the electronic device 100 may search for a pixel point symmetrical to the position in the original image according to the following formula 1:

[0142] If the position point is located in the extension direction of the row of the original image, X represents the column number of the pixel point in the original image, Y represents the row number of the pixel point in the original image, W represents the maximum column number of the original image, n represents the column number of the position point outside the original image, m represents the row number of the position point outside the original image, and abs() represents the absolute value.

[0143] If the position point is located in the extension direction of the column of the original image, X represents the row number of the pixel point in the original image, Y represents the column number of the pixel point in the original image, W represents the maximum row number in the original image, n represents the row number of the position point outside the original image, m represents the column number of the position point outside the original image, and abs() represents the absolute value.

[0144] To better understand the above formula, FIG9 shows a schematic diagram of another original image provided in an embodiment of the present application.

[0145] As shown in (a) and (b) of Figure 9, the leftmost column of the original image is the starting column, the column number of the pixels in this column is 0, and it increases to the right and decreases to the left. The top row of the original image is the starting row, the row number of the pixels in this row is 0, and it increases downward and decreases upward.

[0146] For example, if the blur radius used when blurring the original image is 1, then the pixels located in the outermost row and column of the original image are edge pixels, and the remaining pixels are intermediate pixels. Referring to Figure 9(a), assuming a blur radius of 1, the pixels in columns 0 and 8, and rows 0 and 7 in the original image are edge pixels (see the gray area in Figure 9(a)). The remaining pixels in columns 1-7 and rows 1-6 are intermediate pixels (see the white area in Figure 9(a)).

[0147] In addition, referring to (a) in FIG9 , if the pixel points in the current sampling range include a position point C outside the original image, and it is known that the width of the original image is 9, the maximum column number of the original image is 8, and the column number of the position point C is 9 and the row number is 3, then according to formula 1, the pixel point D symmetrical to it in the original image can be found, and the column number of the pixel point D is X=8-abs(abs(9)-8)=7, and the row number is Y=3.

[0148] For another example, if the blur radius used when blurring the original image is 2, then the pixels located in the outermost 2 rows and 2 columns of the original image are edge pixels, and the remaining pixels are intermediate pixels. Referring to Figure 9(b), assuming a blur radius of 2, the pixels in columns 0-1 and 8-9, and rows 0-1 and 4-5 in the original image are edge pixels (see the gray area in Figure 9(b)). The remaining pixels in columns 2-7 and rows 2-3 are intermediate pixels (see the white area in Figure 9(b)).

[0149] In addition, referring to (b) in FIG9 , if the pixel points in the current sampling range include a position point E outside the original image, and the height of the original image is known to be 5, the maximum row number of the original image is 6, and the row number of the position point E is -2, and the column number is 1. Then, according to formula 1, the pixel point F symmetrical to it in the original image can be found. The row number of the pixel point F is X=6-abs(abs(-2)-6)=2, and the column number is Y=1.

[0150] b) The location point is not located in the extension direction of the original image rows and columns

[0151] The position point is not located in the extension direction of the rows and columns of the original image, which means that the position point is not only located outside the original image, but also can be located on the left and right and up and down of the original image, but is located on the diagonal line of the original image.

[0152] As shown in FIG8 , the position point that is not located in the extension direction of the rows and columns of the original image may be a position point in region_2.

[0153] If the position point is not located in the extension direction of the rows and columns of the original image, the pixel points symmetrical to the position point in the original image may be pixel points symmetrical to the position point with the target pixel point as the center point.

[0154] Specifically, the electronic device 100 may search for a pixel point symmetrical to the position in the original image according to the following formula 2:

[0155] Where n represents the row number of the position point outside the original image, m represents the column number of the position point outside the original image, X represents the row number of the pixel point in the original image, Y represents the column number of the pixel point in the original image, W1 represents the maximum row number of the original image, W2 represents the maximum column number of the original image, and abs() represents the absolute value.

[0156] It can be understood that if the blur algorithm used by the electronic device 100 is a Gaussian blur algorithm, the blur process can be divided into horizontal blur and vertical blur, and the pixel value of the position point that is not located in the extension direction of the rows and columns of the original image can also be determined according to the above formula 1.

[0157] This is because if the electronic device 100 first performs horizontal blurring and then performs vertical blurring, the electronic device 100 can determine the pixel values ​​of the position points located in the extension direction of the original image rows, that is, the pixel values ​​of the position points located on the left and right sides of the original image, in a horizontally symmetrical manner during the horizontal blurring of the original image, thereby obtaining an intermediate image. Thereafter, when the intermediate image is vertically blurred, the pixel values ​​of the position points located in the extension direction of the original image columns and the pixel values ​​of the position points not located in the extension direction of the original image rows and columns can be determined in a vertically symmetrical manner, thereby obtaining the target icon. Among them, the pixel values ​​of the position points not located in the extension direction of the original image rows and columns are the pixel values ​​obtained by performing vertical symmetry on the pixel values ​​of the position points located in the extension direction of the original image rows.

[0158] To better understand the above principle, assume that the pixels in the original image and the position points around the image are arranged as follows:

[0159] Pixels 5, 6, 8, and 9 are pixels in the original image, while positions 1, 2, 3, 4, and 7 are outside the image. Positions 4 and 7 lie along the rows of the original image, while positions 2 and 3 lie along the columns of the image. Position 1 is not located in either row or column direction.

[0160] When the original image is horizontally blurred, the symmetrical point of position point 4 can be determined as pixel point 6, and the symmetrical point of position point 7 can be determined as pixel point 9 by means of horizontal symmetry. Therefore, the pixel value of position point 4 is the pixel value of pixel point 6, and the pixel value of position point 7 is the pixel value of pixel point 9.

[0161] When performing vertical blurring, the symmetrical point of position point 2 can be determined as pixel point 8, the symmetrical point of position point 3 as pixel point 9, and the symmetrical point of position point 1 as position point 7 through a vertical symmetric method. Therefore, the pixel value of position point 2 is the pixel value of pixel point 8, the pixel value of position point 3 is the pixel value of pixel point 9, and the pixel value of position point 1 is the pixel value of position point 7, that is, the pixel value of pixel point 9 determined during horizontal blurring.

[0162] That is to say, if the target pixel point is an edge pixel point of the original image, the sampling input value of the target pixel point includes not only the pixel value of the pixel point in the original image, but also the pixel value of the position point outside the original image. The electronic device 100 can determine the blurred pixel value of the target pixel point based on the pixel value of the pixel point in the original image and the pixel value of the position point outside the original image.

[0163] S106. The electronic device 100 calculates the blurred pixel value of the target pixel based on the sampled input value of the target pixel.

[0164] After obtaining the sampled input values ​​of the target pixel, these sampled input values ​​can be used to calculate the pixel value of the target pixel after blurring. Different blur algorithms use different calculation methods, but all use these sampled input values ​​to reduce the difference between the pixel value of the target pixel and the pixel values ​​of its surrounding pixels.

[0165] Exemplarily, if the blur algorithm is an average blur algorithm, the average of the sampled input values ​​may be determined as the pixel value of the target pixel after blurring.

[0166] For example, if the blur algorithm is a Gaussian blur algorithm, the pixel value of the blurred target pixel can be determined by taking a weighted average of the sampled input values. The weighted average can be achieved by constructing a weight matrix, which can also be called a sampler or a convolution kernel.

[0167] The working principle of the Gaussian blur algorithm is described in detail below with reference to Figures 10 to 12.

[0168] FIG10 is a schematic diagram showing the blurring principle of the pixel point in the middle of the image during the Gaussian blurring of the original image.

[0169] In FIG10 , the rectangular image on the left is the original image, the rectangular image on the right is the target image, and the rectangle in the middle is the weight matrix. Each square in the weight matrix has a weight value. Specifically, the weight values ​​in the weight matrix can be distributed according to the function waveform diagram of the two-dimensional Gaussian distribution function shown in FIG11 . As can be seen from FIG11 , the closer to the center point of the matrix, the larger the weight value, and the farther away from the center point of the matrix, the smaller the weight value.

[0170] As shown in Figure 10, the dark sampling range shown in the original image is the sampling range when sampling one of the target pixels in the original image. All pixel values ​​contained in this sampling range are the sampling input values ​​mentioned in step S104. By multiplying the pixel value of each pixel in the sampling range by the weight value in the corresponding square of the weight matrix and summing all the multiplied values, the blurred pixel value of the target pixel in the target image can be obtained.

[0171] For example, suppose the pixel values ​​in the sample range are arranged as follows:

[0172] The weights in the weight matrix are arranged as follows:

[0173] The pixel values ​​in the sampling range are multiplied by the corresponding weights in the weighted mean, and the output is as follows:

[0174] Finally, the sum of the values ​​obtained after the above multiplication is: 1.132638+1.77477+1.51587+2.83963+3.69403+3.07627+3.22121+4.14113+3.4107=25.00026. The obtained value 25.00026 is the pixel value of the target pixel after blurring.

[0175] Among them, the width and height of the weight matrix are equal to 2r+1, where r is the blur radius. In Figure 10, a blur radius of 1 is used as an example. It can be understood that the above Figure 10 and the exemplified data are only for understanding the implementation principle of the solution and do not constitute a limitation on the embodiments of the present application. In addition, Figure 10 only shows the blurring principle for one of the middle pixels in the original image. The blurring principle for other middle pixels in the original image is similar and will not be repeated here. Among them, the middle pixel of the original image can be the small square marked with a black dot as shown in Figure 10.

[0176] In addition, it should be noted that if the image is a color image, it is necessary to calculate the blurred pixel values ​​of the image under the three color channels respectively.

[0177] In addition, FIG12 shows a schematic diagram of the blurring principle of pixel points at the edge of an image during the Gaussian blurring of the original image.

[0178] Among them, the rectangular image shown on the left side of Figure 12 is the original image, the rectangular image shown on the right side is the target image, and the matrix in the middle is the weight matrix. This weight matrix is ​​the same as the weight matrix shown in Figure 10, and will not be repeated here.

[0179] As shown in Figure 12(a), the dark areas and the diagonal texture areas in the original image together constitute the sampling range. It can be seen that part of the sampling range is outside the original image. Therefore, before determining the sampling input value, the mirror padding process shown in Figure 12(b) is required.

[0180] As shown in FIG12( b ), the pixel value of a position outside the original image in the sampling range can be determined by mirroring and filling.

[0181] As shown in FIG12(c), after pixel padding, all pixel values ​​within the sampling range are the sampled input values ​​mentioned in step S105. By multiplying each pixel value within the sampling range by the weight value in the corresponding square of the weight matrix and summing all the multiplied values, the blurred pixel value of the target pixel in the target image can be obtained.

[0182] The specific calculation process of the blurred pixel value can be found in the above-mentioned related content and will not be repeated here. In addition, it can be understood that Figure 12 only illustrates the blurring principle for one edge pixel in the original image. The blurring principle for other edge pixels in the original image is similar and will not be repeated here. The edge pixels of the original image can be the small squares marked with black dots as shown in Figure 12.

[0183] It should be noted that the above weight matrix can be split as follows:

[0184] Therefore, in the actual process of performing Gaussian blur, the above two-dimensional weight matrix can be split into two one-dimensional weight matrices 0.3078012 0.384397 0.3078012 and First, the original image is horizontally blurred by the weight matrix 0.3078012 0.384397 0.3078012, and then the weight matrix Blur the horizontally blurred image vertically, or first use the weight matrix The original image is blurred vertically, and then the image after vertical blur is blurred horizontally through the weight matrix to obtain the target image.

[0185] That is to say, for each pixel in the original image, the electronic device 100 needs to execute steps S102-S106 twice, the first time for performing horizontal blur or vertical blur to obtain an intermediate image, and the second time for performing vertical blur or horizontal blur on the intermediate image to obtain a target image.

[0186] For example, it is taken as an example that horizontal blurring is performed first and vertical blurring is performed later.

[0187] During the first execution of steps S102 to S106 , the electronic device 100 may traverse each pixel point in the original image, calculate the pixel value after horizontal blurring, and obtain an intermediate image.

[0188] Specifically, the electronic device 100 can traverse each pixel point in the original image and sample the pixel point using matrix 1 with a weight matrix of 1*(2r+1). In addition, in the process of sampling the target pixel point, if the sampling range includes a position point outside the original image, the position point should be located in the extension direction of the row of the original image. The electronic device 100 can determine the pixel value of the position point in a horizontally symmetrical manner. In this way, the electronic device 100 can calculate the pixel value of the target pixel point in the original image after horizontal blurring, and the pixel value is the pixel value of the pixel point corresponding to the target pixel point in the intermediate image, thereby obtaining the intermediate image.

[0189] In the process of executing steps S102 to S106 again, the electronic device 100 may obtain the target image by traversing each pixel point in the intermediate image and calculating the pixel value after longitudinal blurring.

[0190] Specifically, the electronic device 100 can traverse each pixel point in the intermediate image and sample the pixel point using matrix 2 with a weight matrix of (2r+1)*1. In the process of sampling the target pixel point, if the sampling range includes a position point outside the intermediate image, the position point is located in the extension direction of the column of the intermediate image. In this way, the electronic device 100 can determine the pixel value of the position point in a longitudinally symmetrical manner. In this way, the electronic device 100 can calculate the pixel value of the target pixel point in the intermediate image after longitudinal blurring, and this pixel value is the pixel value of the pixel point corresponding to the target pixel point in the target image, thereby obtaining the target image.

[0191] S107. The electronic device 100 obtains a target image based on the blurred pixel value of each pixel in the original image.

[0192] The pixels in the target image correspond one-to-one with the pixels in the original image. The pixel value of one of the target pixels is the pixel value of the corresponding pixel in the original image after blurring.

[0193] That is to say, by traversing each pixel in the original image and executing the above steps S103-S106 for each pixel respectively, the pixel value of each pixel in the original image after blurring can be calculated, thereby forming the target image and blurring the original image.

[0194] After the electronic device 100 acquires the target image, it may display the target image and present the blurred image to the user.

[0195] Exemplarily, the target image may be the image displayed in the image display area 101 shown in FIG. 5B , or may be a blurred screenshot of the user interface 20 shown in FIG. 6B .

[0196] It should be noted that the original image acquired by the electronic device 100 may be a downsampled image. In this case, after the electronic device 100 obtains the target image, it is necessary to upsample the target image to restore it to its original resolution. Therefore, the image ultimately displayed by the electronic device 100 may be the upsampled image of the target image.

[0197] In an embodiment of the present application, the original image may also be referred to as the first image, the pixel points located at the edge of the original image may also be referred to as edge pixel points, the pixel points located in the middle of the image may also be referred to as intermediate pixel points, the target image may also be referred to as the second image, the pixel point corresponding to the intermediate pixel point in the target image may also be referred to as the first pixel point, the pixel point corresponding to the edge pixel point in the target image may also be referred to as the second pixel point, the position point outside the first image within the sampling range of the edge pixel point may also be referred to as the first position point, the pixel point in the first image that is symmetrical to the first position point may also be referred to as the third pixel point, the intermediate image may also be referred to as the third image, and the pixel point corresponding to the first pixel point in the third image may also be referred to as the fourth pixel point.

[0198] In general, the method provided in the embodiment of the present application can avoid the darkening of the edges of the blurred image, so that the blurred image can achieve a better visual effect. In addition, the method can map pixels during the sampling process, avoiding the need to fill in the edges of the image, reducing power consumption, reducing task load, and eliminating the need to add unnecessary processes, thereby improving performance.

[0199] Figure 13 shows a comparison of the image blurring effects of the image processing method provided by the present embodiment and SKIA's native Gaussian blur method. Figure 13 (a) shows the image blurring effect of the image processing method provided by the present embodiment, while Figure 13 (b) shows the image blurring effect of SKIA's native Gaussian blur method.

[0200] By comparing (a) and (b) in FIG13 , it can be seen that after blurring the image, the image processing method provided in the embodiment of the present application does not present black edges at the edges of the image, and the visual effect is more uniform and natural.

[0201] Table 1 shows the difference in running time between the image processing method provided in the embodiment of the present application and the native Gaussian blur method of SKIA during the image blurring process.

[0202] Table 1

[0203] As can be seen from Table 1, image blurring requires the participation of modules such as CPU load, GPUDriver, and GPU hardware. Compared with the Gaussian blur method, the image processing method provided in the embodiment of the present application has an optimized running time of 104 μs on the CPU load, an optimized running time of 56.51 μs on the GPUDriver, and an optimized running time of 94.44 μs on the GPU hardware. The total optimized running time is 254.95 μs, which significantly shortens the time spent on image blurring, reduces power consumption, and improves device performance.

[0204] FIG14 shows a schematic diagram of the hardware structure of the electronic device 100 .

[0205] The electronic device 100 may be a mobile phone, a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a cellular phone, a personal digital assistant (PDA), an augmented reality (AR) device, a virtual reality (VR) device, an artificial intelligence (AI) device, a wearable device, an in-vehicle device, a smart home device, and / or a smart city device. The embodiments of the present application do not impose any particular restrictions on the specific type of the electronic device.

[0206] The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0207] It should be understood that the structure illustrated in the embodiments of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0208] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0209] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0210] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0211] In some embodiments, the processor 110 may be used to obtain an original image and blur the original image based on a blur radius to obtain a target image. Specifically, the processor 110 may sample each pixel in the original image and calculate the pixel value of the pixel corresponding to the pixel in the target image based on the pixel value within the sampling range of the pixel. In the process of blurring the original image, the processor 110 may also determine whether the sampled pixel is a pixel at the edge of the original image. If the pixel is a pixel at the edge of the original image, then when determining the pixel value of a position point within the sampling range that is outside the original image, the pixel value may be determined as the pixel value of a pixel point in the original image that is symmetrical to the position point.

[0212] The target image can be obtained by blurring the original image based on the blur radius using the GPU in the processor 110. For example, the image processing method provided in the embodiment of the present application can be stored in a shader file in the form of an algorithm.

[0213] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0214] Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In other embodiments, the antennas can be used in conjunction with a tuning switch.

[0215] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0216] The modem processor may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to the speaker 170A, the receiver 170B, etc.) or displays an image or video through the display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0217] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices that integrate at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, demodulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0218] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with a network and other devices through wireless communication technologies. The wireless communication technologies may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0219] Electronic device 100 implements display functionality through a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0220] The display screen 194 is used to display images, videos, etc. In some embodiments, the electronic device may include 1 or N display screens 194 , where N is a positive integer greater than 1.

[0221] In some embodiments, the display screen 194 can be used to display an image before blurring, such as an original image, and an image after blurring, such as a target image, etc. For details, please refer to Figures 5A-5B and 6A-6C above.

[0222] The electronic device 100 can implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.

[0223] The ISP processes data fed back by camera 193. For example, when taking a photo, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, which is then passed to the ISP for processing and converted into a visible image. The ISP can also perform algorithmic optimization on image noise and brightness. It can also optimize parameters such as exposure and color temperature of the captured scene. In some embodiments, the ISP can be located within camera 193.

[0224] The camera 193 is used to capture still images or videos.

[0225] The internal memory 121 may include one or more random access memories (RAM) and one or more non-volatile memories (NVM).

[0226] The random access memory can be directly read and written by the processor 110, and can be used to store executable programs (such as machine instructions) of the operating system or other running programs, and can also be used to store user and application data.

[0227] The non-volatile memory may also store executable programs and user and application data, etc., and may be loaded into the random access memory in advance for direct reading and writing by the processor 110 .

[0228] In some embodiments, the internal memory 121 can be used to store an image before blurring, such as an original image, and an image after blurring, such as a target image, as well as sampling input values ​​obtained in the process of sampling pixel points in the original image, and blurred pixel values ​​calculated based on the sampling input values, etc.

[0229] The electronic device 100 can implement audio functions such as music playback and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0230] The touch sensor 180K is also called a "touch-sensitive device." The touch sensor 180K can be disposed on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen." The touch sensor 180K is used to detect touch operations applied thereto or in the vicinity thereof. The touch sensor can transmit the detected touch operations to the application processor to determine the type of touch event. Visual output related to the touch operations can be provided via the display screen 194. In other embodiments, the touch sensor 180K can also be disposed on the surface of the electronic device 100, at a location different from that of the display screen 194.

[0231] The electronic device can be a portable terminal device equipped with Harmony, iOS, Android, Microsoft or other operating systems, such as a mobile phone, tablet computer, wearable device, etc., and can also be a non-portable terminal device such as a laptop computer with a touch-sensitive surface or touch panel, a desktop computer with a touch-sensitive surface or touch panel. The software system of the electronic device 100 can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a microservice architecture, or a cloud architecture. The embodiment of the present invention takes the Android system with a layered architecture as an example to illustrate the software structure of the electronic device 100.

[0232] FIG15 is a block diagram of the software structure of the electronic device 100 according to an embodiment of the present application.

[0233] A layered architecture divides software into several layers, each with distinct roles and responsibilities. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers: the application layer, the application framework layer, the Android runtime and system libraries, and the kernel layer.

[0234] The application layer can include a series of application packages.

[0235] As shown in FIG15 , the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and short message.

[0236] The application framework layer provides an application programming interface (API) and programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0237] As shown in FIG15 , the application framework layer may include a window manager, a content provider, a view system, a telephony manager, a resource manager, a notification manager, and the like.

[0238] The window manager is used to manage window programs. The window manager can obtain the display size, determine whether there is a status bar, lock the screen, take screenshots, etc.

[0239] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0240] The view system includes visual controls, such as those for displaying text and images. The view system is used to build applications. A display interface can consist of one or more views. For example, a display interface containing a text notification icon might include a view for displaying text and a view for displaying images.

[0241] The phone manager is used to provide communication functions of the electronic device 100, such as management of call status (including answering, hanging up, etc.).

[0242] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0243] The Notification Manager allows applications to display notifications in the status bar. These messages can be displayed briefly and then disappear automatically without user interaction. For example, the Notification Manager is used to notify users of completed downloads and message reminders. The Notification Manager can also display notifications in the top status bar of the system as icons or scrolling text, such as notifications from background applications, or as dialog windows on the screen. Examples include text messages in the status bar, beeps, vibrations on electronic devices, and flashing indicator lights.

[0244] Android Runtime includes core libraries and a virtual machine. Android runtime is responsible for scheduling and management of the Android system.

[0245] The core library consists of two parts: one is the function that needs to be called by the Java language, and the other is the Android core library.

[0246] The application layer and application framework layer run in a virtual machine. The virtual machine executes Java files in the application layer and application framework layer as binary files. The virtual machine manages object lifecycles, stack management, thread management, security and exception management, and garbage collection.

[0247] The system library can include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0248] The surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications.

[0249] The media library supports playback and recording of a variety of common audio and video formats, as well as static image files. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0250] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0251] A 2D graphics engine is a drawing engine for 2D drawings.

[0252] The kernel layer is the layer between hardware and software. The kernel layer includes at least display driver, camera driver, audio driver, and sensor driver.

[0253] The following describes the workflow of the software and hardware of the electronic device 100 in conjunction with capturing a photo scene.

[0254] When the touch sensor 180K receives a touch operation, the corresponding hardware interrupt is sent to the kernel layer. The kernel layer processes the touch operation into a raw input event (including touch coordinates, touch operation timestamp, and other information). The raw input event is stored in the kernel layer. The application framework layer obtains the raw input event from the kernel layer and identifies the control corresponding to the input event. For example, if the touch operation is a touch single-click operation and the control corresponding to the single-click operation is the control of the camera application icon, the camera application calls the interface of the application framework layer to start the camera application, and then starts the camera driver by calling the kernel layer to capture a still image or video through the camera 193.

[0255] It should be understood that each step in the above method embodiment can be completed by hardware integrated logic circuits in a processor or by software instructions. The method steps disclosed in the embodiments of the present application can be directly embodied as being executed by a hardware processor, or by a combination of hardware and software modules in a processor.

[0256] The present application also provides an electronic device, which may include a memory and a processor. The memory may be used to store a computer program, and the processor may be used to call the computer program in the memory so that the electronic device executes the method executed by the electronic device 100 in any of the above embodiments.

[0257] The present application provides a chip, which is applied to an electronic device. The chip may include one or more processors, and the processors may be used to call a computer to execute so that the electronic device executes the method executed by the electronic device 100 in any of the above embodiments.

[0258] The present application also provides a chip system, which includes at least one processor for implementing the functions involved in the method performed by the electronic device 100 in any of the above embodiments.

[0259] In one possible design, the chip system further includes a memory, which is used to store program instructions and data, and the memory is located inside or outside the processor.

[0260] The chip system can be composed of chips, or can include chips and other discrete devices.

[0261] Optionally, there may be one or more processors in the chip system. The processor may be implemented in hardware or software. When implemented in hardware, the processor may be a logic circuit, an integrated circuit, etc. When implemented in software, the processor may be a general-purpose processor implemented by reading software code stored in a memory.

[0262] Optionally, the memory in the chip system may be one or more. The memory may be integrated with the processor or may be provided separately from the processor, which is not limited in the embodiments of the present application. For example, the memory may be a non-transient processor, such as a read-only memory (ROM), which may be integrated with the processor on the same chip or provided on different chips. The embodiments of the present application do not specifically limit the type of memory or the configuration of the memory and the processor.

[0263] Exemplarily, the chip system may be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a microcontroller unit (MCU), a programmable logic device (PLD) or other integrated chips.

[0264] The present application also provides a computer program product, which includes: a computer program (also referred to as code, or instructions), which, when executed, enables a computer to execute any method executed by the electronic device 100 in any of the above embodiments.

[0265] The present application also provides a computer-readable storage medium storing a computer program (also referred to as code or instruction). When the computer program is executed, the computer executes the method executed by any one of the electronic devices 100 in any of the above embodiments.

[0266] It should be understood that the processor in the embodiments of the present application can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above-mentioned method embodiment can be completed by hardware integrated logic circuits in the processor or by software instructions. The above-mentioned processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The various methods, steps, and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of the present application can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above-mentioned method.

[0267] In addition, embodiments of the present application further provide a device. Specifically, the device may be a component or module, and may include one or more processors and a memory connected to each other. The memory is configured to store a computer program. When the computer program is executed by one or more processors, the device performs the methods described in each of the above method embodiments.

[0268] The apparatus, computer-readable storage medium, computer program product, or chip provided in the embodiments of the present application are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects of the corresponding methods provided above, and will not be repeated here.

[0269] The various implementation modes of this application can be combined arbitrarily to achieve different technical effects.

[0270] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments may be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described herein are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated therein. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0271] Those skilled in the art will appreciate that all or part of the process steps in the above-described method embodiments can be implemented by a computer program instructing the relevant hardware. The program can be stored in a computer-readable storage medium, and when executed, the program can include the process steps in the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

[0272] In short, the above description is only an embodiment of the technical solution of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent replacements, improvements, etc. made based on the disclosure of the present invention should be included in the scope of protection of the present invention.

Claims

1. An image processing method, characterized in that: The method is applied to an electronic device, and includes: Acquire a first image; the first image includes middle pixels and edge pixels, the edge pixels are located at the edge of the first image, and the edge pixels surround the middle pixels; Performing blur processing on the first image to obtain a second image; the second image includes a first pixel corresponding to the middle pixel and a second pixel corresponding to the edge pixel; The sampling range of the intermediate pixels is all pixel points of the first image, and the pixel values of the first pixels are obtained according to the pixel values of the pixels of the first image; There is a first position point outside the first image within the sampling range of the edge pixel point, and the pixel value of the second pixel point is obtained based on the pixel value of the pixel point of the first image and the pixel value of the first position point; the pixel value of the first position point is equal to the pixel value of the third pixel point in the first image, and the third pixel point and the first position point are symmetrical relative to the edge pixel point.

2. The method according to claim 1, characterized in that The pixel value of the first pixel point is equal to the mean or weighted average of the pixel values of the pixel points within the sampling range of the middle pixel point; The pixel value of the second pixel point is equal to the pixel values of the pixel points within the sampling range of the edge pixel point, and the mean or weighted average of the pixel values of the first position point.

3. The method according to claim 1 or 2, characterized in that Performing blur processing on the first image to obtain a second image specifically includes: blurring the first image based on a blur radius to obtain a second image; The sampling range of a pixel point refers to the range of the blur radius centered on the pixel point.

4. The method according to any one of claims 1 to 3, characterized in that Performing blur processing on the first image to obtain a second image specifically includes: Performing horizontal blur processing on the first image to obtain a third image; Performing longitudinal blur processing on the third image to obtain a second image.

5. The method according to any one of claims 1 to 4, characterized in that The first position point is located in the extension direction of the row or column of the first image, and the third pixel point and the first position point are symmetrical with respect to the column or row where the first pixel point is located as the symmetry axis.

6. The method according to claim 5, characterized in that If the first position point is located in the extension direction of the row of the first image, the third pixel point and the first position point are symmetrical with the column where the edge pixel point is located as the symmetry axis; If the first position point is located in the extension direction of the column of the first image, the third pixel point and the first position point are symmetrical with respect to the row where the edge pixel point is located as the axis of symmetry.

7. The method according to claim 6, characterized in that The row or column of the third pixel is determined by the following formula: X = W-abs(abs(n)-W) If the first point is located in the extension direction of a row of the first image, n represents the column number of the first point, W represents the maximum column number of the first image, X represents the column number of the third pixel point, and abs() represents the absolute value function; the row number of the third pixel point is equal to the row number of the first point; If the first position point is located in the extension direction of the column of the first image, n represents the row number of the first position point, W represents the maximum row number of the first image, X represents the row number of the third pixel point, and abs() represents the absolute value function; the column number of the third pixel point is equal to the column number of the first position point.

8. The method according to any one of claims 1 to 7, characterized in that The first position point is not located in the row and column extension directions of the first image, and the third pixel point and the first position point are symmetrical with the edge pixel point as the center point.

9. The method according to claim 8, characterized in that The row and column of the third pixel are determined by the following formula: X = W1-abs(abs(n)-W1), Y = W2-abs(abs(m)-W2) Among them, n represents the row number of the first position point, m represents the column number of the first position point, X represents the row number of the third pixel point, Y represents the column number of the third pixel point, W1 represents the maximum row number of the first image, W2 represents the maximum column number of the first image, and abs() represents the absolute value function.

10. The method according to any one of claims 1 to 9, characterized in that Acquiring a first image specifically includes: downsampling the third image to obtain the first image; After blurring the first image based on the blur radius to obtain a second image, the method further includes: The second image is up-sampled to obtain a fourth image, where the fourth image has the same resolution as the third image.

11. The method according to claim 10, characterized in that The method further comprises: The fourth image is displayed.

12. The method according to any one of claims 1 to 11, characterized in that Performing blur processing on the first image to obtain a second image specifically includes: The first image is blurred by a graphics processing unit (GPU) to obtain a second image.

13. An electronic device, characterized in that: The electronic device comprises a memory, one or more processors, and one or more programs; when the one or more processors execute the one or more programs, the electronic device implements the method according to any one of claims 1 to 12.

14. A computer-readable storage medium comprising instructions, characterized in that: When the instructions are executed on an electronic device, the electronic device is caused to execute the method according to any one of claims 1 to 12.

15. A chip, characterized in that: The chip is applied to an electronic device, and the chip includes one or more processors, and the processor is used to call computer instructions to enable the electronic device to execute the method according to any one of claims 1 to 12.

Images