Image blurring method and device

Abstract

The present application is applicable to the technical field of image processing, and provides an image blurring method and device, wherein the image blurring method comprises: acquiring an image to be processed, and calling a central processing unit to perform a first operation task to obtain a sampled image of a preset scale and image processing data, wherein the first operation task comprises performing a pre-processing operation on the image to be processed. Calling a graphics processing unit to perform a second operation task to obtain a blurred image of the image to be processed, wherein the second operation task comprises blurring the sampled image according to the image processing data. Through the above scheme, the computing resources of the central processing unit and the graphics processing unit are reasonably allocated. The powerful parallel computing capability of the graphics processing unit is fully utilized to cooperate with the central processing unit to complete the blurring process, thereby improving the parallelism of the blurring process and thereby improving the efficiency of the blurring process.

Description

Technical Field

[0001] The present application belongs to the technical field of image processing, and in particular relates to an image blurring method and device. Background Art

[0002] As smartphones and other terminal devices develop, their functionality becomes increasingly richer, and performance requirements become increasingly demanding. Among these, image blurring, a must-have feature on mobile phones, is increasingly valued by users. However, blurring on mobile devices often requires post-processing with software.

[0003] However, in traditional blurring technology, due to the high complexity of blurring processing, the efficiency of traditional blurring processing is low. Summary of the Invention

[0004] In view of this, the embodiments of the present application provide an image blurring method and apparatus, which can solve the technical problem in traditional blurring technology that the traditional blurring processing efficiency is low due to the high complexity of the blurring processing.

[0005] A first aspect of an embodiment of the present application provides an image blurring method, the method comprising:

[0006] Acquire an image to be processed, and call a central processing unit to perform a first operation task to obtain a sampled image of a preset scale and image processing data, wherein the first operation task includes performing a preprocessing operation on the image to be processed;

[0007] A graphics processor is called to execute a second operation task to obtain a blurred image of the image to be processed, wherein the second operation task includes blurring the sampled image according to the image processing data.

[0008] A second aspect of an embodiment of the present application provides an image blurring device, comprising:

[0009] a first processing unit configured to acquire an image to be processed and call a central processing unit to execute a first operation task to obtain a sampled image of a preset scale and image processing data, wherein the first operation task includes performing a preprocessing operation on the image to be processed;

[0010] The second processing unit is configured to call a graphics processor to execute a second operation task to obtain a blurred image of the image to be processed, wherein the second operation task includes blurring the sampled image according to the image processing data.

[0011] A third aspect of an embodiment of the present application provides an image blurring system including a central processing unit and a graphics processing unit, wherein:

[0012] The central processing unit is used to perform a first operation task after acquiring the image to be processed, to obtain a sampled image of a preset scale and image processing data, wherein the first operation task includes performing a pre-processing operation on the image to be processed;

[0013] The graphics processor is used to perform a second operation task to obtain a blurred image of the image to be processed, and the second operation task includes blurring the sampled image according to the image processing data.

[0014] The fourth aspect of an embodiment of the present application provides a terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in the first aspect when executing the computer program.

[0015] A fifth aspect of an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of the method described in the first aspect are implemented.

[0016] Compared with the prior art, the beneficial effects of the embodiments of the present application are as follows: the present application obtains a sampled image of a preset scale and image processing data by acquiring an image to be processed and calling a central processing unit to perform a first operation task, wherein the first operation task includes performing a pre-processing operation on the image to be processed. The graphics processing unit is called to perform a second operation task to obtain a blurred image of the image to be processed, wherein the second operation task includes blurring the sampled image according to the image processing data. Through the above scheme, the computing resources of the central processing unit and the graphics processing unit are reasonably allocated. The powerful parallel computing capability of the graphics processing unit is fully utilized to cooperate with the central processing unit to complete the blurring process, thereby improving the parallelism of the blurring process and thereby improving the efficiency of the blurring process. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0018] Figure 1 A schematic diagram of an image blurring system provided by an embodiment of the present application is shown;

[0019] Figure 2 A schematic flow chart of an image blurring method provided by the present application is shown;

[0020] Figure 3A specific schematic flow chart of step 201 in an image blurring method provided by the present application is shown;

[0021] Figure 4 A specific schematic flow chart of step 202 in an image blurring method provided by the present application is shown;

[0022] Figure 5 A specific schematic flow chart of step 202 in an image blurring method provided by the present application is shown;

[0023] Figure 6 A schematic flow chart of another image blurring method provided by the present application is shown;

[0024] Figure 7 A schematic flow chart of another image blurring method provided by the present application is shown;

[0025] Figure 8 A schematic diagram showing the functional division of a processor in an image blurring method provided by the present application is shown;

[0026] Figure 9 The figure shows an image to be processed in an image blurring method provided by the present application;

[0027] Figure 10 shows a depth image in an image blurring method provided by the present application;

[0028] Figure 11 The image blurring method provided by the present application is shown;

[0029] Figure 12 A schematic diagram illustrating the cache life cycle of a graphics processor in an image blurring method provided by the present application is shown;

[0030] Figure 13 A schematic diagram of an image blurring device provided by the present application is shown;

[0031] Figure 14 A schematic diagram of a terminal device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0032] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail.

[0033] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or collections thereof.

[0034] It will also be understood that the term "and / or" used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0035] As used in this specification and the appended claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0036] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0037] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0038] Traditional blurring techniques often rely on a central processing unit (CPU) for blurring. However, the CPU isn't suited for computation, relying on a multi-instruction, single-dataflow architecture that excels at logic control. Therefore, this application uses a graphics processor to obtain multiple downsampled images of varying scales from the CPU, along with the blur boundary corresponding to each downsampled image.

[0039] In view of this, embodiments of the present application provide an image blurring method, apparatus, image blurring system, terminal device, and computer-readable storage medium.

[0040] See Figure 1 , Figure 1 A schematic diagram of an image blurring system provided in one embodiment of the present application is shown.

[0041] The image blurring system 1 includes a central processing unit (CPU) 11 and a graphics processing unit (GPU) 12. The CPU 11 is configured to, after acquiring an image to be processed, perform a first operation task to obtain a sampled image of a preset scale and image processing data. The first operation task includes performing a preprocessing operation on the image to be processed. The GPU 12 is configured to perform a second operation task to obtain a blurred image of the image to be processed. The second operation task includes blurring the sampled image based on the image processing data.

[0042] The computer program (i.e., executable file) running on the CPU 11 and the GPU 12 uses an operating framework such as OpenCL or CUDA, which is oriented to heterogeneous systems. In order to better explain the technical solution, this application uses OpenCL as an example for explanation.

[0043] Based on the above hardware environment and operating framework, this application provides an image blurring method, the execution subject of this method is the blurring terminal. Figure 2 , Figure 2 A schematic flow chart of an image blurring method provided in the present application is shown.

[0044] like Figure 2 As shown, the method may include the following steps:

[0045] Step 201 : Acquire an image to be processed, and call a central processing unit to execute a first operation task to obtain a sampled image of a preset scale and image processing data. The first operation task includes executing a pre-processing operation on the image to be processed.

[0046] Since the central processing unit is a multi-instruction single data flow architecture, it is better at logic control and is not suitable for large-scale computational processing. Therefore, this application assigns the first operation task with smaller computational complexity to the central processing unit for processing. The first operation task includes performing a preprocessing operation on the image to be processed. The preprocessing operation includes but is not limited to downsampling processing (compression processing) of the image to be processed and obtaining image processing data. The image processing data includes but is not limited to the blur boundary and blur radius of the sampled image.

[0047] Specifically, the image processing data includes but is not limited to the blur boundary and blur radius of the sampled image. The calling of the central processing unit to perform the first operation task to obtain the sampled image and image processing data of the preset scale includes the following steps, see Figure 3 , Figure 3A specific schematic flow chart of step 201 in an image blurring method provided in the present application is shown.

[0048] Step 2011: compress the image to be processed according to a preset scale by the central processing unit to obtain the sampled image of the preset scale.

[0049] Due to the large amount of data in the image to be processed, direct blurring of the image to be processed would result in low processing efficiency. Therefore, in this embodiment, the image to be processed is compressed by a central processing unit before blurring to obtain a sampled image. The compression scale and number of sampled images can be determined based on actual needs.

[0050] The sampled image is used for blurring processing. Compared with directly blurring the image to be processed, the blurring processing of the sampled image has less computational complexity.

[0051] Specifically, the sampled images of the preset scale include a first-scale sampled image and a second-scale sampled image, wherein the first scale is smaller than the second scale.

[0052] For example, in this embodiment, the scale of the sampling image is half that of the image to be processed and one-quarter that of the image to be processed. Different regions of the image are blurred using the sampling image at half the scale and the sampling image at one-quarter the scale, respectively. The sampling image at one-quarter the scale has a smaller amount of data and can therefore be used to blur the background region of the image. The sampling image at one-quarter the scale has a larger amount of data and can therefore be used to blur the transition region of the image (the transition region between the foreground and the background). It will be appreciated that if the image to be processed requires blurring of multiple regions, more or fewer sampling images can be used.

[0053] In step 2012, the central processing unit determines the blurring boundary and blurring radius of the sampled image according to the pixel depth values ​​of the depth image corresponding to the image to be processed.

[0054] A depth image, also known as a range image, is an image that uses the distance (depth) from the image collector to each point in the scene as its pixel value. Methods for acquiring depth images include, but are not limited to, LiDAR depth imaging, computer stereo vision, coordinate measuring machines, moiré fringe methods, and structured light methods.

[0055] The blurred boundary refers to the boundary corresponding to different blurred areas in the image to be processed. The calculation method of the blurred boundary includes but is not limited to the Grabcu algorithm, the LSC algorithm and the SLC algorithm. According to the depth values ​​of different pixels, different blurred areas are divided to obtain the blurred boundary. For example: the depth values ​​of different pixels in the sampled image are used as the basis for division, and different depth of field ranges are set (if the depth value distribution of the sampled image is represented by data from 0 to 1, the position closer to 0 indicates the distance to the foreground, and the position closer to 1 indicates the distance to the foreground, the depth range of the background area and the transition area can be determined between 0 and 1 according to actual calculation requirements). The depth values ​​of all pixels in the sampled image are traversed, and the set of pixels whose depth values ​​meet the depth range corresponding to the background area is taken as the background area. The set of pixels whose depth values ​​meet the depth range corresponding to the transition area is taken as the transition area. Among them, the blurred boundary is composed of the coordinates of multiple pixels.

[0056] Each pixel within a blurred area corresponds to a different blur radius. The blur radius refers to the degree to which a pixel is blurred. Pixels with higher depth values ​​have larger blur radii, i.e., a higher degree of blur. Different depth values ​​correspond to different blur radii. In this embodiment, different blur radii are preset for different depth values, so that different blurred areas are blurred to varying degrees.

[0057] Step 2013: Cache the first-scale sampling image, the second-scale sampling image, and the image processing data into a cache unit in the graphics processor.

[0058] As one embodiment of the present application, data transmission between the graphics processor and the central processing unit (CPU) requires CPU resources, which inadvertently increases CPU overhead. Furthermore, data transmission between the CPU and the graphics processor is relatively slow. For these reasons, in this embodiment, after obtaining the first-scale sampling image, the second-scale sampling image, and the image processing data, the CPU caches the first-scale sampling image, the second-scale sampling image, and the image processing data in a cache unit within the graphics processor.

[0059] Step 202: Calling a graphics processor to execute a second operation task to obtain a blurred image of the image to be processed. The second operation task includes blurring the sampled image according to the image processing data.

[0060] The defocusing terminal calls the graphics processor to obtain a sampled image of a preset scale and image processing data from the central processing unit. The graphics processor performs defocusing processing on the sampled image according to the image processing data to obtain a defocused image of the image to be processed.

[0061] The second operation task includes but is not limited to performing blurring, ray tracing, and edge smoothing on the sampled image according to the image processing data.

[0062] Specifically, the calling of the graphics processor to perform the second operation task to obtain the blurred image of the image to be processed includes the following steps, see Figure 4 , Figure 4 A specific schematic flow chart of step 202 in an image blurring method provided in the present application is shown.

[0063] Step 2021: Defocus the first scaled sampling image and the second scaled sampling image according to the image processing data by the graphics processor to obtain a first defocused image and a second defocused image.

[0064] The graphics processor reads the first-scale sampling image, the second-scale sampling image, and the image processing data from the cache unit, and blurs the first-scale sampling image and the second-scale sampling image according to the blur radius in the image processing data. The blurring process is as follows: the graphics processor traverses the first blur radius of each pixel in the first-scale sampling image. If the first blur radius is greater than or equal to a first threshold, the corresponding pixel in the first-scale sampling image is blurred. If the first blur radius is less than the threshold, no processing is performed. The graphics processor traverses the second blur radius of each pixel in the second-scale sampling image. If the second blur radius is greater than or equal to the second threshold, the corresponding pixel in the second-scale sampling image is blurred. If the second blur radius is less than the threshold, no processing is performed. In this way, the first and second blurred images are obtained.

[0065] The blurring method includes but is not limited to Gaussian blur or kernel functions in OpenCL. For example, the kernel function in OpenCL used in this example performs blurring on the first scale sampled image. The kernel function code is as follows:

[0066]

[0067]

[0068] Where bokehRadius represents the blur radius. If the bokehRadius corresponding to a pixel is greater than 5, filtering and blurring are performed; if bokehRadius is less than 5, no filtering and blurring are performed.

[0069] The blurring process for the second-scale sampled image is similar to the kernel function described above and will not be further described here. However, the kernel function for the second-scale sampled image has a bokehRadius value less than 5. When the bokehRadius corresponding to a pixel is less than 5, filtering blurring is performed; when the bokehRadius value is greater than 5, no processing is performed.

[0070] Because data transmission between the GPU and the CPU requires CPU resources, it inadvertently increases CPU overhead. Furthermore, data transmission between the CPU and the GPU is relatively slow. For these reasons, this embodiment stores processed data in the GPU's cache. Due to the limited cache space of the GPU, this embodiment caches the data required for interpolation processing in the video memory's image object, while other data is cached in the buffer object.

[0071] Step 2022: Cache the first defocused image and the second defocused image in the cache unit, and clear the first scaled sampling image and the second scaled sampling image in the cache unit.

[0072] Since the cache space in the graphics processor is limited, after the first blurred image and the second blurred image are cached in the cache unit, the first scaled sampling image and the second scaled sampling image in the cache unit need to be cleared to release the cache space.

[0073] Specifically, the calling of the graphics processor to perform the second operation task to obtain the blurred image of the image to be processed includes the following steps, see Figure 5 , Figure 5 A specific schematic flow chart of step 202 in an image blurring method provided in the present application is shown.

[0074] Step 202a: read the first blurred image and the second blurred image in the cache unit, and perform interpolation and fusion processing on the first blurred image and the second blurred image to obtain the blurred image of the image to be processed.

[0075] Since the first sampled image and the second sampled image have different scales, and the first blurred image is obtained by processing the first sampled image, and the second blurred image is obtained by processing the second sampled image, the scales of the first blurred image and the second blurred image are different. Therefore, in this embodiment, the second blurred image needs to be interpolated according to the scale of the first blurred image to obtain a third blurred image. For example, if the first blurred image is half the scale of the image to be processed, and the second blurred image is one-quarter the scale of the image to be processed, the second blurred image needs to be interpolated to obtain a third blurred image of half the scale.

[0076] As an embodiment of the present application, the first blurred image and the second blurred image may be interpolated simultaneously to obtain a first blurred image and a second blurred image having the same scale. The scales of the first blurred image and the second blurred image may be the same as the scale of the image to be processed, or any other scale, depending on the requirements of the actual application scenario.

[0077] After obtaining the third blurred image, the first blurred image and the third blurred image are fused using a kernel function in OpenCl to obtain a blurred image of the image to be processed.

[0078] The fusion process is as follows: obtain the first blurred boundary of the first blurred image and obtain the second blurred boundary of the third blurred image. Based on the first blurred boundary and the second blurred boundary, a fusion process is performed to obtain the blurred image of the image to be processed. The kernel function code of the fusion process in OpenCL is as follows:

[0079]

[0080]

[0081] Step 202b: Clear the first blurred image and the second blurred image in the cache unit, and cache the blurred images in the cache unit.

[0082] In this embodiment, a sampled image of a preset scale and image processing data are obtained by acquiring an image to be processed and invoking a central processing unit (CPU) to perform a first operation task, wherein the first operation task includes performing a preprocessing operation on the image to be processed. A second operation task is then invoked on a graphics processing unit (GPU) to obtain a blurred image of the image to be processed, wherein the second operation task includes blurring the sampled image based on the image processing data. This scheme rationally allocates the computing resources of the CPU and GPU. The powerful parallel computing capabilities of the GPU are fully utilized to coordinate the blurring process with the CPU, thereby increasing the parallelism of the blurring process and, in turn, improving the efficiency of the blurring process.

[0083] Optionally, in the above Figure 2 Based on the embodiment shown, the method further includes the following steps, see Figure 6 , Figure 6 FIG. 6 is a schematic flow chart of another image blurring method provided by the present application. In this embodiment, steps 601 to 602 are similar to Figure 2 In the embodiment shown, steps 201 to 202 are the same. For details, please refer to Figure 2 The descriptions of steps 201 to 202 in the illustrated embodiment are not repeated here.

[0084] Step 601: Acquire an image to be processed, and call a central processing unit to execute a first operation task to obtain a sampled image of a preset scale and image processing data. The first operation task includes executing a pre-processing operation on the image to be processed.

[0085] Step 602: Calling a graphics processor to execute a second operation task to obtain a blurred image of the image to be processed. The second operation task includes blurring the sampled image according to the image processing data.

[0086] Step 603: Perform edge smoothing and ray tracing processing on the blurred image of the image to be processed by a graphics processor to obtain a first fused image.

[0087] Because the blurred image to be processed is a fusion of the first and third blurred images, the blur effect near the fusion boundary between the first and third blurred images varies significantly. Therefore, edge smoothing and ray tracing are required to reduce the blur difference near the fusion boundary.

[0088] Step 604 : Using a graphics processor, interpolation processing is performed on the first fused image according to the size of the image to be processed to obtain a second fused image. The second fused image has the same size as the image to be processed.

[0089] Step 605: Using a graphics processor, fuse the second fused image with the foreground image of the image to be processed to obtain a target image.

[0090] In this embodiment, a graphics processor performs edge smoothing and ray tracing on the blurred image of the image to be processed to obtain a first fused image. The graphics processor also performs interpolation processing on the first fused image based on the size of the image to be processed to obtain a second fused image. The graphics processor also fuses the second fused image with the foreground image of the image to be processed to obtain a target image. Through this solution, the fused image is post-processed to make the blurred image appear more natural, thereby improving the blurring effect.

[0091] Optionally, in the above Figure 2 Based on the embodiment shown, before obtaining the image to be processed, the following steps are also included, see Figure 7 , Figure 7 FIG. 1 is a schematic flow chart of another image blurring method provided by the present application. In this embodiment, steps 702 and 703 are similar to Figure 2 In the embodiment shown, steps 201 to 202 are the same. For details, please refer to Figure 2 The descriptions of steps 201 to 202 in the illustrated embodiment are not repeated here.

[0092] Step 701: Store initialization information of an executable file as a static global variable, wherein the executable file includes a first operation task and a second operation task, and the initialization information includes device information, device operation context information, task queue information, and platform information.

[0093] The executable file is a program running in the OpenCl framework. Before the executable file is executed, the OpenCl framework needs to be initialized. The initialization parameters include but are not limited to device information, device running context information, task queue information, platform information, and compiled kernel functions. Since the compilation time of kernel functions is long, as the number of kernel functions increases with the increase of functions, this part of time consumption cannot be underestimated. Therefore, this example encapsulates all framework initialization operations into a class and stores the initialization information as a static global variable (the storage subject can be either the central processing unit or the graphics processing unit). This allows the initialization information to continue to be used after the virtual thread is killed.

[0094] Step 702: Acquire an image to be processed, and call a central processing unit to execute a first operation task to obtain a sampled image of a preset scale and image processing data. The first operation task includes executing a pre-processing operation on the image to be processed.

[0095] Step 703: Calling a graphics processor to execute a second operation task to obtain a blurred image of the image to be processed. The second operation task includes blurring the sampled image according to the image processing data.

[0096] In this embodiment, by storing the initialization information of the executable file as a static global variable, the executable file includes the first operation task and the second operation task. Through the above solution, the initialization information is reused without the need to repeatedly obtain the initialization information, thereby improving the efficiency of the virtualization process.

[0097] In order to better explain the technical solutions in the above embodiments, this application is based on Figure 8 The above embodiment is further explained. In this embodiment, the functions of the CPU and the GPU are divided as follows: Figure 8 See Figure 8 , Figure 8 FIG. 1 is a schematic diagram showing the functional division of a processor in an image blurring method provided by the present application. Figure 8As shown, this example assigns the initialization of the OpenCl framework and the first operation task (preprocessing operation) to the central processing unit, and the second operation task (blurring operation) to the graphics processing unit. The first operation task includes downsampling and obtaining image processing data. The second operation task includes blurring, image fusion, ray tracing, and edge smoothing. Specifically, the central processing unit and the graphics processing unit perform the following steps: obtain the image to be processed, and call the central processing unit to perform the first operation task to obtain a sampled image and image processing data of a preset scale. Call the graphics processing unit to perform the second operation task to obtain a blurred image of the image to be processed. The graphics processing unit performs edge smoothing and ray tracing on the blurred image of the image to be processed to obtain a first fused image. The graphics processing unit performs interpolation on the first fused image according to the size of the image to be processed to obtain a second fused image, which has the same size as the image to be processed. The graphics processing unit fuses the second fused image with the foreground image of the image to be processed to obtain a target image. Through the above-mentioned functional division method, the part with smaller computational complexity is assigned to the central processing unit. The parts with larger computational workload are distributed among the image processing processors, thus improving the parallelism of blurring processing.

[0098] In order to more intuitively demonstrate the technical effects of this application, Figures 9 to 11 Show the blurring result of this solution. Figure 9 As shown, Figure 9 FIG. 1 shows an image to be processed in an image blurring method provided by the present application. Figure 10 As shown, Figure 10 FIG. 1 shows a depth image in an image blurring method provided by the present application. Figure 11 As shown, Figure 11 The figure shows a blurred image in an image blurring method provided by the present application. It can be seen that the present application improves the efficiency of blurring processing while ensuring the blurring effect.

[0099] In order to better explain the technical solutions in the above embodiments, this application is based on Figure 12 The above embodiment is further explained. This example describes the cache life cycle of the graphics processor. Figure 12 See Figure 12 , Figure 12 A schematic diagram of the cache life cycle of a graphics processor in an image blurring method provided by this application is shown. Figure 12As shown, the graphics processor copies the first sample image and the blurred boundary of the first sample image in the central processing unit (host), and creates a cache for the first sample image and the blurred boundary of the first sample image respectively. The first sample image and the blurred boundary of the first sample image are read from the cache, and blurred processing is performed to obtain and cache the blurred result of the first sample image. The graphics processor copies the second sample image and the blurred boundary of the second sample image in the central processing unit, and creates a cache for the second sample image and the blurred boundary of the second sample image respectively. The cache of the first sample image and the blurred boundary of the first sample image is destroyed. The second sample image and the blurred boundary of the second sample image are read from the cache, and blurred processing is performed to obtain and cache the blurred result of the second sample image. The blurred result of the first sample image and the blurred result of the second sample image are read from the cache, and the blurred result of the first sample image and the blurred result of the second sample image are fused to obtain and cache the fused result. The cache of the blurred result of the first sample image and the blurred result of the second sample image is destroyed. Read the second sampling image, the blurred boundary of the second sampling image, and the fusion result from the cache, perform ray tracing processing based on the second sampling image, the blurred boundary of the second sampling image, and the fusion result, and obtain and cache the ray tracing result. Destroy the cache of the second sampling image and the blurred boundary of the second sampling image. Obtain the depth information from the central processing unit and create a cache of the depth information. Read the ray tracing result and the depth information from the cache, perform edge smoothing processing on the ray tracing result based on the depth information, and obtain and cache the edge smoothing processing result. Destroy the cache of the ray tracing result and the edge smoothing depth information. Obtain the image to be processed from the central processing unit and create a cache of the image to be processed. Read the image to be processed and the smoothing processing result from the cache, and fuse the image to be processed and the smoothing processing result to obtain the final blurred image. Transfer the final blurred image back to the central processing unit and destroy all caches in the graphics processing unit.

[0100] like Figure 13 This application provides an image blurring device 13, see Figure 13 , Figure 13 A schematic diagram of an image blurring device provided by the present application is shown in FIG. Figure 13 An image blurring device is shown, comprising:

[0101] The first processing unit 131 is configured to obtain an image to be processed and call a central processing unit to perform a first operation task to obtain a sampled image of a preset scale and image processing data, wherein the first operation task includes performing a pre-processing operation on the image to be processed;

[0102] The second processing unit 132 is configured to call a graphics processor to execute a second operation task to obtain a blurred image of the image to be processed. The second operation task includes blurring the sampled image according to the image processing data.

[0103] The present application provides an image blurring device, which obtains a sampled image of a preset scale and image processing data by acquiring an image to be processed and calling a central processing unit to perform a first operation task, wherein the first operation task includes performing a preprocessing operation on the image to be processed. The device then calls a graphics processing unit to perform a second operation task to obtain a blurred image of the image to be processed, wherein the second operation task includes blurring the sampled image based on the image processing data. Through the above scheme, the computing resources of the central processing unit and the graphics processing unit are reasonably allocated. The powerful parallel computing capability of the graphics processing unit is fully utilized to coordinate with the central processing unit to complete the blurring process, thereby improving the parallelism of the blurring process and thereby improving the efficiency of the blurring process.

[0104] Figure 14 FIG1 shows a schematic diagram of a terminal device provided by an embodiment of the present application. Figure 14 As shown, a terminal device 14 of this embodiment includes: a processor 140, a memory 141, and a computer program 142 stored in the memory 141 and executable on the processor 140, such as an image blurring program. When the processor 140 executes the computer program 142, the steps of each of the above-mentioned image blurring method embodiments are implemented, such as Figure 2 Alternatively, when the processor 140 executes the computer program 142, the functions of the units in the above-mentioned device embodiments are realized, for example, Figure 13 Units 131 to 132 are shown.

[0105] Exemplarily, the computer program 142 may be divided into one or more units, which are stored in the memory 141 and executed by the processor 140 to complete the present application. The one or more units may be a series of computer program instruction segments capable of performing specific functions, which are used to describe the execution process of the computer program 142 in the terminal device 14. For example, the computer program 142 may be divided into a first processing unit and a second processing unit, each of which has the following specific functions:

[0106] The first processing unit is used to obtain the image to be processed and call the central processing unit to perform a first operation task to obtain a sampled image of a preset scale and image processing data. The first operation task includes performing a preprocessing operation on the image to be processed.

[0107] The second processing unit is configured to call a graphics processor to execute a second operation task to obtain a blurred image of the image to be processed, wherein the second operation task includes blurring the sampled image according to the image processing data.

[0108] The terminal device 14 may be a computing device such as a mobile terminal, a desktop computer, a notebook, a PDA, or a cloud server. The terminal device may include, but is not limited to, a processor 140 and a memory 141. Those skilled in the art will understand that Figure 14 It is only an example of a terminal device 14 and does not constitute a limitation on the terminal device 14. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the terminal device may also include input and output devices, network access devices, buses, etc.

[0109] The processor 140 may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), graphics processing units (GPU), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0110] The memory 141 may be an internal storage unit of the terminal device 14, such as a hard disk or memory of the terminal device 14. The memory 141 may also be an external storage device of the terminal device 14, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device 14. Furthermore, the memory 141 may include both an internal storage unit of the terminal device 14 and an external storage device. The memory 141 is used to store the computer program and other programs and data required by the terminal device. The memory 141 may also be used to temporarily store data that has been output or is to be output.

[0111] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0112] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0113] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0114] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0115] In the embodiments provided in this application, it should be understood that the disclosed terminal devices and methods can be implemented in other ways. For example, the terminal device embodiments described above are merely illustrative. For example, the division of the modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0116] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0117] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0118] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by the processor, it can implement the steps of the above-mentioned various method embodiments. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device that can carry the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0119] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A method for blurring an image, characterized in that: The method comprises: Acquire an image to be processed and call a central processing unit to execute a first operation task to obtain a sampled image of a preset scale and image processing data, wherein the first operation task includes performing a preprocessing operation on the image to be processed; the sampled image of the preset scale includes a sampled image of a first scale and a sampled image of a second scale, wherein the first scale is smaller than the second scale; the sampled image of the first scale is used to blur the background area of ​​the image; and the sampled image of the second scale is used to blur the transition area between the foreground and the background; A graphics processor is called to execute a second operation task to obtain a blurred image of the image to be processed, wherein the second operation task includes blurring the sampled image according to the image processing data.

2. The image blurring method according to claim 1, wherein: The image processing data includes a blur boundary and a blur radius of the sampled image, and the calling of the central processing unit to perform the first operation task to obtain the sampled image and the image processing data of a preset scale includes: Compressing the image to be processed according to a preset scale by the central processing unit to obtain a sampled image of the preset scale; The central processing unit determines the blurring boundary and blurring radius of the sampled image according to the pixel depth value of the depth image corresponding to the image to be processed.

3. The image blurring method according to claim 1, wherein: After obtaining the sampled image and image processing data of the preset scale, the method further includes: The first scale-sampled image, the second scale-sampled image, and the image processing data are cached in a cache unit in the graphics processor.

4. The image blurring method according to claim 3, wherein: The calling of the graphics processor to perform the second operation task to obtain a blurred image of the image to be processed includes: Performing blurring processing on the first scale sampling image and the second scale sampling image according to the image processing data by the graphics processor to obtain a first blurring image and a second blurring image; The first defocused image and the second defocused image are cached in the cache unit, and the first scaled sampling image and the second scaled sampling image in the cache unit are cleared.

5. The image blurring method according to claim 4, wherein: The calling of the graphics processor to perform the second operation task to obtain a blurred image of the image to be processed includes: Reading the first blurred image and the second blurred image in the cache unit, and performing interpolation and fusion processing on the first blurred image and the second blurred image to obtain a blurred image of the image to be processed; The first and second blurred images in the cache unit are cleared, and the blurred images are cached in the cache unit.

6. The image blurring method according to any one of claims 1 to 5, wherein: The method further comprises: Performing edge smoothing and ray tracing processing on the blurred image of the image to be processed by a graphics processor to obtain a first fused image; performing interpolation processing on the first fused image according to the size of the image to be processed by a graphics processor to obtain a second fused image, wherein the second fused image has the same size as the image to be processed; The second fused image is fused with the foreground image of the image to be processed by a graphics processor to obtain a target image.

7. The image blurring method according to any one of claims 1 to 5, wherein: Before obtaining the image to be processed, the method further includes: Initialization information of an executable file is stored as a static global variable, the executable file includes a first operation task and a second operation task, and the initialization information includes device information, device operation context information, task queue information, and platform information.

8. An image blurring device, characterized in that: The image blurring device comprises: A first processing unit is configured to obtain an image to be processed and call a central processing unit to perform a first operation task to obtain a sampled image of a preset scale and image processing data, wherein the first operation task includes performing a preprocessing operation on the image to be processed; the sampled image of the preset scale includes a sampled image of a first scale and a sampled image of a second scale, wherein the first scale is smaller than the second scale; the sampled image of the first scale is used to blur the background area of ​​the image; and the sampled image of the second scale is used to blur the transition area between the foreground and the background; The second processing unit is configured to call a graphics processor to execute a second operation task to obtain a blurred image of the image to be processed, wherein the second operation task includes blurring the sampled image according to the image processing data.

9. An image blurring system, characterized in that: Including central processing unit, graphics processing unit, including: The central processing unit is configured to execute a first operation task after acquiring an image to be processed to obtain a sampled image of a preset scale and image processing data, wherein the first operation task includes performing a preprocessing operation on the image to be processed; the sampled image of the preset scale includes a sampled image of a first scale and a sampled image of a second scale, wherein the first scale is smaller than the second scale; the sampled image of the first scale is applied to blur the background area of ​​the image; and the sampled image of the second scale is applied to blur the transition area between the foreground and the background; The graphics processor is used to perform a second operation task to obtain a blurred image of the image to be processed, and the second operation task includes blurring the sampled image according to the image processing data.

10. A terminal device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method according to any one of claims 1 to 7 when executing the computer program.

11. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images