Image processing method and device, electronic equipment, chip and storage medium

Abstract

The invention provides an image processing method and device, electronic equipment, a chip and a storage medium, and relates to the field of image processing.The method comprises the steps that depth information and focal length information corresponding to a to-be-processed picture are fused, and fusion information is obtained; performing feature extraction and feature enhancement processing on the fusion information to obtain target features; determining a blurring radius of each pixel point in the picture to be processed according to the target feature; and performing blurring processing on the to-be-processed picture according to the blurring radius of each pixel point to obtain a target picture. Therefore, the mode of dynamically calculating the blurring degree based on the depth and the focal length is more flexible and efficient, the picture main body can be more effectively highlighted, the space sense and the layering sense of the picture can be enhanced, the overall visual effect can be remarkably improved, the depth-of-field effect of an optical lens can be truly simulated, the blurring transition in the picture is more natural and smooth, and the user experience is improved. Therefore, the visual experience of the user is greatly improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to an image processing method and device, electronic equipment, a chip and a storage medium. BACKGROUND

[0002] In the fields of photography, film production, teaching, virtual reality (VR) and augmented reality (AR), etc., background blurring (or background blur) is a common technical means for emphasizing the subject, enhancing the visual level and aesthetic sense, and providing a more comfortable and more engaging visual experience for the viewer. For example, in the fields of VR and AR, background blurring can be used to improve the realism and immersion of the user experience and enhance spatial perception. SUMMARY

[0003] The present application proposes an image processing method and device, electronic equipment, a chip and a storage medium to at least partially solve one of the technical problems in the related art.

[0004] An embodiment of the present application provides an image processing method, comprising:

[0005] fusing depth information and focal length information corresponding to a to-be-processed picture to obtain fused information;

[0006] performing feature extraction and feature enhancement processing on the fused information to obtain target features;

[0007] determining a blurring radius of each pixel point in the to-be-processed picture according to the target features;

[0008] performing blurring processing on the to-be-processed picture according to the blurring radius of each pixel point to obtain a target picture.

[0009] Another embodiment of the present application provides an image processing device, comprising:

[0010] a fusion module configured to fuse depth information and focal length information corresponding to a to-be-processed picture to obtain fused information;

[0011] a processing module configured to perform feature extraction and feature enhancement processing on the fused information to obtain target features;

[0012] a determination module configured to determine a blurring radius of each pixel point in the to-be-processed picture according to the target features;

[0013] a blurring module configured to perform blurring processing on the to-be-processed picture according to the blurring radius of each pixel point to obtain a target picture.

[0014] The embodiment of the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, when the processor executes the program, the image processing method in the foregoing aspect is realized.

[0015] The embodiment of the present application also provides a chip, including an interface circuit and a processing circuit which are coupled with each other, the interface circuit is used for inputting or outputting signals, and the processing circuit is configured to execute the image processing method in the foregoing aspect.

[0016] The embodiment of the present application also provides a non-transitory computer readable storage medium, having stored thereon computer program instructions, when the computer program instructions are executed by a processor, the image processing method in the foregoing aspect is realized.

[0017] The embodiment of the present application also provides a computer program product, having stored thereon a computer program, when the computer program is executed by a processor, the image processing method in the foregoing aspect is realized.

[0018] The image processing method, device, electronic device, chip and storage medium provided by the present application comprehensively determine the blurring radius of each pixel point in the to-be-processed picture according to the depth information and the focal length information corresponding to the to-be-processed picture, which means that when the image blurring processing is performed, the fine blurring operation can be performed on different regions in the to-be-processed picture according to the respective corresponding blurring degree. Compared with the uniform blurring processing manner (i.e., the uniform blurring strength blurring manner) in the related art, the method of dynamically calculating the blurring degree based on the depth and the focal length is more flexible and efficient, which not only can more effectively highlight the picture subject and enhance the spatial sense and the level sense of the picture, but also can significantly improve the overall visual effect and truly simulate the depth of field effect of the optical lens, so that the blurring transition in the picture is more natural and smooth, thereby greatly improving the visual experience of the user. For example, when a scene including a complex depth of field structure is processed, the method can accurately control the blurring strength of each region according to the relative distance between each scene element (such as an object, a person, an animal, a scenery, etc.) and the camera, in combination with the focal length information, thereby avoiding the problems of excessive blurring or insufficient blurring, and better meeting the actual blurring needs of the user in different application scenarios.

[0019] Additional aspects and advantages of the present application will be made apparent by the following description and the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0020] The above and / or additional aspects and advantages of the present application will become apparent and be readily understood from the following description, taken in conjunction with the accompanying drawings, in which:

[0021] Figure 1A flowchart of an image processing method provided by an example embodiment of the present application;

[0022] Figure 2 A structure diagram of a prediction model provided by an example embodiment of the present application;

[0023] Figure 3 A flowchart of another image processing method provided by an example embodiment of the present application;

[0024] Figure 4 A flowchart of still another image processing method provided by an example embodiment of the present application;

[0025] Figure 5 A flowchart of yet another image processing method provided by an example embodiment of the present application;

[0026] Figure 6 A schematic diagram of a virtualization principle of a game screen provided by an example embodiment of the present application;

[0027] Figure 7 A structure diagram of an image processing device provided by an example embodiment of the present application;

[0028] Figure 8 A structure diagram of an electronic device provided by an example embodiment of the present application;

[0029] Figure 9 A structure diagram of a chip provided by an example embodiment of the present application. DETAILED DESCRIPTION

[0030] Embodiments of the present application are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar notations represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by reference to the drawings are exemplary and are intended to explain the present application, and cannot be understood as limiting the present application.

[0031] In the field of games, the depth of field (DOF) technology simulates the focusing characteristics of a real camera to blur objects outside a specific distance in a game screen, thereby effectively enhancing the prominence of the main body and the spatial level, and improving the visual immersion and artistic expression.

[0032] Exemplarily, the foreground and the background in a game screen can be classified, and the background can be blurred, and the blurred background and the foreground can be fused to generate a game screen with a depth of field blur effect. However, this scheme uses isotropic blur for the background, which cannot simulate different blur degrees brought by different depths, and the virtualization effect is not realistic.

[0033] Early depth-of-field effects mainly rely on basic post-processing algorithms such as Gaussian blur (smooth blur is achieved by weighted average of pixel values) and bokeh effect (generate light spots with geometric features by simulating lens aperture shape). Although these methods are computationally efficient, they have certain limitations in terms of realism and detail restoration.

[0034] With the evolution of graphics application programming interface (API), depth rendering technology has gradually upgraded. For example, the introduction of geometry shaders allows developers to achieve more detailed layered rendering by splitting depth maps, improving the accuracy of the blur effect; for example, by optimizing texture sampling efficiency, reducing the load on the graphics processing unit (GPU), and providing support for more complex post-processing effects.

[0035] The popularity of ray tracing technology has driven depth-of-field blur to a physically realistic level. By simulating the propagation path of light in a virtual lens, the system can calculate the blur level of objects at different distances in real time, especially in the light source area, generating natural and realistic bokeh spot effects.

[0036] Although the depth-of-field blur technology has made great progress, it still faces certain challenges. On the one hand, the depth effect driven by ray tracing consumes a lot of GPU performance, and usually requires hardware acceleration support to run smoothly; on the other hand, excessive use or improper parameter settings may cause picture distortion, unclear focus, and even cause visual fatigue for players. Therefore, how to achieve a good balance between realism, artistic expression and user experience is still a key issue that needs to be fine-tuned in the current development process.

[0037] To solve at least one of the problems in the related art, the present application provides an image processing method, device, electronic equipment, chip and storage medium.

[0038] The image processing method, device, electronic equipment, chip and storage medium of the embodiments of the present application are described below with reference to the accompanying drawings. Before specifically describing the embodiments of the present application, in order to facilitate understanding, first introduce the commonly used technical words:

[0039] Focal length information, used to indicate the focal length parameter of a camera generating or capturing a picture. Taking the photography field as an example, the focal length information is used to indicate the lens focal length used by a real camera in capturing a shot picture of a current frame, reflecting the width and narrowness of the imaging angle of view and the magnification ratio of the photographed object in the shot picture; taking the game field as an example, the focal length information is used to indicate the focal length parameter of a virtual camera rendering a game picture of a current frame, wherein the virtual camera simulates the optical behavior of a physical camera in the real world, including but not limited to focal length, field of view (FOV), depth of field (DOF) and other characteristics, so as to realize a more realistic visual effect and an immersive experience.

[0040] Depth information, used to indicate the depth value of each pixel point in a picture relative to the camera angle of view, i.e. the distance relationship of the pixel point in the three-dimensional space from the camera.

[0041] RGB: R (Red): represents the red channel, controls the intensity of red in the image, and the intensity corresponds to the value range of 0-255; G (Green): represents the green channel, controls the intensity of green in the image, and the intensity corresponds to the value range of 0-255; B (Blue): represents the blue channel, controls the intensity of blue in the image, and the intensity corresponds to the value range of 0-255.

[0042] Figure 1 A flowchart of an image processing method provided by an exemplary embodiment of the present application.

[0043] It should be noted that the image processing method of the embodiments of the present application can be applied to an image processing device, and in some possible embodiments, the image processing device can be configured in an electronic device or a chip, so that the electronic device or the chip can perform an image processing function. In addition, in some possible embodiments, the image processing device can also be software in an electronic device, etc.

[0044] In any one of the embodiments of the present application, the chip can be integrated into an electronic device. The chip includes a central processing unit (CPU), an image signal processor (ISP), a graphic processing unit (GPU), a neural processing unit (NPU), an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a field-programmable gate array (FPGA), a system on chip (SOC), a reduced instruction set computer (RISC), and the like, which are not listed one by one.

[0045] The electronic device includes, but is not limited to, a terminal, a personal computer, and the like. The terminal is an entity on the user side for receiving or transmitting signals, such as a mobile phone. The terminal can also be referred to as a terminal device (terminal), a user equipment (UE), a mobile station (MS), a mobile terminal (MT), and the like. The terminal can be a car with communication function, a smart car, a mobile phone, a wearable device, a tablet computer (Pad), a computer with wireless transceiver function, a virtual reality (VR) terminal, an augmented reality (AR) terminal, a wireless terminal in industrial control, a wireless terminal in self-driving, a wireless terminal in remote medical surgery, a wireless terminal in smart grid, a wireless terminal in transportation safety, a wireless terminal in smart city, a wireless terminal in smart home, and the like. The embodiments of the present application do not limit the specific technology and specific device form of the terminal.

[0046] As Figure 1 shown, the image processing method can include the following steps S101-S104:

[0047] Step S101, fusing the depth information and the focal length information corresponding to the to-be-processed picture to obtain fused information.

[0048] The depth information is used to describe the distance relationship between each visual element (such as objects, people, animals, scenery, etc.) in the to-be-processed picture and the camera. Generally, the farther the object is from the camera, the larger the blur radius is needed in the blurring process to create a stronger blur effect, thereby highlighting the subject closer to the camera, for example, in a scene of shooting a person and a background, the background object is farther from the camera, and a larger blur radius can make it more blurred, making the person subject more eye-catching.

[0049] The focal length information affects the angle of view and the depth of field range of the camera. At different focal lengths, the depth of field effect of the to-be-processed picture is different, and the corresponding blurring degree also differs. It should be understood that determining the blur radius of each pixel point in the to-be-processed picture according to the focal length information can more accurately simulate the blurring effect at different focal lengths when the to-be-processed picture is shot, so that the generated blur radius is more in line with the optical principle and visual habit.

[0050] The application does not limit the acquisition method of the to-be-processed picture.

[0051] Illustratively, the to-be-processed picture can be a picture taken by the user, i.e., a shooting picture taken by responding to a shooting operation triggered by the user, wherein the shooting picture includes an original picture taken by a camera, or a new picture obtained by image processing on the original picture.

[0052] Illustratively, the to-be-processed picture can be a picture collected online by the user, such as collecting the to-be-processed picture online by using web crawler technology.

[0053] Illustratively, the to-be-processed picture can be a game picture to be rendered in a game, such as a game picture of the current frame in the game.

[0054] Illustratively, the to-be-processed picture can be a video picture to be rendered in a video, such as a video picture of the current frame in the video.

[0055] Illustratively, the to-be-processed picture can be an image resource selected from a local storage resource by responding to a selection operation triggered by the user.

[0056] It should be noted that the above acquisition method of the to-be-processed picture is only illustrative, and the application is not limited thereto, for example, the to-be-processed picture can also be an artificially synthesized picture, etc.

[0057] In the embodiments of the present application, the depth information and the focal length information corresponding to the to-be-processed picture can be fused based on a fusion algorithm to obtain fusion information.

[0058] In step S102, feature extraction and feature enhancement processing are performed on the fusion information to obtain target features.

[0059] In the embodiments of the present application, the feature extraction and feature enhancement processing can be performed on the fusion information based on a deep learning technology or a machine learning technology to obtain the target features.

[0060] As an example, a multi-layer fully connected network in a prediction model can be used to perform the feature extraction and the feature enhancement processing on the fusion information to obtain the target features. The multi-layer fully connected networks are connected by using an activation function, such as a Rectified Linear Unit (ReLU) activation function.

[0061] In the feature extraction, the multi-layer fully connected network (such as implemented in the form of 1x1 convolution) plays a role of feature extraction. Each layer of the fully connected network performs linear transformation on the fusion information, and maps the fusion information to a new feature space through a learned weight matrix. In this process, the prediction model can gradually extract high-level features related to the blurring radius. For example, the front fully connected network can extract some simple local features, such as basic patterns of depth changes, and the rear fully connected network can capture more complex global features, such as the relationship between the depth distribution of the entire scene and the focal length.

[0062] In the feature enhancement, the activation function plays a role of feature enhancement between each two fully connected networks. For example, the ReLU function sets negative values to 0 and retains positive values. This nonlinear transformation can increase the sparsity of the features, highlight important feature information, and suppress irrelevant or noise information. Through the alternating use of the multi-layer fully connected network and the activation function, the prediction model can continuously enhance and optimize the features, so that the extracted features are more representative and distinguishable.

[0063] For example, taking a prediction model including four layers of fully connected networks (FC) as an example, the structure of the prediction model can be as shown in Figure 2 wherein the ReLU activation function is used to connect between each two adjacent layers of the fully connected networks, and the last layer of the prediction model is the ReLU activation function. The numbers (such as 2, 4, and 1) on the input and output arrows of the fully connected networks represent the feature dimension numbers.

[0064] In step S103, the blurring radius of each pixel point in the to-be-processed picture is determined according to the target features.

[0065] The blur radius, also known as the diffusion radius, is used to indicate the degree of blurring of a pixel. The blur radius and the degree of blurring are positively correlated; that is, the larger the blur radius, the higher the degree of blurring.

[0066] The presentation methods for the blur radius of each pixel in the image to be processed include, but are not limited to, images and arrays. Taking an image as an example, the blur radius of the corresponding pixel in the image to be processed can be indicated by a blur radius map. That is, the size of the blur radius map matches the size of the image to be processed, and the pixel value of each pixel in the blur radius map is used to indicate the blur radius of the corresponding pixel in the image to be processed.

[0067] For example, taking an image in the RGB color space as the image to be processed, the image to be processed is labeled as I. input If the blurred radius of the graph is B, then I input ∈R H*W*3 (H is the image height, W is the image width, and 3 represents the R, G, and B color channels), B∈R H*W*1 The pixel value of the pixel in the i-th row and j-th column of B is used to indicate I. input Let i be the blur radius of the pixel in the i-th row and j-th column of the array; where i is a positive integer not greater than H and j is a positive integer not greater than W.

[0068] In the embodiments of this application, the blur radius of each pixel in the image to be processed can be determined based on deep learning technology or machine learning technology and according to the target features.

[0069] For example, taking the rendering method of the blur radius of each pixel in the image to be processed as an example, it can be done by, as follows: Figure 2 The prediction model shown outputs a blur radius map; where the pixel value of each pixel in the blur radius map is used to indicate the blur radius of the corresponding pixel in the image to be processed.

[0070] Step S104: Based on the blur radius of each pixel, blur the image to be processed to obtain the target image.

[0071] In this embodiment of the application, an image blurring processing algorithm can be used to perform targeted blurring processing on the image area where the corresponding pixel is located in the image to be processed, based on the blurring radius of each pixel (such as the blurring radius indicated by each pixel in the blurring radius diagram), to obtain the target image.

[0072] Image blurring algorithms include, but are not limited to: Gaussian blur algorithm, bokeh blur algorithm, or other blur algorithms.

[0073] The image processing method of the embodiment of the present application comprehensively determines the blurring radius of each pixel point in the to-be-processed picture according to the depth information and the focal length information corresponding to the to-be-processed picture, which means that when the image blurring processing is performed, the fine blurring operation can be performed according to the respective blurring degree of different regions in the to-be-processed picture. Compared with the uniform blurring processing manner (i.e., the uniform blurring strength blurring manner) in the related art, the method of dynamically calculating the blurring degree based on the depth and the focal length is more flexible and efficient, which not only can more effectively highlight the main body of the picture and enhance the spatial sense and the level sense of the picture, but also can significantly improve the overall visual effect and realistically simulate the depth of field effect of the optical lens, so that the blurring transition in the picture is more natural and smooth, thereby greatly improving the visual experience of the user. For example, when a scene containing a complex depth of field structure is processed, the method can accurately control the blurring strength of each region according to the relative distance between each scene element (such as an object, a person, an animal, a scenery, etc.) and the camera in combination with the focal length information, thereby avoiding the problems of excessive blurring or insufficient blurring and better meeting the actual blurring needs of the user in different application scenarios.

[0074] As a possible implementation manner, Figure 3 Another flowchart of an image processing method provided by an exemplary embodiment of the present application.

[0075] It should be noted that the image processing method can be executed alone, or can be executed in combination with any one of the embodiments or the possible implementation manners in the embodiments, or can be executed in combination with any one of the technical solutions in the related art, and the embodiments of the present application do not limit this.

[0076] As Figure 3 indicated, the image processing method can include the following steps S301 to S305:

[0077] Step S301: performing a downsampling operation on a depth map corresponding to the to-be-processed picture to obtain a sampled depth map.

[0078] The depth information of the to-be-processed picture is in the form of a depth map, wherein the pixel value of each pixel point in the depth map is used to indicate the depth of the corresponding pixel point in the to-be-processed picture.

[0079] In the embodiment of the present application, a downsampling algorithm can be used to perform a size reduction operation on the depth map based on a set downsampling multiple to obtain a downsampled depth map, which is denoted as a sampled depth map in the present application.

[0080] The downsampling algorithm includes but is not limited to: mean sampling algorithm (dividing the depth map into non-overlapping blocks of a size corresponding to the downsampling multiple, and taking the average value of the pixel values in each block as the pixel value of the corresponding position of the down-sampled depth map), nearest neighbor downsampling algorithm (selecting the pixel value of a fixed position (such as the upper left corner) in each block as the pixel value of the corresponding position of the down-sampled depth map), bilinear interpolation algorithm or bicubic interpolation algorithm, etc.

[0081] In step S302, the focal length information of the to-be-processed picture and the sampling depth map are fused to obtain fusion information.

[0082] In the embodiments of the present application, the focal length information of the to-be-processed picture and the sampling depth map can be fused based on a fusion algorithm to obtain fusion information.

[0083] In step S303, feature extraction and feature enhancement processing are performed on the fusion information to obtain target features.

[0084] In step S304, the blurring radius of each pixel point in the to-be-processed picture is determined according to the target features.

[0085] It should be noted that the explanation and description of steps S303 to S304 can refer to the related description in any embodiment of the present application, which will not be repeated here.

[0086] In any embodiment of the present application, the presentation mode of the blurring radius of each pixel point in the to-be-processed picture is taken as an example for an image. In the present application, first, the target features can be mapped to a dimension matching the sampling depth map to obtain an initial blurring radius map; the size of the initial blurring radius map matches the sampling depth map, and then an upsampling operation can be performed on the initial blurring radius map to obtain a target blurring radius map; the size of the target blurring radius map matches the depth map, and the pixel value of a pixel point in the target blurring radius map is used to indicate the blurring radius of the corresponding pixel point in the to-be-processed picture.

[0087] Exemplarily, an upsampling algorithm can be used to perform a size enlargement operation on the initial blurring radius map based on a set upsampling multiple to obtain an upsampled initial blurring radius map, which is denoted as a target blurring radius map in the present application.

[0088] The upsampling multiple is equal to the downsampling multiple.

[0089] In summary, a deep learning technology such as a neural network can be used to generate the initial blurring radius map and the target blurring radius map in milliseconds through parallel computing and optimization architecture, meeting the real-time application requirements.

[0090] In any embodiment of the present application, the target blurring radius map can be generated by the following steps A to C:

[0091] Step A: performing an up-sampling operation on the initial blurring radius map to obtain a first blurring radius map.

[0092] Exemplarily, an up-sampling algorithm can be adopted to perform a size enlargement operation on the initial blurring radius map based on a set up-sampling multiple to obtain the first blurring radius map.

[0093] Exemplarily, taking 2 as the up-sampling multiple and the down-sampling multiple, and taking the prediction model as f(·) and the first blurring radius map as B pred , then B pred The following formula can be used to obtain:

[0094] B pred = [f(f input , (D input )↓2)]↑2; (1)

[0095] Wherein, f input is the focal length information, D input is the depth map, (·)↓2 is the two times down-sampling, (·)↑2 is the two times up-sampling, and B pred is the first blurring radius map output by the prediction model.

[0096] Step B: performing an integer operation on the first blurring radius map to obtain a second blurring radius map.

[0097] The integer operation includes an upward integer operation and a downward integer operation.

[0098] It should be noted that the blurring radius map generated by the neural network is usually floating-point data, but the actual blurring operation (such as Gaussian blur, circular light spot simulation, etc.) usually needs integer blurring radius. If the floating-point blurring radius is directly used, the following problems may occur: calculation error: when the blur kernel size is not an integer, interpolation or approximation processing is needed, which may introduce edge artifacts or brightness deviation; invalid operation: the floating-point blurring radius may be forced to be truncated, resulting in unpredictable blurring results.

[0099] To solve the above problems, in the present application, an integer operation can be performed on the first blurring radius map output by the neural network to convert the floating-point blurring radius to an integer, ensuring the effectiveness of the blurring operation.

[0100] Step C: performing a truncation operation on the second blurring radius map according to the set maximum blurring radius and minimum blurring radius to obtain a target blurring radius map.

[0101] That is, the pixel value of each pixel point in the target blurring radius map is between the minimum blurring radius and the maximum blurring radius. The minimum blurring radius and the maximum blurring radius can be adjusted according to actual application requirements.

[0102] It should be noted that the blurring radius output by the neural network may have unreasonable values due to training data bias or input abnormalities (such as noise, extreme light), for example, an excessively large radius: exceeding the image size (such as a blurring radius > image width / 2), resulting in excessive blurring of the entire image and loss of key details; an excessively small radius: the blurring radius is close to 0 or negative (because the output of the neural network is not constrained), resulting in ineffective blurring or program errors.

[0103] To solve the above problems, in the present application, the second blurring radius map can be subjected to a clipping operation according to the set maximum blurring radius and minimum blurring radius, so as to forcibly limit the blurring radius within a reasonable range, so as to avoid the situation that the subject and the background are completely mixed due to an excessively large blurring radius, or the blurring effect is invisible due to an excessively small blurring radius.

[0104] For example, taking the minimum blurring radius as 1 and the maximum blurring radius as 15 as an example, and marking the target blurring radius map as B, B can be obtained by using the following formula:

[0105] B = clip(ceil(B pred ), 1, 15); (2)

[0106] Wherein, ceil(·) is a rounding up operation, and clip(·) is a clipping operation.

[0107] In summary, performing rounding and clipping operations on the blurring radius map can improve the stability of the blurring effect, optimize the calculation efficiency, enhance the scene adaptability and robustness.

[0108] In step S305, the blurring radius of each pixel point is used to perform blurring processing on the to-be-processed picture to obtain a target picture.

[0109] It should be noted that the explanation of step S305 can refer to the related description in any embodiment of the present application, which will not be repeated here.

[0110] The image processing method of the present application down-samples the high-resolution depth map to a low-resolution, and then predicts the blurring radius of each pixel point in the to-be-processed picture according to the low-resolution depth map and the focal length information, which can significantly reduce the demand for computing resources, reduce the memory occupation, and improve the prediction speed. In addition, appropriate down-sampling can remove unnecessary detail noise while retaining important global information, thereby improving the accuracy of prediction.

[0111] As a possible implementation manner, Figure 4A flowchart of another image processing method provided for the exemplary embodiments of the present application is shown.

[0112] It should be noted that the image processing method can be executed alone, or can be executed in combination with any of the embodiments or possible implementation manners of the embodiments, or can be executed in combination with any of the technical solutions in the related art, and the embodiments of the present application do not limit this.

[0113] As shown in Figure 4 The image processing method can include the following steps S401-S405.

[0114] Step S401, fusing depth information and focal length information corresponding to a to-be-processed picture to obtain fused information.

[0115] Step S402, performing feature extraction and feature enhancement processing on the fused information to obtain target features.

[0116] It should be noted that the explanations and descriptions of steps S401-S402 can be referred to the related descriptions in any of the embodiments of the present application, which will not be repeated here.

[0117] Step S403, determining a target blurring radius map according to the target features; wherein a pixel value of a pixel point in the target blurring radius map is used to indicate a blurring radius of a corresponding pixel point in the to-be-processed picture.

[0118] In the embodiments of the present application, the target blurring radius map can be predicted according to the target features based on a deep learning technology or a machine learning technology; wherein a pixel value of a pixel point in the target blurring radius map is used to indicate a blurring radius of a corresponding pixel point in the to-be-processed picture.

[0119] Step S404, traversing each pixel point in the target blurring radius map, and determining a target blurring radius of a corresponding pixel point in the to-be-processed picture according to a pixel value of a currently traversed pixel point.

[0120] In the embodiments of the present application, each pixel point in the target blurring radius map can be traversed in sequence, and a pixel value of a currently traversed pixel point, such as an i-th row and j-th column pixel point, is taken as a target blurring radius of an i-th row and j-th column pixel point in the to-be-processed picture.

[0121] Step S405, performing blurring processing on a to-be-blurred region where a corresponding pixel point in the to-be-processed picture is located according to the target blurring radius, to obtain a target picture.

[0122] The to-be-blurred region refers to a surrounding pixel range that needs to be blurred, which is determined in combination with the target blurring radius and with the i-th row and j-th column pixel point in the to-be-processed picture as the center.

[0123] In the embodiment of the present application, the target blurring radius of the i-th row and j-th column pixel point in the to-be-processed picture can be used to determine the to-be-blurred region where the i-th row and j-th column pixel point is located, and the target blurring radius can be used to perform blurring processing on the to-be-blurred region, so as to obtain the target picture.

[0124] In any one of the embodiments of the present application, the Gaussian blur algorithm can be used to perform Gaussian blur blurring processing on the to-be-blurred region where the i-th row and j-th column pixel point in the to-be-processed picture is located according to the target blurring radius of the i-th row and j-th column pixel point in the to-be-processed picture.

[0125] As an example, first, the Gaussian kernel and the to-be-blurred region can be determined according to the target blurring radius of the i-th row and j-th column pixel point in the to-be-processed picture; wherein the size and weight distribution of the Gaussian kernel can be adjusted according to the target blurring radius, the larger the target blurring radius, the larger the Gaussian kernel, and the more dispersed the weight distribution, so as to achieve a more obvious blur effect; then, the Gaussian blur blurring processing can be performed on the to-be-blurred region according to the Gaussian kernel, so as to obtain the target picture.

[0126] In summary, using the Gaussian blur algorithm to perform Gaussian blur blurring processing on the to-be-processed picture can reduce the computational complexity, improve the efficiency of the depth-of-field blurring processing, ensure the real-time performance of the blurring processing, and reduce the rendering power consumption of the image.

[0127] The image processing method of the embodiment of the present application can break the fixed dependence between the blurring intensity (blur degree) and the pixel spatial position, decouple the blurring intensity and the spatial position, independently control the blurring intensity of each pixel point, achieve fine and flexible image blurring effect, and ultimately improve the visual quality (naturalness, realism, artistic expression) of the blurred image.

[0128] As a possible implementation manner, taking the to-be-processed picture as a game picture in the game field as an example, Figure 5 Another flowchart of an image processing method provided by an exemplary embodiment of the present application is shown.

[0129] It should be noted that the image processing method can be executed alone, or can be executed in combination with any one of the embodiments or the possible implementation manners of the embodiments of the present application, or can be executed in combination with any one of the technical solutions in the related art, and the embodiments of the present application do not limit this.

[0130] As shown in the figure, Figure 5 The image processing method can include the following steps S501 to S505:

[0131] Step S501, preprocessing the game picture; wherein the preprocessing includes at least one of the following: vertex shading, rasterization, and lighting rendering.

[0132] Wherein, vertex shading includes geometry transformation, vertex attribute processing, and animation implementation.

[0133] Wherein, geometry transformation: various transformations such as translation, rotation, and scaling are performed on the vertices of three-dimensional models in the game. Through these transformations, three-dimensional models can be accurately placed in the appropriate position of the game world and presented in the correct pose. For example, when the character moves or the scene changes, the vertex coordinates of the three-dimensional model can be adjusted in real time to ensure that the dynamic display of the three-dimensional model conforms to the game logic.

[0134] Wherein, vertex attribute processing: processing various attributes of vertices such as color, normal, etc. For example, according to the game settings, different three-dimensional models are given unique colors, or the normal direction of the vertex is adjusted to provide accurate information for subsequent lighting calculations, making the three-dimensional model more realistic under lighting.

[0135] Wherein, animation implementation: through dynamic modification of vertex positions, effects such as skeletal animation are realized. By calculating the influence of bone movement on vertices, characters can make various natural movements, enhancing the liveliness of the game.

[0136] Wherein, rasterization is the key step of converting three-dimensional models into two-dimensional pixel images. The three-dimensional models in the game are composed of vertices and edges, and the rasterization process determines which pixels each triangle (the basic unit of three-dimensional models) covers, and interpolates the vertex attributes (such as color, texture coordinates, etc.) of the triangle to these pixels, accurately presenting the three-dimensional model on the two-dimensional screen.

[0137] Wherein, lighting rendering can simulate the propagation and interaction of light in the real world, making the surface of objects in the game scene present realistic light and shade changes, shadow effects, and color reflections. For example, objects under sunlight will have bright light-receiving surfaces and darker backlit surfaces, and natural shadows will also be produced between objects. These effects make the game scene look more realistic, as if the player is in the real world.

[0138] Step S502, fusing the depth information and focal length information corresponding to the preprocessed game picture to obtain fusion information.

[0139] Step S503, performing feature extraction and feature enhancement processing on the fusion information to obtain target features.

[0140] Step S504, determining the blurring radius of each pixel point in the preprocessed game picture according to the target features.

[0141] Step S505, according to the blurring radius of each pixel point, the preprocessed game picture is blurred to obtain the target picture.

[0142] It should be noted that the explanation of steps S502 to S505 can refer to the related description in any embodiment of the present application, which will not be repeated here.

[0143] In any embodiment of the present application, after obtaining the target picture, the target picture can also be subjected to a graphical user interface rendering operation for display.

[0144] Among them, the graphical user interface rendering operation is a process of presenting the designed user interface elements, such as buttons, icons, texts, progress bars, etc., to the picture in a graphical manner according to specific rules and layouts. Thus, it can provide an interactive interface and information display for the player, facilitate the player to interact with the game, understand the game state and perform operations. For example, displaying start button, task prompt text, health value progress bar, etc. on the game screen can improve the interactive experience of the player.

[0145] The image processing method of the embodiments of the present application can create high-quality, realistic and expressive pictures for the game through vertex shading, rasterization and lighting rendering of the game picture, greatly improving the game experience of the player.

[0146] In any embodiment of the present application, taking the real-time game rendering scene of the terminal as an example, the depth information and focal length information of the game picture can be used for depth-of-field blur processing, and the out-of-focus area of the far view and the near view can be blurred in real time. Through the blur post-processing, the rendering result is more realistic. For example, the neural network can be used to generate the blurring radius of each pixel point in the game picture according to the depth information and focal length information of the game picture, and the Gaussian blur algorithm can be used for parallel blur processing, which can reduce the computational complexity and reduce the rendering power consumption.

[0147] As an example, the blurring principle of the game picture can be as shown in Figure 6 , which mainly includes the following steps:

[0148] 1. After the game picture is subjected to vertex shading and rasterization, lighting rendering is performed to obtain a clear current frame image I input ∈R H*W*3 ;

[0149] 2. Obtain the depth information D input ∈R H*W*1 ;

[0150] 3. Obtain the focal length information f input ∈R 1 ;

[0151] 4. Using the focal length information and the depth information, calculate the blur radius map B.

[0152] Exemplarily, a neural network (denoted as a prediction model in the present application) as shown in Figure 2 may be used for inference, and the prediction model is denoted as f(·), and the following equation is obtained:

[0153] B pred = [f(f input (D input )↓2)]↑2.

[0154] In some possible implementations, the B pred may also be subjected to an integer operation and a truncation operation to obtain B, as follows:

[0155] B = clip(ceil(B pred ), 1, 15).

[0156] 5. For each pixel point in I input , use the pixel value of the corresponding pixel point in the blur radius map B as the blur radius to perform Gaussian blur processing, and replace the original pixel value with the blurred pixel value to obtain the blurred output image I blur .

[0157] 6. Perform subsequent processing such as user interface (User Interface, referred to as UI) drawing on I blur , and send it to display.

[0158] To sum up, the scheme provided by the present application has at least the following advantages: based on the depth learning technology to calculate the depth of field, and using the Gaussian blur algorithm to reduce the calculation complexity, which can improve the efficiency of the depth of field blur processing, ensure the real-time performance of the image processing, and reduce the rendering power consumption.

[0159] In order to realize the above-mentioned embodiments, the embodiments of the present application further provide an image processing device.

[0160] Figure 7 A structural schematic diagram of an image processing device provided by an exemplary embodiment of the present application.

[0161] As shown in Figure 7 , the image processing device 700 can include a fusion module 710, a processing module 720, a determination module 730, and a blur module 740.

[0162] The fusion module 710 is configured to fuse the depth information and the focal length information corresponding to the to-be-processed picture to obtain fusion information.

[0163] The processing module 720 is configured to perform feature extraction and feature enhancement processing on the fusion information to obtain target features.

[0164] The determining module 730 is configured to determine the blurring radius of each pixel point in the to-be-processed picture according to the target features.

[0165] The blurring module 740 is configured to perform blurring processing on the to-be-processed picture according to the blurring radius of each pixel point to obtain the target picture.

[0166] In an implementation manner of the embodiment of the present application, the depth information is in the form of a depth map, and a pixel value of each pixel point in the depth map is used to indicate the depth of a corresponding pixel point in the to-be-processed picture.

[0167] The fusion module 710 is configured to perform downsampling on the depth map to obtain a sampled depth map, and fuse the sampled depth map and the focal length information to obtain the fusion information.

[0168] In an implementation manner of the embodiment of the present application, the determining module 730 is configured to map the target features to a dimension matched with the sampled depth map to obtain an initial blurring radius map, wherein the initial blurring radius map matches the size of the sampled depth map; and perform upsampling on the initial blurring radius map to obtain a target blurring radius map, wherein the target blurring radius map matches the size of the depth map, and a pixel value of each pixel point in the target blurring radius map is used to indicate the blurring radius of a corresponding pixel point in the to-be-processed picture.

[0169] In an implementation manner of the embodiment of the present application, the determining module 730 is configured to perform upsampling on the initial blurring radius map to obtain a first blurring radius map, perform rounding on the first blurring radius map to obtain a second blurring radius map, and perform truncation on the second blurring radius map according to a set maximum blurring radius and a set minimum blurring radius to obtain the target blurring radius map.

[0170] In an implementation manner of the embodiment of the present application, the blurring module 740 is configured to traverse each pixel point in the target blurring radius map, and determine the target blurring radius of a corresponding pixel point in the to-be-processed picture according to a pixel value of a currently traversed pixel point; and perform blurring processing on a to-be-blurred region where the corresponding pixel point is located in the to-be-processed picture according to the target blurring radius to obtain the target picture.

[0171] In an implementation manner of the embodiment of the present application, the blurring module 740 is configured to determine a Gaussian kernel and a to-be-blurred region according to the target blurring radius; and perform Gaussian blur blurring processing on the to-be-blurred region according to the Gaussian kernel to obtain the target picture.

[0172] In an implementation form of the embodiment of the application, the to-be-processed picture comprises at least one of the following: a game picture to be rendered in a game; a shooting picture obtained in response to a shooting operation; a video picture to be rendered in a video; an image resource selected from a local storage resource in response to a selection operation.

[0173] In an implementation form of the embodiment of the application, in response to the to-be-processed picture comprising a game picture, the image processing apparatus 700 can further comprise:

[0174] a preprocessing module, configured to perform preprocessing on the game picture; wherein the preprocessing comprises at least one of the following: vertex shading, rasterization, and illumination rendering.

[0175] In an implementation form of the embodiment of the application, the image processing apparatus 700 can further comprise:

[0176] an execution module, configured to perform a graphical user interface rendering operation on the target picture to display.

[0177] It should be noted that the foregoing explanation and description of the image processing method embodiment also apply to the image processing apparatus of this embodiment, which will not be described here again.

[0178] In the image processing apparatus of the embodiment of the application, the depth information and the focal length information corresponding to the to-be-processed picture are comprehensively determined to determine the blurring radius of each pixel point in the to-be-processed picture, which means that when performing image blurring processing, a fine blurring operation can be performed on different regions in the to-be-processed picture according to their respective blurring degrees. Compared with the uniform blurring processing manner (i.e., the uniform blurring manner with a uniform blurring strength) in the related art, this manner of dynamically calculating the blurring degree based on the depth and the focal length is more flexible and efficient, which not only can more effectively highlight the main body of the picture and enhance the spatial sense and the level sense of the picture, but also can significantly improve the overall visual effect and realistically simulate the depth of field effect of an optical lens, so that the blurring transition in the picture is more natural and smooth, thereby greatly improving the visual experience of the user. For example, when processing a scene containing a complex depth of field structure, the application can accurately control the blurring strength of each region according to the relative distance between each scene element (such as an object, a person, an animal, a scenery, etc.) and the camera, in combination with the focal length information, thereby avoiding the problems of excessive blurring or insufficient blurring, and better meeting the actual blurring needs of the user in different application scenarios.

[0179] In order to implement the above-mentioned embodiments, the application further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor executes the program to implement the image processing method according to any one of the preceding embodiments.

[0180] Figure 8A structural diagram of an electronic device is provided for the exemplary embodiments of the present application. For example, the electronic device 800 can be a mobile phone, a computer, a digital broadcasting terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, or the like.

[0181] Referring to Figure 8 The electronic device 800 can include one or more of the following components: a processing component 802, a memory 804, a power component 806, a multimedia component 808, an audio component 810, an input / output (I / O) interface 812, a sensor component 814, and a communication component 816.

[0182] The processing component 802 generally controls the overall operations of the electronic device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 can include one or more processors 820 to execute instructions to complete all or part of steps of the above methods. In addition, the processing component 802 can include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 can include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0183] The memory 804 is configured to store various types of data to support operations of the electronic device 800. Examples of these data include instructions for any application or method operating on the electronic device 800, contact data, phonebook data, messages, pictures, videos, and the like. The memory 804 can be implemented by any type of volatile or non-volatile storage devices or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0184] The power component 806 provides power to the various components of the electronic device 800. The power component 806 can include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for the electronic device 800.

[0185] The multimedia component 808 includes a screen providing an output interface between the electronic device 800 and a user. In some embodiments, the screen can include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gesture on the touch panel. The touch sensor can not only sense a boundary of a touching or sliding action, but also detect duration and pressure related to the touching or sliding action. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the electronic device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each of the front camera and the rear camera can be a fixed optical lens system or have a focal length and optical zoom capability.

[0186] The audio component 810 is configured to output and / or input an audio signal. For example, the audio component 810 includes a microphone (MIC) configured to receive an external audio signal when the electronic device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 804 or transmitted via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting an audio signal.

[0187] The I / O interface 812 provides an interface between the processing component 802 and peripheral interface modules, which can be a keyboard, a click wheel, a button, and the like. The buttons can include, but are not limited to, a home button, a volume button, a start button, and a lock button.

[0188] The sensor component 814 includes one or more sensors for providing status assessments for various aspects of the electronic device 800. For example, the sensor component 814 can detect an open / closed position of the electronic device 800, relative positioning of components of the electronic device 800, such as a display and a keypad of the electronic device 800, changes in position of the electronic device 800 or a component of the electronic device 800, presence or absence of user contact with the electronic device 800, orientation or acceleration / deceleration / g-force and temperature changes of the electronic device 800. The sensor component 814 can include an accelerometer to measure a change in direction or speed of the electronic device 800. The sensor component 814 can further include a proximity sensor configured to detect presence of nearby objects without any physical contact. The sensor component 814 can also include a light sensor, such as a complementary metal-oxide-semiconductor (CMOS) or charge-coupled device (CCD) image sensor, utilized in imaging applications. In some embodiments, the sensor component 814 can further include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0189] The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 can access a wireless network based on a communication standard, such as WiFi, 4G, or 5G, or a combination thereof. In an example embodiment, the communication component 816 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In an example embodiment, the communication component 816 can further include a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra-Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0190] In an example embodiment, the electronic device 800 can be implemented by one or more Application-Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field-Programmable Gate Arrays (FPGAs), controllers, micro-controllers, microprocessors, or other electronic elements for executing the above-described methods.

[0191] In an example embodiment, a non-transitory computer-readable storage medium including instructions, such as the memory 804 including instructions, is also provided, which can be executed by the processor 820 of the electronic device 800 to complete the above-described methods. For example, the non-transitory computer-readable storage medium can be a Read-Only Memory (ROM), a Random Access Memory (RAM), a Compact Disc Read-Only Memory (CD-ROM), a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0192] To implement the above-described embodiments, the present application also proposes a chip, wherein the chip includes an interface circuit and a processing circuit coupled with each other, the interface circuit is configured to input or output a signal, and the processing circuit is configured to execute the image processing method provided in any of the preceding embodiments.

[0193] Figure 9 is a structural schematic diagram of a chip according to an example embodiment of the present application. For example, the structural schematic diagram of the chip 900 can be seen from Figure 9 but is not limited thereto.

[0194] The chip 900 includes a processing circuit 901, and the processing circuit 901 is configured to execute any of the above image processing methods.

[0195] In some embodiments, chip 900 further includes one or more interface circuits 902. In some possible implementations, interface circuits 902 are connected to memory 903, and can be configured to receive signals from memory 903 or other devices, and to send signals to memory 903 or other devices. For example, interface circuits 902 can read instructions stored in memory 903 and transmit the instructions to processing circuit 901.

[0196] In some embodiments, interface circuits 902 perform at least one of the communication steps of transmitting and / or receiving in the above-described methods, and processing circuit 901 performs the other steps.

[0197] In some embodiments, the terms interface circuit, interface, transceiver pin, transceiver, etc. can be replaced by each other.

[0198] In some embodiments, chip 900 further includes one or more memories 903 for storing instructions. In some possible implementations, all or part of memory 903 can be outside of chip 900.

[0199] To implement the above-described embodiments, the present application further proposes a non-transitory computer-readable storage medium having a computer program stored thereon, the program being executed by a processor to implement the image processing method according to any one of the preceding method embodiments.

[0200] To implement the above-described embodiments, the present application further proposes a computer program product having a computer program stored thereon, the program being executed by a processor to implement the image processing method according to any one of the preceding method embodiments.

[0201] In the description of the present specification, the description referring to the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any appropriate manner in any one or more embodiments or examples. In addition, the person skilled in the art can combine and combine the different embodiments or examples described in the present specification and the features of the different embodiments or examples, without contradiction.

[0202] Moreover, the terms "first", "second", "third", etc. are used herein only to describe different steps or categories of steps. Thus, the use of the term "first" does not imply that different steps are in any way prioritized, but simply serves as labels for different steps so as to improve the intelligibility of the description. Furthermore, the terms "first", "second", etc. are used only for descriptive purposes and should not be construed as indicating or implying relative importance or an indicated number of the technical features indicated. Thus, the features defined with "first", "second" can explicitly or implicitly comprise at least one of the features. In the description of the present application, the meaning of "plurality" is at least two, e.g. two, three, etc., unless explicitly specified otherwise.

[0203] Any process or method descriptions or blocks in flow charts or otherwise described herein represent embodiments of modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps, and alternate implementations are possible. The steps have been described as a certain order of execution; however, alternate implementations can perform the functions in a different order or out of sequence. Certain implementations can combine two or more steps into a single step or a single step into two or more steps; other implementations can perform certain steps in parallel.

[0204] The logic and / or steps represented in flow charts or otherwise described herein, for example, can be embodied in computer-readable medium containing executable instructions for execution by a processor-based system, apparatus, or device. The logic and / or steps can also be embodied in hardware, such as a processor, or in a combination of hardware and software. The computer-readable medium can be, for example, a floppy disk, a compact disc, a CD-ROM, a DVD, a Blu-ray disc, a magnetic tape, a RAM, a ROM, an EPROM, an EEPROM, a flash memory, a hard disk, a portable computer disk, a RAM, a ROM, a programmable logic device (PLD), a reprogrammable logic device (RPLD), a ROM, a PROM, a memory chip, a carrier wave, a modem, and the like. The computer-readable medium can also be a computer memory that can be any available memory or storage medium. The computer-readable medium can also be a computer application, data file, data structure, table, database, etc. that can be any available computer-readable medium. The computer-readable medium can also be a computer program product that can be any available computer program product. The computer-readable medium can also be a computer program that can be any available computer program. The computer-readable medium can also be a computer program product that can be any available computer program product. The computer-readable medium can also be a computer program that can be any available computer program.

[0205] It should be understood that each part of the present application can be realized by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be realized by software or firmware stored in a memory and executed by a suitable instruction execution system. As in another embodiment, if realized by hardware, any one or a combination of the following technologies known in the art can be used: discrete logic circuit with logic gate circuit for implementing logic function on data signal, application specific integrated circuit with suitable combination logic gate circuit, programmable gate array (PGA), field programmable gate array (FPGA) and the like.

[0206] Those skilled in the art of the present technology can understand that all or part of the steps carried out by the above-mentioned embodiment method can be completed by a program instructing the relevant hardware, and the program can be stored in a computer readable storage medium. When the program is executed, it includes one of the steps of the method embodiment or a combination thereof.

[0207] In addition, each functional unit in each embodiment of the present application can be integrated in one processing module, or each unit can exist physically, or two or more units can be integrated in one module. The above-mentioned integrated module can be realized in the form of hardware or in the form of software functional module. The integrated module, if realized in the form of software functional module and sold or used as an independent product, can also be stored in a computer readable storage medium.

[0208] The above-mentioned storage medium can be read-only memory, disk or optical disk, etc. Although the embodiments of the present application have been shown and described above, it should be understood that the above-mentioned embodiments are exemplary and cannot be understood as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above-mentioned embodiments within the scope of the present application.

Claims

1. An image processing method, characterized in that, include: The depth and focal length information corresponding to the image to be processed are fused to obtain fused information; The fused information is subjected to feature extraction and feature enhancement processing to obtain target features; Based on the target features, determine the blur radius of each pixel in the image to be processed; Based on the blur radius of each pixel, the image to be processed is blurred to obtain the target image.

2. The method according to claim 1, characterized in that, The depth information is presented in the form of a depth map, and the pixel value of each pixel in the depth map is used to indicate the depth of the corresponding pixel in the image to be processed. The depth information and focal length information corresponding to the image to be processed are fused to obtain fused information, including: Perform a downsampling operation on the depth map to obtain a sampled depth map; The sampled depth map and the focal length information are fused to obtain the fused information.

3. The method according to claim 2, characterized in that, The step of determining the blur radius of each pixel in the image to be processed based on the target features includes: The target features are mapped to a dimension that matches the sampling depth map to obtain an initial blur radius map; wherein the initial blur radius map matches the size of the sampling depth map; An upsampling operation is performed on the initial blur radius map to obtain the target blur radius map; The target blur radius map is sized to match the depth map, and the pixel values ​​of the pixels in the target blur radius map are used to indicate the blur radius of the corresponding pixel in the image to be processed.

4. The method according to claim 3, characterized in that, The step of performing an upsampling operation on the initial blur radius map to obtain the target blur radius map includes: An upsampling operation is performed on the initial bokeh radius map to obtain a first bokeh radius map; Perform a rounding operation on the first blurred radius map to obtain the second blurred radius map; Based on the set maximum and minimum blur radius, a truncation operation is performed on the second blur radius map to obtain the target blur radius map.

5. The method according to claim 3, characterized in that, The step of blurring the image to be processed based on the blur radius of each pixel to obtain the target image includes: Traverse each pixel in the target blur radius map; Based on the pixel value of the currently traversed pixel, determine the target blur radius of the corresponding pixel in the image to be processed; Based on the target blur radius, the region to be blurred in the image to be processed is blurred to obtain the target image.

6. The method according to claim 5, characterized in that, The step of blurring the region to be blurred in the image to be processed, based on the target blur radius, to obtain the target image, includes: Based on the target blur radius, determine the Gaussian kernel and the region to be blurred; Based on the Gaussian kernel, the area to be blurred is subjected to Gaussian blurring to obtain the target image.

7. The method according to any one of claims 1-6, characterized in that, The screen to be processed includes at least one of the following: The game screen to be rendered; The captured image in response to the shooting operation; The video frame to be rendered in the video; In response to a selection operation, select an image resource from local storage resources.

8. The method according to claim 7, characterized in that, In response to the fact that the image to be processed includes a game image, before fusing the depth information and focal length information corresponding to the image to be processed to obtain fused information, the method further includes: The game screen is preprocessed; wherein the preprocessing includes at least one of the following: vertex shading, rasterization, and lighting rendering.

9. The method according to claim 8, characterized in that, The method further includes: Perform a graphical user interface rendering operation on the target screen to display it.

10. An image processing apparatus, characterized in that, include: The fusion module is used to fuse the depth information and focal length information of the image to be processed to obtain fused information. The processing module is used to perform feature extraction and feature enhancement processing on the fused information to obtain target features; The determining module is used to determine the blur radius of each pixel in the image to be processed based on the target features; The blurring module is used to blur the image to be processed according to the blurring radius of each pixel to obtain the target image.

11. The apparatus according to claim 10, characterized in that, The depth information is presented in the form of a depth map, and the pixel value of each pixel in the depth map is used to indicate the depth of the corresponding pixel in the image to be processed. The fusion module is used for: Perform a downsampling operation on the depth map to obtain a sampled depth map; The sampled depth map and the focal length information are fused to obtain the fused information.

12. The apparatus according to claim 11, characterized in that, The determining module is used for: The target features are mapped to a dimension that matches the sampling depth map to obtain an initial blur radius map; wherein the initial blur radius map matches the size of the sampling depth map; An upsampling operation is performed on the initial blur radius map to obtain the target blur radius map; The target blur radius map is sized to match the depth map, and the pixel values ​​of the pixels in the target blur radius map are used to indicate the blur radius of the corresponding pixel in the image to be processed.

13. The apparatus according to claim 12, characterized in that, The determining module is used for: An upsampling operation is performed on the initial bokeh radius map to obtain a first bokeh radius map; Perform a rounding operation on the first blurred radius map to obtain the second blurred radius map; Based on the set maximum and minimum blur radius, a truncation operation is performed on the second blur radius map to obtain the target blur radius map.

14. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, it implements the steps of the method as described in any one of claims 1 to 9.

15. A non-transitory computer-readable storage medium having computer program instructions stored thereon, characterized in that, When executed by a processor, the program instructions implement the steps of the method according to any one of claims 1 to 9.

16. A chip, characterized in that, The chip includes an interface circuit and a processing circuit that are coupled to each other. The interface circuit is used to input or output signals, and the processing circuit is used to implement the method of any one of claims 1 to 9.

17. A computer program product, characterized in that, It includes a computer program, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 9.

Images