Image processing method and device, electronic equipment and storage medium

By dividing the image frame into multiple pixel blocks and determining the motion vector of each pixel block, the image frame is blurred, and the problem of large amount of motion blur image generation operations in the prior art is solved, and efficiency is improved.

CN118941677APending Publication Date: 2024-11-12NANJING OPPO SOFTWARE TECH CORP LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310534750.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-11
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

The prior art has a large amount of calculation when generating motion blur images, resulting in low efficiency.

Method used

By dividing the continuous image frames of multiple frames into multiple pixel blocks, the motion vector of each pixel block is obtained, and the target area is blurred based on these vectors, thereby reducing the calculation amount of the motion vector.

Benefits of technology

The calculation amount when generating motion blur images is reduced, and the generation efficiency of motion blur images is improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118941677A_ABST
    Figure CN118941677A_ABST
Patent Text Reader

Abstract

The invention discloses an image processing method and device, electronic equipment and a storage medium. The image processing method comprises the steps that a target image frame in multiple continuous image frames is divided into multiple pixel blocks; a motion vector corresponding to each pixel block in the plurality of pixel blocks is obtained, and the motion vector corresponding to each pixel block is determined according to the motion vectors corresponding to part of pixel points in each pixel block; and based on the motion vector corresponding to each pixel block, performing fuzzy processing on a target area in the target image frame to obtain a fuzzy image corresponding to the target image frame, the target area being a foreground area in the target image frame or a background area except the foreground area. The method can reduce the operand of motion blur drawing.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and more specifically, to an image processing method, device, electronic device and storage medium. Background Art

[0002] Motion blur is a common blur effect in image blurring. Motion blur refers to the blurry dragging traces in the picture caused by the movement of the subject, which is an important part of visual production. However, in the related technologies for generating motion blurred images, it usually consumes a large amount of calculation. Summary of the invention

[0003] The present application proposes an image processing method, device, electronic device and storage medium, which can reduce the computational complexity of motion blur rendering.

[0004] In a first aspect, an embodiment of the present application provides an image processing method, comprising: dividing a target image frame in a plurality of continuous image frames into a plurality of pixel blocks; obtaining a motion vector corresponding to each pixel block in the plurality of pixel blocks, wherein the motion vector corresponding to each pixel block is determined based on a motion vector corresponding to a portion of pixels in each pixel block; and blurring a target area in the target image frame based on the motion vector corresponding to each pixel block to obtain a blurred image corresponding to the target image frame, wherein the target area is a foreground area in the target image frame or a background area other than the foreground area.

[0005] In a second aspect, an embodiment of the present application provides an image processing device, comprising: an image division module, a vector acquisition module and an image blur module, wherein the image division module is used to divide a target image frame in multiple consecutive image frames into multiple pixel blocks; the vector acquisition module is used to obtain a motion vector corresponding to each pixel block in the multiple pixel blocks, wherein the motion vector corresponding to each pixel block is determined based on the motion vector corresponding to a part of the pixels in each pixel block; the image blur module is used to blur a target area in the target image frame based on the motion vector corresponding to each pixel block to obtain a blurred image corresponding to the target image frame, wherein the target area is a foreground area in the target image frame or a background area other than the foreground area.

[0006] In a third aspect, an embodiment of the present application provides an electronic device, comprising: one or more processors; a memory; one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, and the one or more applications are configured to execute the image processing method provided in the first aspect above.

[0007] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a program code is stored. The program code can be called by a processor to execute the image processing method provided in the first aspect above.

[0008] The solution provided by the present application is to divide a target image frame in a plurality of continuous image frames into a plurality of pixel blocks, obtain a motion vector corresponding to each pixel block in the plurality of pixel blocks, wherein the motion vector corresponding to each pixel block is determined according to the motion vector corresponding to a portion of the pixels in each pixel block; based on the motion vector corresponding to each pixel block, blur the target area in the target image frame to obtain a blurred image corresponding to the target image frame, wherein the target area is the foreground area in the target image frame or the background area other than the foreground area. Thus, after dividing the image frame into a plurality of pixel blocks, the motion vector of the pixel block is determined, and the image frame is blurred according to the determined motion vector of the pixel block, thereby reducing the amount of calculation when determining the motion vector, and further reducing the amount of calculation when generating a motion blurred image, thereby improving the generation efficiency of the motion blurred image. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0010] Figure 1 A flow chart of an image processing method according to an embodiment of the present application is shown.

[0011] Figure 2 A flow chart of an image processing method according to another embodiment of the present application is shown.

[0012] Figure 3 A flow chart of an image processing method according to yet another embodiment of the present application is shown.

[0013] Figure 4 A flow chart of an image processing method according to another embodiment of the present application is shown.

[0014] Figure 5 A schematic diagram of pixel block division provided in an embodiment of the present application is shown.

[0015] Figure 6 A flow chart of an image processing method according to yet another embodiment of the present application is shown.

[0016] Figure 7A schematic diagram of pixel weight setting provided in an embodiment of the present application is shown.

[0017] Figure 8 A schematic diagram of pixel weight setting provided in an embodiment of the present application is shown.

[0018] Fig. 9 A schematic diagram of pixel weight setting provided in an embodiment of the present application is shown.

[0019] Fig.10 A schematic diagram of pixel weight setting provided in an embodiment of the present application is shown.

[0020] Fig.11 A schematic diagram of an effect provided by an embodiment of the present application is shown.

[0021] Fig.12 A schematic diagram of an effect provided by an embodiment of the present application is shown.

[0022] Fig.13 A block diagram of an image processing device according to an embodiment of the present application is shown.

[0023] Fig.14 It is a block diagram of an electronic device according to an embodiment of the present application for executing the image processing method according to an embodiment of the present application.

[0024] Fig.15 It is a storage unit of an embodiment of the present application for storing or carrying a program code for implementing the image processing method according to an embodiment of the present application. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0026] Drawing motion blur is an important part of visual effects production. Motion blur refers to the blurred dragging traces in the picture caused by the movement of the subject. Fast-moving objects will jitter without motion blur effect, jumping multiple pixels between frames. It is essentially a problem of time domain sampling, so motion blur can be considered as anti-aliasing in the time domain. Motion blur relies on relative motion. If an object moves from left to right on the screen, it will be blurred horizontally on the screen.

[0027] When generating an image with motion blur effect, the motion blur effect can usually be captured by a camera or produced by image processing technology. In the related technology of producing motion blur effect by image processing, it is usually necessary to calculate the blur direction and blur range for each pixel point by calculating motion vector, optical flow method, etc., and then determine the pixel value to be adjusted for each pixel point according to the determined blur direction and blur range. In this way, it will consume a large amount of calculation when generating motion blurred images.

[0028] In view of the above problems, the inventors have proposed the image processing method, device, electronic device and storage medium provided in the embodiments of the present application, which can realize after dividing the image frame into multiple pixel blocks, by determining the motion vector of the pixel block, and blurring the image frame according to the determined motion vector of the pixel block, thereby reducing the amount of calculation when determining the motion vector, and further reducing the amount of calculation when generating a motion blurred image, thereby improving the generation efficiency of the motion blurred image. Among them, the specific image processing method is described in detail in the subsequent embodiments.

[0029] The image processing method provided in the embodiment of the present application is described in detail below with reference to the accompanying drawings.

[0030] See also Figure 1 , Figure 1 FIG. 1 is a flow chart of an image processing method provided by an embodiment of the present application. In a specific embodiment, the image processing method is applied to Fig.13 The image processing device 600 and the electronic device 100 ( Fig.14 ). The following will take an electronic device as an example to illustrate the specific process of this embodiment. Of course, it can be understood that the electronic device used in this embodiment can be a smart phone, a tablet computer, a smart watch, an e-book, etc., which is not limited here. Figure 1 The process shown is described in detail, and the image processing method may specifically include the following steps:

[0031] Step S110: dividing a target image frame among a plurality of continuous image frames into a plurality of pixel blocks.

[0032] The target image frame may be an image frame for which a motion blur effect is to be generated, and the target image frame may be one of the multiple continuous image frames mentioned above. It can be understood that when generating a motion blur effect for an image, it is necessary to determine the motion vectors at different positions in the image, so as to determine the blur direction and blur range at different positions, and when determining the motion vectors at different positions in the image, it is usually necessary to determine the motion vectors based on at least one adjacent image frame, so the target image frame for which a motion blur effect is to be generated may be one of the multiple continuous image frames, so as to determine the motion vectors at different positions in the target image frame based on other image frames adjacent to the target image frame.

[0033] In a possible implementation, the above multiple image frames may be continuous image frames in a generated video. In this case, the application scenario in which the electronic device performs blur processing on the image may be a video processing scenario. For example, for the video to be processed, the continuous frames in the video generate jitters due to frame jumps, so a motion blur effect may be added to the image frames in the video, so that the jitters may be eliminated in the processed video image. The video may be a video captured by the electronic device through its image acquisition device, or a video obtained from other devices, such as a video downloaded from a server.

[0034] In a possible implementation, the above multiple image frames may also be image frames collected during the process of shooting a video, that is, the application scenario in which the electronic device performs blur processing on an image may be a video shooting scenario. When the electronic device shoots a video, a motion blur effect may be added to each frame of the collected video image, so that the captured video image does not have jitter caused by jumping between frames.

[0035] In a possible implementation, the above multiple image frames may also be image frames that constitute an animation generated during animation production, that is, the application scenario in which the electronic device performs blur processing on an image may be an animation production scenario. When performing animation production, a motion blur effect may be added to continuous animation image frames, so that there is no jitter in the animation image caused by jumping between frames.

[0036] In a possible implementation, the above multiple image frames may also be multiple preview images captured by the image acquisition device of the electronic device when taking a photo. In this case, the target image frame may be the image captured when the electronic device receives the shooting operation. According to the preview image frame before the target image frame, the motion vectors at different positions in the target image frame may be determined, and then the target image frame may be blurred according to the determined motion vector, so that the captured image has a motion blur effect. In this way, it is possible to capture an image with a motion blur effect in a photo shooting scene.

[0037] In an embodiment of the present application, when blur processing is performed on a target image frame so that the target image frame has a motion blur effect, the target image frame can be divided into multiple pixel blocks, that is, divided into multiple blocks, so that the motion vectors at different positions in the target image frame can be subsequently determined in units of pixel blocks, and then the target image frame can be blurred with a motion blur effect according to the motion vectors of each pixel block.

[0038] In some embodiments, when the target image frame is divided into a plurality of pixel blocks, the target image frame can be divided according to a target division rule. Optionally, the electronic device can divide the target image frame into a plurality of pixel blocks of the same size according to the target division rule; optionally, the electronic device can also divide different regions in the target image frame into pixel blocks of different sizes according to the importance of the detail display corresponding to the different regions, wherein the higher the importance of the detail display, the smaller the size corresponding to the region, so as to perform more sophisticated blurring processing on it. Of course, the specific division rule of the pixel blocks in the embodiment of the present application may not be limited.

[0039] Step S120: Obtain a motion vector corresponding to each pixel block in the plurality of pixel blocks, wherein the motion vector corresponding to each pixel block is determined based on motion vectors corresponding to some pixel points in each pixel block.

[0040] In an embodiment of the present application, after the above target image frame is divided into a plurality of pixel blocks, the motion vector corresponding to each pixel block can be determined for each pixel block in the target image frame, and when determining the motion vector corresponding to each pixel block, the motion vector of each pixel block is determined according to the motion vectors of some pixels in each pixel block, and the determined motion vector corresponding to the pixel block represents the motion vector corresponding to each pixel in the pixel block. Among them, the motion vector of some pixels in each pixel block can be the motion vector of one pixel or the motion vectors of multiple pixels, which is not limited here. Therefore, when blurring the target image frame with motion blur effect, it is not necessary to determine the motion vector for each pixel, thereby reducing the amount of calculation when generating an image with motion blur effect.

[0041] In some embodiments, when determining the motion vector corresponding to each pixel block, the motion vectors of at least some of the pixels in each pixel block may be determined first, and then the motion vector corresponding to the pixel block may be determined based on the determined motion vectors of at least some of the pixels. When determining the motion vector of any pixel, the motion vector corresponding to the pixel may be determined based on the first coordinate position of the pixel in the target image frame and the second coordinate position of the pixel matching the pixel in the image frame adjacent to the target image frame.

[0042] In a possible implementation, the displacement of the pixel point on the horizontal axis can be determined as a first displacement based on the horizontal coordinate corresponding to the second coordinate position and the horizontal coordinate corresponding to the first coordinate position, and the displacement of the pixel point on the vertical axis can be determined as a second displacement based on the vertical coordinate corresponding to the second coordinate position and the vertical coordinate corresponding to the first coordinate position. Based on the first displacement and the second displacement, a motion vector corresponding to the pixel point can be obtained.

[0043] Step S130: Based on the motion vector corresponding to each pixel block, blurring the target area in the target image frame to obtain a blurred image corresponding to the target image frame, wherein the target area is the foreground area in the target image frame or the background area other than the foreground area.

[0044] In an embodiment of the present application, after obtaining the motion vector corresponding to each pixel block in the target image frame, the target area (foreground area or background area) in the target image frame can be blurred based on the motion vector corresponding to each pixel block, thereby obtaining a blurred image corresponding to the target image frame. It can be understood that when adding a blur effect to an image, a motion blur effect can be added to a moving foreground, or a blur effect can be added to a background. Therefore, according to the actual application scenario, the foreground area in the target image frame or the background area other than the foreground area can be blurred to obtain a blurred image with a motion blur effect. Among them, the foreground area can refer to the image area where the target object in the target image frame is located, and the background area can refer to the image area other than the target object in the target image frame; the target object can be a moving object of interest in the target image frame, for example, the target image frame can be a street view image, and the target object can be a vehicle in the street view image.

[0045] In some embodiments, when blurring a target area in a target image frame based on a motion vector corresponding to each pixel block, blur parameters (such as blur direction and blur range) corresponding to each pixel point in the target image frame to which a motion blur effect needs to be added can be determined according to the motion vector corresponding to each pixel block, and then the pixel value of each pixel point to which a motion blur effect needs to be added is adjusted according to the blur range and blur direction, so that the target image frame after blurring has a motion blur effect.

[0046] In some embodiments, in the process of blurring the target image frame to add a motion blur effect, the electronic device can perform foreground identification on the target image frame to obtain the foreground area in the target image frame (i.e., the foreground area), and the background area in the target image frame excluding the foreground area; based on the identified foreground area and background area, the above target area in the target image frame can be blurred based on the motion vector corresponding to each of the above pixel blocks, thereby obtaining a blurred image with a motion blur effect.

[0047] The image processing method provided in the embodiment of the present application, when performing motion blur processing on the target image frame, divides the image frame into multiple pixel blocks, determines the motion vector of the pixel block, and blurs the image frame according to the determined motion vector of the pixel block, thereby reducing the amount of calculation in determining the motion vector in the process of generating a blurred image with motion blur effect, thereby reducing the amount of calculation in generating the motion blurred image, and improving the generation efficiency of the motion blurred image.

[0048] See also Figure 2 , Figure 2 A schematic diagram of a flow chart of an image processing method provided by another embodiment of the present application is shown. The image processing method is applied to the above electronic device. Figure 2 The process shown is described in detail, and the image processing method may specifically include the following steps:

[0049] Step S210: Divide a target image frame among a plurality of continuous image frames into a plurality of pixel blocks.

[0050] In the embodiment of the present application, step S210 can refer to the contents of other embodiments and will not be repeated here.

[0051] Step S220: for each pixel block in the plurality of pixel blocks, selecting a target number of pixel points from the pixel block, where the target number is smaller than the total number of pixel points in the pixel block.

[0052] In an embodiment of the present application, when determining the corresponding motion vector for each pixel block obtained by division, a target number of pixel points can be selected from the pixel block for each pixel block, and the target number is less than the total number of pixel points in the pixel block. That is to say, for each pixel block, some pixel points are selected from each pixel block so that the motion vector corresponding to each pixel block can be determined subsequently based on the selected pixel points.

[0053] In some embodiments, the above target number may be 1. In this case, when selecting the pixel points for which the motion vector needs to be determined for each pixel block, the electronic device may select any one pixel point from each pixel block for each of the above multiple pixel blocks. That is, for each pixel block, its corresponding motion vector is determined based on the motion vector of one of the pixel points. Thus, in the process of determining the motion vector corresponding to each pixel block, it is only necessary to determine the motion vector corresponding to one pixel point for each pixel block, which can greatly reduce the amount of calculation when generating a motion blurred image.

[0054] In a possible implementation, when the target number is 1, the electronic device can obtain a pixel located in the central area of ​​the pixel block when selecting one pixel point for each pixel block. Since the pixel point located in the central area of ​​the pixel block is usually more representative of the motion of the pixel block, the accuracy of the motion vector corresponding to the determined pixel block can be improved by selecting the pixel point in the central area of ​​the pixel block as the pixel point for which the motion vector needs to be determined.

[0055] In some embodiments, the above target number may also be multiple. In this case, when selecting the pixel points for which the motion vector needs to be determined for each pixel block, the electronic device may select the target number of pixel points evenly distributed in each pixel block for each pixel block. In this manner, since the multiple pixel points evenly distributed in the pixel block can better reflect the motion conditions at different positions in the pixel block, the subsequent motion vectors corresponding to these pixel points can improve the accuracy of the motion vector corresponding to the determined pixel block. Exemplarily, the shape of the pixel block is a rectangle, the above target number is 4, and 1 pixel point can be selected from the four corners of the pixel block.

[0056] Step S230: Obtain a motion vector corresponding to each pixel of the target number of pixels.

[0057] In an embodiment of the present application, after selecting a target number of pixel points for each pixel block, the motion vector corresponding to each pixel point can be determined for the target number of pixel points selected in each pixel block, so as to further determine the motion vector of each pixel block based on the motion vector corresponding to the target number of pixel points.

[0058] In some embodiments, when determining the motion vector corresponding to each pixel point in the target number of pixel points selected in each pixel block, the motion vector corresponding to each pixel point can be determined based on the position of the pixel point corresponding to each pixel point in the previous image frame of the target image frame and the position of each pixel point in the target image frame. The pixel point corresponding to the pixel point in the pixel block in the target image frame and the previous image frame refers to the pixel point that the pixel point in the pixel block matches in the previous image frame.

[0059] In a possible implementation, when the electronic device determines the motion vector corresponding to each pixel point in the target number of pixels for each pixel block, it can determine the motion vector corresponding to the pixel point according to the first coordinate position of each pixel point in the target image frame and the second coordinate position of the pixel point matched by the pixel point in the previous image frame. Among them, the displacement of the pixel point on the horizontal axis can be determined as the first displacement according to the horizontal coordinate corresponding to the second coordinate position and the horizontal coordinate corresponding to the first coordinate position, and the displacement of the pixel point on the vertical axis can be determined as the second displacement according to the vertical coordinate corresponding to the second coordinate position and the vertical coordinate corresponding to the first coordinate position, and the motion vector corresponding to the pixel point can be obtained based on the first displacement and the second displacement.

[0060] Step S240: Determine the motion vector corresponding to the pixel block based on the motion vector corresponding to each pixel point.

[0061] In an embodiment of the present application, after determining the motion vector corresponding to each pixel point for a target number of pixel points in each pixel block, the electronic device can determine the motion vector corresponding to each pixel block based on the determined motion vectors corresponding to the above pixel points in each pixel block.

[0062] In some embodiments, when the target number is 1, for each pixel block, the motion vector corresponding to the pixel point obtained by each pixel block can be used as the motion vector corresponding to the pixel block; when the target number is multiple, for each pixel block, the motion vectors of the target number of pixel points can be averaged to obtain the motion vector corresponding to each pixel block. Of course, the motion vectors of the target number can also be weighted summed according to the weights corresponding to different pixel points set to obtain the motion vector corresponding to the pixel block.

[0063] Step S250: Based on the motion vector corresponding to each pixel block, blur the target area in the target image frame to obtain a blurred image corresponding to the target image frame, where the target area is the foreground area in the target image frame or the background area other than the foreground area.

[0064] In the embodiment of the present application, step S250 can refer to the contents of other embodiments and will not be repeated here.

[0065] The image processing method provided by the embodiment of the present application, when performing motion blur processing on the target image frame, divides the image frame into multiple pixel blocks, determines the motion vector of some pixels in the pixel block, then determines the motion vector of the pixel block according to the motion vector of some pixels in the pixel block, and then blurs the image frame according to the determined motion vector of the pixel block. Therefore, in the process of generating a blurred image with motion blur effect, there is no need to determine the motion vector for each pixel in the image frame, thereby reducing the amount of calculation when determining the motion vector, reducing the amount of calculation when generating the motion blurred image, and improving the generation efficiency of the motion blurred image.

[0066] See also Figure 3 , Figure 3 FIG. 1 is a flow chart of an image processing method provided by another embodiment of the present application. The image processing method is applied to the above electronic device. Figure 3 The process shown is described in detail, and the image processing method may specifically include the following steps:

[0067] Step S310: performing foreground recognition on the target image frame to obtain a foreground area in the target image frame and a background area other than the foreground area.

[0068] In an embodiment of the present application, when the electronic device performs blur processing to add a motion blur effect on a target image frame, it may first perform foreground recognition on the target image frame to obtain a foreground area in the target image frame and a background area excluding the foreground area, so as to subsequently divide the target image frame into pixel blocks based on the foreground recognition result, and blur the foreground area or background area in the target image frame based on the foreground recognition result.

[0069] In some embodiments, the electronic device performs foreground recognition on the target image frame to obtain a foreground recognition result, and the foreground recognition result can be used to mark the foreground area in the target image frame, so that based on the foreground recognition result, the foreground area in the target image frame and the background area other than the foreground area can be determined.

[0070] In a possible implementation, the electronic device may extract image features of the target image frame and analyze the image features of the target image frame to determine the foreground area of ​​the target image frame. Optionally, the image features may include but are not limited to edge features, color features, position features, etc.

[0071] In a possible implementation, the electronic device may also input the target image frame into a pre-trained foreground recognition model, and identify the target object contained in the target image frame through the foreground recognition model to obtain the foreground area corresponding to the target object. The foreground recognition model may be trained based on multiple groups of sample training images, each group of sample training images may include sample images, and each sample image may be annotated with a foreground area. The foreground recognition model may include but is not limited to a network based on the deeplab semantic segmentation algorithm, a U-Net network structure, FCN (Fully Convolutional Networks, Fully Convolutional Neural Networks), etc., which are not limited here.

[0072] It should be noted that the electronic device may also use other methods to perform foreground recognition on the target image frame, and the specific method of foreground recognition may not be limited in the embodiment of the present application.

[0073] Step S320: Based on the foreground area and the background area, the target image frame is divided into a plurality of pixel blocks, wherein the size of the pixel blocks in the foreground area is smaller than the size of the pixel blocks in the background area.

[0074] In an embodiment of the present application, when dividing a target image frame into a plurality of pixel blocks, the target image frame can be divided into a plurality of pixel blocks based on the foreground area and the background area in the target image frame determined above, and the divided pixel blocks satisfy: the size of the pixel blocks in the foreground area is smaller than the size of the pixel blocks in the background area. It can be understood that since the details that need to be displayed in the background area are less than the details that need to be displayed in the foreground area, when dividing the pixel blocks, the size of the pixel blocks in the background area can be made larger than the size of the pixel blocks in the foreground area, so that the motion vectors at different positions in the foreground area are more accurate, and then the foreground area can be blurred more finely later, and the use of relatively large pixel blocks in the background area can further reduce the amount of calculation consumed in determining the motion vectors at different positions in the background area.

[0075] In some embodiments, a first size corresponding to a pixel block in a foreground area and a second size corresponding to a pixel block in a background area may be predetermined, and the second size is larger than the first size. When dividing the target image frame into pixel blocks, the foreground area in the target image frame may be divided into pixel blocks of a first size, and the background area may be divided into pixel blocks of a second size.

[0076] In some embodiments, the first size corresponding to the pixel block in the foreground area may include multiple sizes, and the second size corresponding to the pixel block in the background area may also include multiple sizes. When dividing the foreground area in the target image frame into pixel blocks, the importance of detail display corresponding to different content areas in the foreground area can be identified, and the foreground area in the target image frame is divided into pixel blocks according to the importance of detail display corresponding to different content areas in the foreground area; similarly, when dividing the background area in the target image frame into pixel blocks, the importance of detail display corresponding to different content areas in the background area can be identified, and the background area in the target image frame is divided into pixel blocks according to the importance of detail display corresponding to different content areas in the background area; wherein, the higher the importance of detail display, the smaller the size of the pixel blocks in the content area, so as to perform more refined blurring processing on it.

[0077] Step S330: Obtain a motion vector corresponding to each pixel block in the multiple pixel blocks, wherein the motion vector corresponding to each pixel block is determined according to motion vectors corresponding to some pixel points in each pixel block.

[0078] Step S340: Based on the motion vector corresponding to each pixel block, blur the target area in the target image frame to obtain a blurred image corresponding to the target image frame, where the target area is the foreground area in the target image frame or the background area other than the foreground area.

[0079] In the embodiment of the present application, step S330 and step S340 can refer to the contents of other embodiments and will not be repeated here.

[0080] The image processing method provided by the embodiment of the present application, when performing motion blur processing on a target image frame, divides the image frame into multiple pixel blocks, determines the motion vector of the pixel block, and blurs the image frame according to the determined motion vector of the pixel block, thereby reducing the amount of calculation when determining the motion vector in the process of generating a blurred image with a motion blur effect, thereby reducing the amount of calculation when generating the motion blurred image and improving the generation efficiency of the motion blurred image; in addition, when dividing the pixel blocks, the size of the pixel blocks in the foreground area is made smaller than the size of the pixel blocks in the background area, thereby allowing the foreground area to be more finely blurred and further reducing the amount of calculation consumed in determining the motion vectors at different positions in the background area.

[0081] See also Figure 4 , Figure 4 FIG. 1 is a flow chart of an image processing method provided by another embodiment of the present application. The image processing method is applied to the above electronic device. Figure 4The process shown is described in detail, and the image processing method may specifically include the following steps:

[0082] Step S410: performing depth estimation on the content in the target image frame to obtain depth information of the image content in the target image frame.

[0083] In the embodiment of the present application, when the electronic device divides the target image frame into a plurality of pixel blocks, it can also perform depth estimation on the content in the target image frame to obtain depth information of the image content in the target image frame, so as to divide the content areas of different depths in the target image frame into pixel blocks of different sizes according to the determined depth information. The depth information can be used to characterize the distance between the photographed object and the camera, and the greater the depth information, the farther the distance.

[0084] In some embodiments, the electronic device may perform depth estimation on the target image frame by monocular image depth estimation, thereby obtaining depth information of the image content in the target image frame. Alternatively, the electronic device may obtain the depth information of the image content in the target image frame by inputting the target image frame into a pre-trained depth estimation model.

[0085] Exemplarily, the depth estimation model can be a Monodepth2 model, which uses an unsupervised learning method based on monocular images to complete the task of depth estimation. When training the Monodepth2 model, two sets of networks are trained simultaneously in the convolutional neural network according to the principle of the SFM model: a deep network and a pose network. The input of the training network is a continuous multi-sequential image of a video sequence. The deep network inputs the target view, and the pose network inputs the target view and the previous sequential view. The deep network is processed by the convolutional neural network to output the corresponding depth image. The pose network calculates the change in the camera motion posture. The reprojected image is jointly constructed based on the output results of the two sets of networks, and the reprojection error is calculated and introduced into the loss function. The model parameters are updated through the loss function to backpropagate and optimize the training network model.

[0086] Step S420: Based on the depth information, the target image frame is divided into a plurality of pixel blocks, wherein the size of the pixel block is positively correlated with the depth of the image content corresponding to the pixel block.

[0087] In an embodiment of the present application, after the depth information of the image content in the target image frame is obtained, the target image frame can be divided into a plurality of pixel blocks based on the obtained depth information, and the divided pixel blocks satisfy: the size of the pixel block is positively correlated with the depth of the image content corresponding to the pixel block, that is, in the target image frame, the greater the depth of the content area, the larger the size of the pixel block in the content area, and conversely, the smaller the depth of the content area, the smaller the size of the pixel block in the content area. It can be understood that when the depth corresponding to the image content is deeper, the fewer details it needs to display, so when dividing the pixel blocks, the size of the pixel blocks can be made positively correlated with the depth of the image content, thereby further reducing the amount of calculation consumed in determining the motion vectors at different positions.

[0088] In some embodiments, a mapping relationship between depth information of different sizes and the size of a pixel block may be predetermined, and the mapping relationship satisfies that the size of the pixel block is positively correlated with the size of the depth information. When the electronic device divides the target image frame into a plurality of pixel blocks based on the acquired depth information, the size of the pixel blocks in the content area of ​​different depths may be determined according to the mapping relationship, and the target image frame may be divided into pixel blocks according to the determined size of the pixel blocks.

[0089] It should be noted that the method for dividing the target image frame into pixel blocks provided in the embodiment of the present application can also be combined with the method for dividing the target image frame into pixel blocks provided in the previous embodiment. Among them, the electronic device can perform foreground recognition on the target image frame to obtain the foreground area and the background area in the target image frame, and can also perform depth estimation on the image content in the target image frame to obtain the depth information of the image content in the target image frame, and then divide the target image frame into multiple pixel blocks based on the recognized depth information, the foreground area and the background area in the target image frame, and among the pixel blocks obtained by division, the size of any pixel block in the foreground area is smaller than the size of any pixel block in the background area, and in the foreground area and the background area, the size of the pixel block is positively correlated with the depth of the image content.

[0090] For example, see Figure 5 ,exist Figure 5 In the image shown, the area where the vehicle is located is the foreground area, and the areas other than the area where the vehicle is located are the background areas. When dividing the pixel blocks, the size of the pixel blocks in the area where the vehicle is located (1*1 pixel blocks) is smaller than the size of any pixel blocks in the background area (2*1 pixel blocks, 4*4 pixel blocks, 4*2 pixel blocks, and any one of 2*2 pixel blocks). In addition, in the background area, the larger the depth of the area, the larger the size of the pixel block.

[0091] Step S430: Obtain a motion vector corresponding to each pixel block in the multiple pixel blocks, wherein the motion vector corresponding to each pixel block is determined according to motion vectors corresponding to some pixel points in each pixel block.

[0092] Step S440: Based on the motion vector corresponding to each pixel block, blur the target area in the target image frame to obtain a blurred image corresponding to the target image frame, where the target area is the foreground area in the target image frame or the background area other than the foreground area.

[0093] In the embodiment of the present application, step S430 and step S440 can refer to the contents of other embodiments and will not be repeated here.

[0094] The image processing method provided by the embodiment of the present application, when performing motion blur processing on the target image frame, after dividing the image frame into a plurality of pixel blocks, determines the motion vector of the pixel block, and performs blur processing on the image frame according to the determined motion vector of the pixel block, thereby reducing the amount of calculation when determining the motion vector in the process of generating a blurred image with motion blur effect, thereby reducing the amount of calculation when generating the motion blurred image, and improving the generation efficiency of the motion blurred image. In addition, when dividing the pixel blocks, the size of the pixel blocks in the foreground area is made smaller than the size of the pixel blocks in the background area, so that the foreground area can be more finely blurred, and the amount of calculation consumed in determining the motion vectors at different positions in the background area is further reduced.

[0095] See also Figure 6 , Figure 6 FIG. 1 is a flow chart of an image processing method provided by another embodiment of the present application. The image processing method is applied to the above electronic device. Figure 6 The process shown is described in detail, and the image processing method may specifically include the following steps:

[0096] Step S510: Divide a target image frame among a plurality of continuous image frames into a plurality of pixel blocks.

[0097] Step S520: Obtain a motion vector corresponding to each pixel block in the multiple pixel blocks, wherein the motion vector corresponding to each pixel block is determined based on motion vectors corresponding to some pixels in each pixel block.

[0098] In the embodiment of the present application, step S510 and step S520 can refer to the contents of the aforementioned embodiment and will not be described again here.

[0099] Step S530: Determine blur parameters corresponding to the target image frame based on the motion vector corresponding to each pixel block.

[0100] In an embodiment of the present application, after obtaining the motion vector corresponding to each pixel block in the target image frame, the electronic device can determine the blur parameter corresponding to the target image frame based on the motion vector corresponding to each pixel block when blurring the target area in the target image frame. The blur parameter can be the blur direction and blur range of the motion blur. The blur direction and blur range determine the pixel point to be referenced when adjusting the pixel value of the pixel point whose pixel value needs to be adjusted, so that the target area in the target image frame can be blurred subsequently according to the blur direction and blur range.

[0101] In some embodiments, the electronic device can determine the motion vector of the background area based on the motion vector corresponding to each pixel block; if the target area is the foreground area, the electronic device can determine the blur parameter corresponding to each pixel block in the foreground area and the blur parameter corresponding to the pixel block in the background area where the pixel points adjacent to the foreground area are located based on the motion vector corresponding to each pixel block in the foreground area and the motion vector of the background area; if the target area is the background area, the electronic device can determine the blur parameter corresponding to the background area based on the motion vector corresponding to each pixel block in the foreground area and the motion vector of the background area.

[0102] In this embodiment, since the background area is usually stationary, the determined motion vector of the pixel block in the background area may be 0. In this case, the background area can be taken as a whole, and the motion vector of the background area can be determined based on the motion vector corresponding to the determined pixel block, so as to determine the blur parameter of the background area as a whole and the blur parameters of different pixel blocks in the foreground area according to the motion vector of the background area; and, in the case of blurring the foreground area, there may be multiple motion areas in the foreground area, and the motion vectors corresponding to different motion areas are different, so the motion vector corresponding to each pixel block can be determined for different pixel blocks in the foreground area to ensure the accuracy of the blurring. In addition, when blurring the motion blur effect, a transitional blur effect is usually generated at the boundary between the foreground area and the background area, so the blur parameter corresponding to the pixel block in the background area adjacent to the foreground area can also be determined; in the case of blurring the background area, the background area can be taken as a whole to determine the blur parameter corresponding to the background area as a whole. In addition, when blurring the motion blur effect, a transitional blur effect is usually generated at the boundary between the foreground area and the background area, so the blur parameter corresponding to the pixel block in the background area adjacent to the foreground area can also be determined.

[0103] In a possible implementation, when blurring the foreground area, when determining the blur parameters corresponding to each pixel block in the foreground area, the difference between the motion vector corresponding to each pixel block in the foreground area and the motion vector of the background area can be obtained, and the direction represented by the obtained difference is used as the blur direction of each pixel block, and the modulus of the difference is used as the blur range of each pixel block; when determining the blur parameters corresponding to the pixel block where the pixel points in the background area adjacent to the foreground area are located, the direction represented by the motion vector of the background area can be used as the blur direction corresponding to the pixel block where the pixel points in the adjacent foreground area are located, and the modulus of the motion vector of the background area can be used as the blur range corresponding to the pixel block where the pixel points in the adjacent foreground area are located.

[0104] When blurring the background area, when determining the blur parameters corresponding to the background area, the direction represented by the motion vector of the background area can be used as the blur direction of the background area, and the modulus of the motion vector of the background area can be used as the blur range of the background area; when determining the blur parameters corresponding to the pixel block where the pixel points of the foreground area adjacent to the background area are located, the difference between the pixel block where the pixel points of the adjacent background area are located and the motion vector of the background area can be obtained, and the direction represented by the obtained difference can be used as the blur direction of the pixel block where the pixel points of the adjacent background area are located, and the modulus of the difference can be used as the blur range of the pixel block where the pixel points of the adjacent background area are located.

[0105] Step S540: Based on the blur parameter, blur the target area in the target image frame to obtain a blurred image corresponding to the target image frame, where the target area is the foreground area in the target image frame or the background area other than the foreground area.

[0106] In the embodiment of the present application, after the above blur parameters are determined, the target area in the target image frame can be blurred based on the determined blur parameters to obtain a blurred image corresponding to the target image frame. The pixel values ​​of each pixel point to which a motion blur effect is to be added can be adjusted according to the blur range and blur direction in the blur parameters, so that the target image frame after blur processing has a motion blur effect.

[0107] In some embodiments, the electronic device can determine the target pixel points to be adjusted for pixel values ​​based on the blur direction and the blur range, the target pixel points including the pixel points in the inner blur area and the pixel points in the outer blur area adjacent to the inner blur area, the inner blur area is the target area, and the outer blur area is the area other than the target area in the target image frame; based on the blur direction and the blur range, determine the blur window corresponding to each target pixel point; based on the pixel value of the pixel point in the blur window corresponding to each target pixel point, adjust the pixel value of each target pixel point to obtain a blurred image corresponding to the target image frame.

[0108] In which, when determining the target pixel point of the pixel value to be adjusted based on the blur direction and the blur range, all pixel points in the target area (that is, all pixel points in the inner blur area) are pixel points of the pixel value to be adjusted; and, based on the blur parameters of the pixel block where the pixel points in the outer blur area adjacent to the inner blur area are located, the pixel points of the adjusted pixel value in the outer blur area can be determined, wherein the pixel points in the area adjacent to the inner blur area in the outer blur area can be traversed, and for each traversed pixel point, if the blur window determined according to the blur direction and the blur range of the pixel block where it is located includes pixels in both the outer blur area and the inner blur area, it can be determined as the pixel point of the pixel value to be adjusted, if the blur window determined according to the blur direction and the blur range of the pixel block where it is located only includes pixels in the outer blur area, it will not be used as the pixel point of the pixel value to be adjusted.

[0109] In addition, for the above-determined target pixel points, the blur window corresponding to each of the target pixel points can be determined based on the above-determined blur direction and blur range. If the foreground area is blurred, the blur direction and blur range of the pixel block where each pixel point is located can be used as the blur direction and blur range corresponding to the pixel point, and based on the blur direction and blur range corresponding to the pixel point, the blur window corresponding to the pixel point is determined. In the blur window, the pixel point is located in the middle area of ​​the blur window, and the number of pixels included in the blur window (i.e., the size of the blur window) is consistent with the blur range, and the direction of the blur window is consistent with the blur direction (i.e., the straight line where the pixels in the blur window are located is parallel to the blur direction).

[0110] If the background area is blurred, when determining the blur window corresponding to the pixel in the inner blur area, the blur direction and blur range corresponding to the background area can be used as the blur direction and blur range corresponding to the pixel, and based on the blur direction and blur range corresponding to the pixel, the blur window corresponding to the pixel is determined, in which the pixel is located in the middle area of ​​the blur window, and the number of pixels included in the blur window (i.e., the size of the blur window) is consistent with the blur range, and the direction of the blur window is consistent with the blur direction (i.e., the straight line where the pixels in the blur window are located is parallel to the blur direction); when determining the blur window corresponding to the pixel in the outer blur area, the blur direction and blur range of the pixel block where each pixel is located can be used as the blur direction and blur range corresponding to the pixel, and based on the blur direction and blur range corresponding to the pixel, the blur window corresponding to the pixel is determined, in which the pixel is located in the middle area of ​​the blur window, and the number of pixels included in the blur window (i.e., the size of the blur window) is consistent with the blur range, and the direction of the blur window is consistent with the blur direction (i.e., the straight line where the pixels in the blur window are located is parallel to the blur direction).

[0111] In a possible implementation, when the pixel value of each target pixel is adjusted based on the pixel value of the pixel in the blurred window corresponding to each target pixel, the weight corresponding to each pixel in the blurred window corresponding to each target pixel can be determined for each target pixel; the target pixel value corresponding to each target pixel is determined based on the weight corresponding to each pixel in the blurred window and the pixel value of the pixel in the blurred window corresponding to each target pixel; the pixel value of each target pixel in the target image frame is adjusted to the target pixel value corresponding to each target pixel, so as to obtain a blurred image corresponding to the target image frame.

[0112] Optionally, for each target pixel, determining a weight corresponding to each pixel in the fuzzy window corresponding to each target pixel may include:

[0113] If the target pixel is in the inner blurred area, it is determined that the weight corresponding to each pixel in the blurred window is the first weight; if the target pixel is in the outer blurred area, it is determined that the weights of the pixels in the blurred window located in the inner blurred area are the first weight, the weight corresponding to the target pixel is the second weight, and the weights corresponding to the other pixels in the blurred window located in the outer blurred area except the target pixel are 0, and the ratio of the second weight to the first weight is the number of pixels in the blurred window located in the outer blurred area.

[0114] It can be understood that if the target pixel is in the inner blurred area, and the blurred window covers the pixels in the inner blurred area and the pixels in the outer blurred area, then the pixels in the inner blurred area covered are only part of the pixels that need to be referenced, and the remaining part needs to be impersonated by the pixels in the outer blurred area. Therefore, the weight corresponding to each pixel in the blurred window is the same and is the first weight; if the target pixel is in the outer blurred area, and the blurred window covers the pixels in the inner blurred area and the pixels in the outer blurred area, then the pixels in the outer blurred area covered are not all the pixels that need to be referenced, so it is necessary to impersonate the pixels in the inner blurred area. And because the pixels in the outer blurred area are clear (do not need to be blurred), the weights of the pixels in the outer blurred area except the target pixel in the blurred window are 0, and the weights of the pixels in the inner blurred area in the blurred window are all the first weights mentioned above, and the weight corresponding to the target pixel is the second weight, and the ratio of the second weight to the first weight is the number of pixels in the outer blurred area in the blurred window. For example, when the first weight is 1, the second weight can be the number of pixels in the outer blurred area in the blurred window, that is, the difference between the total number of pixels in the blurred window and the number of pixels in the inner blurred area in the blurred window.

[0115] For example, see Figure 7 , when the current pixel (any target pixel) is located in the inner blur area, if the blur range corresponding to the pixel block where the current pixel is located is 7 pixels, then the size of the blur window (the number of pixels contained in the blur window) is 7, and the straight line where the pixels in the blur window are located is parallel to the blur direction. At this time, the blur window covers the pixels in both the inner blur area and the outer blur area, and the weights of all pixels in the blur window are 1; please refer to Figure 8 , when the current pixel (any target pixel) is located in the outer blurred area, if the blur range corresponding to the pixel block where the current pixel is located is 7 pixels, then the size of the blur window (the number of pixels contained in the blur window) is 7, and the straight line where the pixels in the blur window are located is parallel to the blur direction. At this time, the blur window covers the pixels in both the inner and outer blurred areas. The weight of the pixels in the blur window located in the inner blurred area is 1, and the weight of the pixels in the outer blurred area except the current pixel is 0. The weight of the current pixel is the number of pixels in the blur window located in the inner blurred area, that is, 5; please refer to Fig. 9, when the current pixel (any target pixel) is located in the outer blurred area, if the blur range corresponding to the pixel block where the current pixel is located is 7 pixels, then the size of the blur window (the number of pixels contained in the blur window) is 7, and the straight line where the pixels in the blur window are located is parallel to the blur direction. At this time, the blur window covers the pixels in both the inner and outer blurred areas. The weight of the pixels in the blur window located in the inner blurred area is 1, and the weight of the pixels in the outer blurred area except the current pixel is 0. The weight of the current pixel is the number of pixels in the blur window located in the inner blurred area, that is, 4; please refer to Fig.10 , when the current pixel (any target pixel) is located in the outer blurred area, if the blur range corresponding to the pixel block where the current pixel is located is 7 pixels, the size of the blur window (the number of pixels contained in the blur window) is 7, and the straight line where the pixels in the blur window are located is parallel to the blur direction. At this time, the blur window covers the pixels in both the inner and outer blurred areas. The weight of the pixels in the blur window located in the inner blurred area is 1, and the weight of the pixels in the outer blurred area except the current pixel is 0. The weight of the current pixel is the number of pixels in the blur window located in the inner blurred area, that is, 6.

[0116] In the above implementation, when determining the target pixel value corresponding to each target pixel based on the weight corresponding to each pixel in the blur window and the pixel value of the pixel in the blur window corresponding to each target pixel, the pixel values ​​of all pixels in the blur window can be weighted averaged based on the weight corresponding to each pixel in the blur window, and the obtained value is used as the target pixel value.

[0117] For example, see Fig.11 and Fig.12 , Fig.11 This is a schematic diagram of the effect of blurring the background area in the target image frame using the image processing method provided in an embodiment of the present application. Fig.12 This is a schematic diagram of the effect of blurring the foreground area in the target image frame using the image processing method provided in an embodiment of the present application. It can be seen that the image processing method provided in an embodiment of the present application can not only reduce the amount of calculation in the processing process, but also ensure the generation effect of motion blur.

[0118] The image processing method provided by the embodiment of the present application, when performing motion blur processing on a target image frame, divides the image frame into multiple pixel blocks, determines the motion vector of the pixel block, and determines the blur parameter corresponding to the target image frame according to the determined motion vector of the pixel block, and blurs the image frame based on the determined blur parameter, thereby reducing the amount of calculation when determining the motion vector in the process of generating a blurred image with motion blur effect, thereby reducing the amount of calculation when generating a motion blurred image, and improving the generation efficiency of the motion blurred image.

[0119] See also Fig.13 , which shows a structural block diagram of an image processing device 600 provided in an embodiment of the present application. The image processing device 600 applies the above-mentioned electronic device, and the image processing device 600 includes: an image division module 610, a vector acquisition module 620 and an image blur module 630. Among them, the image division module 610 is used to divide the target image frame in multiple continuous image frames into multiple pixel blocks; the vector acquisition module 620 is used to obtain the motion vector corresponding to each pixel block in the multiple pixel blocks, wherein the motion vector corresponding to each pixel block is determined according to the motion vector corresponding to some pixels in each pixel block; the image blur module 630 is used to blur the target area in the target image frame based on the motion vector corresponding to each pixel block to obtain a blurred image corresponding to the target image frame, and the target area is the foreground area in the target image frame or the background area other than the foreground area.

[0120] In some embodiments, the vector acquisition module 620 can be specifically used to: for each pixel block in the multiple pixel blocks, select a target number of pixel points from the pixel block, and the target number is less than the total number of pixel points in the pixel block; obtain the motion vector corresponding to each pixel point in the target number of pixel points; based on the motion vector corresponding to each pixel point, determine the motion vector corresponding to the pixel block.

[0121] In a possible implementation manner, the target number is 1, and the vector acquisition module 620 may also be configured to select any one pixel point from each pixel block in the multiple pixel blocks.

[0122] In a possible implementation manner, the target number is multiple, and the vector acquisition module 620 may also be configured to select, for each pixel block in the multiple pixel blocks, a target number of pixel points uniformly distributed in the pixel block.

[0123] In a possible implementation, the vector acquisition module 620 may also be used to determine the motion vector corresponding to each pixel point based on the position of the corresponding pixel point of each pixel point among the target number of pixels in the previous image frame of the target image frame, and the position of each pixel point in the target image frame.

[0124] In some embodiments, the image processing device 600 may further include a foreground recognition module. The foreground recognition module may be used to perform foreground recognition on the target image frame before dividing the target image frame in the multiple consecutive image frames into a plurality of pixel blocks, and obtain a foreground area in the target image frame and a background area other than the foreground area. In this manner, the image division module 610 may be used to divide the target image frame into a plurality of pixel blocks based on the foreground area and the background area, wherein the size of the pixel blocks in the foreground area is smaller than the size of the pixel blocks in the background area.

[0125] In some embodiments, the image division module 610 can be used to perform depth estimation on the content in the target image frame to obtain depth information of the image content in the target image frame; based on the depth information, the target image frame is divided into multiple pixel blocks, wherein the size of the pixel block is positively correlated with the depth of the image content corresponding to the pixel block.

[0126] In some embodiments, the image blur module 630 can be specifically used to: determine the blur parameters corresponding to the target image frame based on the motion vector corresponding to each pixel block; and blur the target area in the target image frame based on the blur parameters to obtain a blurred image corresponding to the target image frame.

[0127] In a possible implementation, the blur parameters include a blur direction and a blur range, and the image blur module 630 can also be used to determine the target pixel points whose pixel values ​​are to be adjusted based on the blur direction and the blur range, the target pixel points include pixel points in an inner blur area and pixel points in an outer blur area adjacent to the inner blur area, the inner blur area is the target area, and the outer blur area is the area in the target image frame other than the target area; based on the blur direction and the blur range, determine the blur window corresponding to each of the target pixel points; based on the pixel values ​​of the pixel points in the blur window corresponding to each of the target pixel points, adjust the pixel value of each of the target pixel points to obtain a blurred image corresponding to the target image frame.

[0128] Optionally, the image blur module 630 can also be used to determine, for each target pixel, the weight corresponding to each pixel in the blurred window corresponding to each target pixel; determine the target pixel value corresponding to each target pixel based on the weight corresponding to each pixel in the blurred window and the pixel value of the pixel in the blurred window corresponding to each target pixel; adjust the pixel value of each target pixel in the target image frame to the target pixel value corresponding to each target pixel, and obtain a blurred image corresponding to the target image frame.

[0129] Optionally, the image blur module 630 can also be used to determine that if the target pixel is in the inner blur area, the weight corresponding to each pixel in the blur window is the first weight; if the target pixel is in the outer blur area, the weights of the pixels in the blur window located in the inner blur area are the first weight, the weight corresponding to the target pixel is the second weight, and the weights corresponding to other pixels in the blur window located in the outer blur area except the target pixel are 0, and the ratio of the second weight to the first weight is the number of pixels in the blur window located in the outer blur area.

[0130] In a possible implementation, the image blur module 630 can also be used to determine the motion vector of the background area based on the motion vector corresponding to each pixel block; if the target area is the foreground area, then based on the motion vector corresponding to each pixel block in the foreground area and the motion vector of the background area, determine the blur parameter corresponding to each pixel block in the foreground area, as well as the blur parameter corresponding to the pixel block in the background area where the pixel points adjacent to the foreground area are located; if the target area is the background area, then based on the motion vector corresponding to each pixel block in the foreground area and the motion vector of the background area, determine the blur parameter corresponding to the background area, as well as the blur parameter corresponding to the pixel block in the foreground area where the pixel points adjacent to the background area are located.

[0131] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and modules can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here.

[0132] In several embodiments provided in the present application, the coupling between modules may be electrical, mechanical or other forms of coupling.

[0133] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or software functional modules.

[0134] In summary, the solution provided by the present application is to divide the target image frame in multiple continuous image frames into multiple pixel blocks, obtain the motion vector corresponding to each pixel block in the multiple pixel blocks, wherein the motion vector corresponding to each pixel block is determined according to the motion vector corresponding to some pixels in each pixel block; based on the motion vector corresponding to each pixel block, blur the target area in the target image frame to obtain a blurred image corresponding to the target image frame, wherein the target area is the foreground area in the target image frame or the background area other than the foreground area. Thus, after dividing the image frame into multiple pixel blocks, the motion vector of the pixel block is determined, and the image frame is blurred according to the determined motion vector of the pixel block, thereby reducing the amount of calculation when determining the motion vector, and further reducing the amount of calculation when generating a motion blurred image, thereby improving the generation efficiency of the motion blurred image.

[0135] Please refer to Fig.14 , which shows a structural block diagram of an electronic device provided in an embodiment of the present application. The electronic device 100 may be an electronic device capable of running applications, such as a smart phone, a tablet computer, a smart watch, an e-book, etc. The electronic device 100 in the present application may include one or more of the following components: a processor 110, a memory 120, and one or more applications, wherein the one or more applications may be stored in the memory 120 and configured to be executed by one or more processors 110, and the one or more applications are configured to execute the method described in the aforementioned method embodiment.

[0136] The processor 110 may include one or more processing cores. The processor 110 uses various interfaces and lines to connect various parts of the entire electronic device 100, and executes various functions and processes data of the electronic device 100 by running or executing instructions, programs, code sets or instruction sets stored in the memory 120, and calling data stored in the memory 120. Optionally, the processor 110 can be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 110 can integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 110, but may be implemented separately through a communication chip.

[0137] The memory 120 may include a random access memory (RAM) or a read-only memory (ROM). The memory 120 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 120 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the following various method embodiments, etc. The data storage area may also store data (such as a phone book, audio and video data, chat record data) created by the electronic device 100 during use.

[0138] Please refer to Fig.15 , which shows a structural block diagram of a computer-readable storage medium provided in an embodiment of the present application. The computer-readable medium 800 stores program codes, which can be called by a processor to execute the method described in the above method embodiment.

[0139] The computer readable storage medium 800 may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read-only memory), an EPROM, a hard disk, or a ROM. Optionally, the computer readable storage medium 800 includes a non-transitory computer-readable storage medium. The computer readable storage medium 800 has storage space for program code 810 that performs any method step in the above method. These program codes can be read from or written to one or more computer program products. The program code 810 can be compressed, for example, in an appropriate form.

[0140] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An image processing method, characterized in that: The method comprises: Dividing a target image frame among a plurality of continuous image frames into a plurality of pixel blocks; Obtaining a motion vector corresponding to each pixel block in the plurality of pixel blocks, wherein the motion vector corresponding to each pixel block is determined according to motion vectors corresponding to some pixel points in each pixel block; Based on the motion vector corresponding to each pixel block, the target area in the target image frame is blurred to obtain a blurred image corresponding to the target image frame, wherein the target area is a foreground area in the target image frame or a background area other than the foreground area.

2. The method according to claim 1, characterized in that The obtaining of a motion vector corresponding to each pixel block in the plurality of pixel blocks comprises: For each pixel block in the plurality of pixel blocks, selecting a target number of pixel points from the pixel block, where the target number is less than the total number of pixel points in the pixel block; Obtaining a motion vector corresponding to each pixel of the target number of pixels; Based on the motion vector corresponding to each pixel point, the motion vector corresponding to the pixel block is determined.

3. The method according to claim 2, characterized in that The target number is 1, and for each pixel block in the plurality of pixel blocks, selecting a target number of pixel points from the pixel block includes: For each pixel block in the plurality of pixel blocks, any pixel point is selected from the pixel block.

4. The method according to claim 2, characterized in that: The target number is multiple, and for each pixel block in the multiple pixel blocks, selecting the target number of pixel points from the pixel block includes: For each pixel block in the plurality of pixel blocks, a target number of pixel points evenly distributed in the pixel block are selected.

5. The method according to claim 2, characterized in that: The obtaining of the motion vector corresponding to each pixel point of the target number of pixels includes: Based on the position of the corresponding pixel of each pixel among the target number of pixels in the previous image frame of the target image frame and the position of each pixel in the target image frame, the motion vector corresponding to each pixel is determined.

6. The method according to claim 1, characterized in that Before dividing the target image frame among the multiple consecutive image frames into a plurality of pixel blocks, the method further comprises: Performing foreground recognition on the target image frame to obtain a foreground area in the target image frame and a background area other than the foreground area; The step of dividing a target image frame among a plurality of continuous image frames into a plurality of pixel blocks comprises: Based on the foreground area and the background area, the target image frame is divided into a plurality of pixel blocks, wherein the size of the pixel blocks in the foreground area is smaller than the size of the pixel blocks in the background area.

7. The method according to claim 1, characterized in that The step of dividing a target image frame among a plurality of continuous image frames into a plurality of pixel blocks comprises: Performing depth estimation on the content in the target image frame to obtain depth information of the image content in the target image frame; Based on the depth information, the target image frame is divided into a plurality of pixel blocks, wherein the size of the pixel block is positively correlated with the depth of the image content corresponding to the pixel block.

8. The method according to any one of claims 1 to 7, characterized in that: The blurring of the target area in the target image frame based on the motion vector corresponding to each pixel block to obtain a blurred image corresponding to the target image frame includes: Determining blur parameters corresponding to the target image frame based on the motion vector corresponding to each pixel block; Based on the blur parameter, blur processing is performed on the target area in the target image frame to obtain a blurred image corresponding to the target image frame.

9. The method according to claim 8, characterized in that The blur parameters include a blur direction and a blur range. Based on the blur parameters, blurring the target area in the target image frame to obtain a blurred image corresponding to the target image frame includes: Based on the blur direction and blur range, determine the target pixel point whose pixel value is to be adjusted, the target pixel point includes the pixel point in the inner blur area and the pixel point in the outer blur area adjacent to the inner blur area, the inner blur area is the target area, and the outer blur area is the area in the target image frame except the target area; Based on the blur direction and blur range, determining a blur window corresponding to each of the target pixels; Based on the pixel value of the pixel point in the blur window corresponding to each of the target pixel points, the pixel value of each of the target pixel points is adjusted to obtain a blurred image corresponding to the target image frame.

10. The method according to claim 9, characterized in that The step of adjusting the pixel value of each target pixel based on the pixel value of the pixel in the blur window corresponding to each target pixel to obtain a blurred image corresponding to the target image frame includes: For each of the target pixels, determine a weight corresponding to each pixel in the fuzzy window corresponding to each of the target pixels; Determine a target pixel value corresponding to each target pixel based on a weight corresponding to each pixel in the fuzzy window and a pixel value of a pixel in the fuzzy window corresponding to each target pixel; The pixel value of each target pixel point in the target image frame is adjusted to the target pixel value corresponding to each target pixel point, so as to obtain a blurred image corresponding to the target image frame.

11. The method according to claim 10, characterized in that The step of determining, for each target pixel, a weight corresponding to each pixel in the fuzzy window corresponding to each target pixel, comprises: If the target pixel is in the inner fuzzy area, determining that the weight corresponding to each pixel in the fuzzy window is the first weight; If the target pixel is in the outer blurred area, it is determined that the weights of the pixel points in the blurred window located in the inner blurred area are all the first weights, the weight corresponding to the target pixel is the second weight, and the weights corresponding to the other pixel points in the blurred window located in the outer blurred area except the target pixel are 0, and the ratio of the second weight to the first weight is the number of pixel points in the blurred window located in the outer blurred area.

12. The method according to claim 8, characterized in that The determining the blur parameter of the target area based on the motion vector corresponding to each pixel block includes: Determine the motion vector of the background area based on the motion vector corresponding to each pixel block; If the target area is the foreground area, determining a blur parameter corresponding to each pixel block in the foreground area and a blur parameter corresponding to a pixel block in the background area where a pixel point adjacent to the foreground area is located based on a motion vector corresponding to each pixel block in the foreground area and a motion vector of the background area; If the target area is the background area, the blur parameter corresponding to the background area and the blur parameter corresponding to the pixel block in the foreground area adjacent to the background area are determined based on the motion vector corresponding to each pixel block in the foreground area and the motion vector of the background area.

13. An image processing device, characterized in that: The device comprises: an image division module, a vector acquisition module and an image blur module, wherein: The image division module is used to divide a target image frame among a plurality of continuous image frames into a plurality of pixel blocks; The vector acquisition module is used to acquire a motion vector corresponding to each pixel block in the plurality of pixel blocks, wherein the motion vector corresponding to each pixel block is determined according to motion vectors corresponding to some pixel points in each pixel block; The image blur module is used to blur the target area in the target image frame based on the motion vector corresponding to each pixel block to obtain a blurred image corresponding to the target image frame, wherein the target area is the foreground area in the target image frame or the background area other than the foreground area.

14. An electronic device, characterized in that: include: one or more processors; Memory; One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the method according to any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program codes, and the program codes can be called by a processor to execute the method according to any one of claims 1 to 12.