Image processing method and device, electronic device and storage medium

Abstract

The present application discloses an image processing method, comprising: processing an input image to obtain a first image area and a second image area, then performing motion blur processing on the second image area to obtain a blurred image area, and combining the first image area with the blurred image area to obtain a motion image. The image processing method of the embodiment of the present application can achieve dynamic effects such as a clear subject and blurred background in the second image area of ​​the remaining image area, for example, a clear subject. At the same time, the user can interactively adjust the required blur level as needed to obtain a suitable motion effect, which increases user interaction to a certain extent and improves user experience. The present application also discloses an image processing device, an electronic device, and a storage medium.

Description

Technical Field

[0001] The present application relates to the field of image technology, and in particular to an image processing method, an image processing device, an electronic device and a storage medium. Background Art

[0002] With the development of image processing technology, people are becoming more and more interested in various special image processing effects. Especially for image processing with moving objects. For example, the processing method of blurring the background and making the characters clear. At present, because the processing of images with moving objects is more complicated than ordinary image processing, it requires users to react quickly. After determining the subject during shooting, they should quickly focus and track the subject, move the camera in the same direction and speed as the subject, so that the background is blurred and the characters are clear. However, this requires a high level of professionalism for users, especially in the field of mobile phone photography. It is difficult to obtain the above-mentioned first type of motion effects through manual operation by users. Summary of the invention

[0003] In view of this, embodiments of the present application provide an image processing method, an image processing device, an electronic device, and a storage medium.

[0004] The first aspect of the present application provides an image processing method, comprising:

[0005] Processing the input image to obtain a first image region and a second image region;

[0006] Performing motion blur processing on the second image area to obtain a blurred image area;

[0007] The first image region is combined with the blurred image region to obtain a motion image.

[0008] The second aspect of the present application also provides an image processing device, including:

[0009] A first processing module, used for processing the input image to obtain a first image area and a second image area;

[0010] A blur module, used for performing motion blur processing on the second image area to obtain a blurred image area;

[0011] A combining module is used to combine the first image area with the blurred image area to obtain a motion image.

[0012] A third aspect of the present application provides an electronic device, characterized in that the electronic device includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the image processing method of the above embodiment is implemented.

[0013] A fourth aspect of the present application provides a non-volatile computer-readable storage medium of a computer program, which implements the above-mentioned image processing method when the computer program is executed by one or more processors.

[0014] Thus, in the image processing method, image processing device, electronic device and storage medium of the embodiment of the present application, the input image is processed to obtain the first image area and the second image area, and then the second image area is subjected to motion blur processing to obtain the blurred image area, and the first image area and the blurred image area are combined to obtain a moving image. The first image area containing the moving subject can be made clear and the second image area of ​​the remaining image area can be made motion blurred to achieve dynamic effects, such as a clear subject and a blurred background. At the same time, the user can interactively adjust the required blur level as needed to obtain a suitable motion effect, which increases user interaction to a certain extent and improves the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The above and / or additional aspects and advantages of the present application will become obvious and easily understood from the description of the embodiments in conjunction with the following drawings.

[0016] Figure 1 is a flowchart of an image processing method in some embodiments of the present application;

[0017] Figure 2 is a module diagram of an image processing device in some embodiments of the present application;

[0018] Figure 3 is a flowchart of an image processing method in some embodiments of the present application;

[0019] Figure 4 is an example diagram of an image processing method in some embodiments of the present application;

[0020] Figure 5 is a flowchart of an image processing method in some embodiments of the present application;

[0021] Figure 6 is a flowchart of an image processing method in some embodiments of the present application;

[0022] FIG7 is an example diagram of an image processing method according to some embodiments of the present application;

[0023] Figure 8 is a flowchart of an image processing method in some embodiments of the present application;

[0024] Fig. 9 is a flowchart of an image processing method in some embodiments of the present application;

[0025] Fig.10It is a module diagram of an image processing method in some embodiments of the present application;

[0026] Figure 11-a , 11-b , 11-c is an example diagram of an image processing method in certain embodiments of the present application;

[0027] Figure 12-a , 12-b , 12-c is an example diagram of an image processing method in certain embodiments of the present application;

[0028] Fig.13 is a flowchart of an image processing method in some embodiments of the present application;

[0029] Fig.14 It is a module diagram of an image processing method in some embodiments of the present application;

[0030] Fig.15 is a schematic diagram of the image processing device flow chart of certain embodiments of the present application;

[0031] Fig.16 It is a flowchart of the image processing method of certain embodiments of the present application. DETAILED DESCRIPTION

[0032] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0033] See also Figure 1 The present application provides an image processing method, characterized in that it includes:

[0034] S10: Processing the input image to obtain a first image region and a second image region;

[0035] S20: performing motion blur processing on the second image region to obtain a blurred image region;

[0036] S30: Combine the first image area with the blurred image area to obtain a motion image.

[0037] Accordingly, see Figure 2The embodiment of the present application also provides an image processing device 100, and the image processing method of the embodiment of the present application can be implemented by the image processing device 100. The image processing device 100 includes a first processing module 110, a blur module 120 and a combination module 130. S10 can be implemented by the first processing module 110, S20 can be implemented by the blur module 120, and S30 can be implemented by the combination module 130. In other words, the first processing module 110 is used to process the input image to obtain a first image area and a second image area. The blur module 120 is used to perform motion blur processing on the second image area to obtain a blurred image area. The combination module 130 is used to combine the first image area with the blurred image area to obtain a motion image.

[0038] The present application also provides an electronic device. The electronic device includes a memory and a processor. The memory stores a computer program, and the processor is used to process an input image to obtain a first image area and a second image area, then perform motion blur processing on the second image area to obtain a blurred image area, and combine the first image area with the blurred image area to obtain a motion image.

[0039] It should be noted that the source of the input image includes but is not limited to photos or video captured by single-camera mobile phones, multi-camera mobile phones, wearable devices, cameras, camcorders and other electronic devices. Image color modes are not limited, including but not limited to RGB, Mono, RGB+Mono, etc.

[0040] Specifically, in step S10, the input image is processed to obtain a first image area and a second image area. The processing includes but is not limited to dividing the image into a first image area and a second image area by segmentation processing, motion pixel extraction, or automatic recognition of motion areas. The first image area and the second image area each include at least one object, that is, the first image area and the second image area may include one object, or multiple objects. For example, an input image contains a person and two dogs running. Based on multiple frames before and after the image, it is recognized that the person and the dog are in motion. When processing is required, the person and the two dogs can be automatically identified as the first image area, and the remaining objects are the second image area.

[0041] In some embodiments, processing the input image includes segmenting the image to obtain a first image region and a second image region. Image segmentation includes dividing the image into a plurality of non-intersecting image regions according to features such as grayscale, color, spatial texture, and geometric shape, so that these features show consistency or similarity in the same region, but show obvious differences between different regions. According to the actual application, segmentation includes but is not limited to semantic segmentation, instance segmentation, and the like. The segmentation can be performed by deep learning-based segmentation such as region selection segmentation, segmentation method based on active contour model, or segmentation based on edge detection.

[0042] In one example, if an input image shows a person or multiple people running, semantic segmentation can be performed to separate the person from the background, and the person or multiple people can be further segmented into a first image area, and the background of the remaining image area can be segmented into a second image area.

[0043] In another example, an input image shows that passerby A is running, and passersby B and passersby C are walking. Then instance segmentation can be performed to separate each person from each background object. Passerby A can be further segmented into a first image region, and passersby B, passersby C and each background can be segmented into a second image region.

[0044] In this way, the image is processed by the image segmentation method to obtain the first image area and the second image area. Because its algorithm is relatively mature, it can more clearly distinguish the first image area and the second image area compared to methods such as motion area recognition, and the effect of image area separation is better. Different image segmentation processes can be performed according to actual business applications, and it is widely used.

[0045] Preferably, see Figure 3 In some embodiments, step S10 includes:

[0046] S11: performing semantic segmentation on the input image to obtain a first subject and a first background, wherein the first subject includes one or more moving objects;

[0047] S12: Determine the area where the first subject is located as the first image area, and determine the first background area as the second image area.

[0048] In some embodiments, S11 and S12 may be implemented by the first processing module 110. In other words, the first processing module 110 is used to perform semantic segmentation on the input image to obtain a first subject and a first background, wherein the first subject includes one or more moving objects, and the area where the first subject is located is determined as the first image area, and the first background area is determined as the second image area.

[0049] In some embodiments, the processor is used to perform semantic segmentation on the input image to obtain a first subject and a first background, wherein the first subject includes one or more moving objects, the area where the first subject is located is determined as the first image area, and the first background area is determined as the second image area.

[0050] Specifically, semantic segmentation is a classification at the pixel level, where pixels belonging to the same class are grouped together. An image can be semantically segmented into a first subject and a first background. For example, see Figure 4 , if the content of an input image is a person running, semantic segmentation can be performed to determine the person as the first subject, and the remaining image area as the first background. Further, the first subject person is determined as the first image area, and the first background of the remaining image area is determined as the second image area.

[0051] It should be noted that the above-mentioned “first subject” and “first background” are the subject and background in semantic segmentation with well-known meanings. “First” is to distinguish them from the subject and background in other segmentation methods, and there is no restriction on the order of precedence.

[0052] In this way, for some application scenarios, such as the consistent motion state of objects of the same type, or a single subject, the image is processed into a first image area and a second image area through semantic segmentation. Compared with other segmentation methods, the data annotation cost is lower and the image processing efficiency is higher.

[0053] See also Figure 5 In some embodiments, step S10 further includes:

[0054] S13: extracting motion pixels from the input image to obtain a second subject, wherein the second subject includes one or more moving objects;

[0055] S14: determining the area where the second subject is located as the first image area; and

[0056] S15: Determine a second background area excluding the second subject in the input image as a second image area.

[0057] In some embodiments, S13-S15 can be implemented by the first processing module 110. In other words, the first processing module 110 is used to extract motion pixels from the input image to obtain a second subject, wherein the second subject includes one or more moving objects, and the area where the second subject is located is determined as the first image area; and the second background area in the input image except the second subject is determined as the second image area.

[0058] In some embodiments, the processor is used to extract motion pixels from the input image to obtain a second subject, wherein the second subject includes one or more moving objects, and determine the area where the second subject is located as the first image area; and determine a second background area in the input image excluding the second subject as the second image area.

[0059] Specifically, motion pixel extraction is performed on the input image to obtain the second subject. The motion pixel extraction method has a general meaning, including methods such as determining motion pixels by screening preset previous and next frame position conditions, or determining motion pixels by contour area matching.

[0060] Further, after the motion pixels are extracted, the motion object including one or more motion pixels is determined as the second subject, and then the area where the second subject is located is determined as the first image area, and the remaining image areas can be determined as the second image area.

[0061] In one example, an input image shows a person A and two dogs running, and two other people, person B and person C, walking. Then, through motion pixel extraction, person A and the two dogs can be determined as the moving subjects, and person A and the two dogs are the first image region. However, person B and person C do not contain motion pixels, and can be determined as the second image region together with the rest of the image region.

[0062] In this way, multiple moving objects can be detected in real time through motion pixel extraction. Compared with segmentation, it can handle more complex images such as application scenarios with cluttered backgrounds and many moving objects.

[0063] In some embodiments, segmentation can be combined with motion pixel extraction, such as first segmenting each object through instance segmentation, and then extracting motion pixels from each segmented object, where the motion object with motion pixels is determined as the first image area, and the remaining objects are determined as the second image area.

[0064] Furthermore, in step S20, the second image region is subjected to motion blur processing to obtain a blurred image region. The motion blur processing is to perform blur processing on the second image region, and the processing method includes convolution processing through blur kernel, blur processing by filter such as Gaussian filter, etc.

[0065] In some embodiments, the second image area includes multiple objects or multiple types of objects, and different motion blur processing can be performed on the multiple objects or multiple types of objects through further segmentation or identification. For example, when blurring is performed through Gaussian filtering, different blurring degrees can be performed on different objects. When blurring is performed through a blur kernel, blurring in different directions can also be performed according to the direction to obtain different motion effects.

[0066] Preferably, see Figure 6 In some embodiments, step S20 includes:

[0067] S21: Acquire multiple frames of images corresponding to the input image, wherein the multiple frames of images are multiple frames of images before and after the input image;

[0068] S22: determining a moving direction and a moving speed of a moving object in the first subject or a moving object in the second subject according to the multiple frames of images;

[0069] S23: determining a motion blur kernel according to the motion direction and the motion speed;

[0070] S24: performing convolution processing on the second image region according to the motion blur kernel to obtain a blurred image region.

[0071] In some embodiments, S21-S24 may be implemented by the blur module 120. In other words, the blur module 120 is used to obtain multiple frames of images corresponding to the input image, determine the moving direction and moving speed of the moving object in the first subject or the moving object in the second subject according to the multiple frames of images, then determine the motion blur kernel according to the moving direction and moving speed, and perform convolution processing on the second image area according to the motion blur kernel to obtain a blurred image area.

[0072] In some embodiments, the processor is used to obtain multiple frames of images corresponding to the input image, determine the movement direction and speed of the moving object in the first subject or the moving object in the second subject based on the multiple frames of images, then determine the motion blur kernel based on the movement direction and the motion speed, and perform convolution processing on the second image area based on the motion blur kernel to obtain a blurred image area.

[0073] Specifically, the input image includes multiple frames of images, and the multiple frames of images are multiple frames before and after the input image is taken. The optical flow can be calculated by shooting multiple frames before and after the preview, and the optical flow vector diagram is summed and averaged to obtain the movement direction and movement speed of the object in the first image area. Then the motion blur kernel is determined according to the movement direction and movement speed. Please refer to Figures 7a-d, which are schematic diagrams of the estimated motion blur kernel, from left to right, respectively, moving from the lower right corner to the upper left corner, moving from the upper left corner to the lower right corner and shifting horizontally to the right, and moving from the lower left corner to the upper right corner and shifting horizontally to the right. For example, when the movement direction is from the lower right corner to the upper left corner, the motion blur kernel can be a kernel with a larger weight in the upper left corner and a smaller weight towards the lower right corner. The same applies to other directions.

[0074] Further, after the motion blur kernel is determined, the second image region is convolved according to the motion blur kernel to blur the second image region to obtain a blurred image region. The size of the blur kernel is variable, that is, the blur degree is adjustable, including but not limited to setting the system to automatically adjust, or the user to interactively adjust the blur degree through related control operations to achieve a suitable motion effect.

[0075] In this way, by determining the blur kernel according to the motion direction and speed of the first image region and performing motion blur processing, the blur processing effect of the second image region can be more coordinated with the motion of the first image region.

[0076] Preferably, see Figure 8 In some embodiments, step S20 further includes:

[0077] S25: Obtain the fuzziness level of the user input;

[0078] Accordingly, step S23 includes:

[0079] S231: determining a motion blur kernel according to the blur degree, motion direction and motion speed;

[0080] In some implementations, S25 may be implemented by the blur module 120. In other words, the blur module 120 is further configured to obtain the blur degree of the user input, and determine the motion blur kernel according to the blur degree, the motion direction, and the motion speed.

[0081] In some embodiments, the processor is configured to obtain a blur degree of the user input, and determine a motion blur kernel according to the blur degree, the motion direction, and the motion speed.

[0082] Specifically, the user can interactively and manually adjust the blur level through the interface control, and can obtain the blur level adjusted by the user each time and determine the size of the motion blur kernel according to the blur level, and determine other parameters of the motion blur kernel in combination with the motion direction and motion speed. The final motion blur result is reflected to the user in real time, so that the user can make further adjustments based on the real-time result.

[0083] In this way, the user can manually adjust the blur level through interactive interaction, so that the user can obtain appropriate motion effects according to the desired blur level, which increases user interaction and improves the user experience to a certain extent.

[0084] Further, in step S30, the first image area is combined with the blurred image area to obtain a motion image. After the second image area is processed by adding motion blur to obtain a blurred image area, the first image area can be combined with the blurred image area, so that the image processing is implemented to obtain the desired motion image with a certain degree of motion effects. The combination includes but is not limited to directly combining the first image area with the blurred image area according to the positions of the first image area and the second image area before the original image is segmented, or adjusting the first image area and the blurred image area by appropriate adjustments such as light, filter, special effects combination, etc. and then combining them. The adjustment can also be set to automatic adjustment by the system, or manual interactive adjustment by the user.

[0085] The present application processes the input image to obtain the first image area and the second image area, then performs motion blur processing on the second image area to obtain a blurred image area, and combines the first image area with the blurred image area to obtain a processed motion image, which can make the first image area containing the moving subject clear and the second image area of ​​the remaining image area motion blurred to achieve dynamic effects, such as a clear subject and blurred background. At the same time, users can interactively adjust the required blur level as needed to obtain appropriate motion effects, which increases user interaction and improves user experience to a certain extent.

[0086] See also Fig. 9 In some embodiments, before step S10, the method further includes:

[0087] S01: Acquire an original image and detect the image quality of the original image;

[0088] S02: When the detection result of the original image is blurred, the original image is deblurred to obtain a fully clear input image;

[0089] S03: When the detection result of the original image is that the entire image is clear, the original image is used as an input image.

[0090] Accordingly, see Fig.10 , the image processing device 100 further includes a detection module 140. S01-S03 can be implemented by the detection module 140. In other words, the detection module 140 is used to obtain an original image and detect the image quality of the original image. When the detection result of the original image is blurred, the original image is de-motion blurred to obtain an input image with full image clarity. When the detection result of the original image is that the original image is full image clarity, the original image is used as the input image.

[0091] In some embodiments, the processor is used to obtain an original image and detect the image quality of the original image. When the detection result of the original image is blurred, the original image is de-motion blurred to obtain an input image with full clarity. When the detection result of the original image is that the original image is full clarity, the original image is used as the input image.

[0092] It can be understood that when the photographed object is in motion, the final original image may be clear in the whole image, or may be blurred in different situations, including a full-image motion blurred image in which both the subject and the background are blurred to varying degrees, or a partially blurred image in which the subject is blurred and the background is clear. In this way, the image can be preprocessed before image segmentation. The preprocessing includes detecting whether the original image is clear.

[0093] When the original image is detected as a fully motion blurred image, the original image is deblurred to obtain a fully clear image. The deblurring process may adopt a state-of-the-art deblurring algorithm based on deep learning. Please refer to Figures 11a-c, Figure 11-a is a fully blurred image, which is deblurred by the state-of-the-art deep learning-based deblurring algorithm. Figure 11-b Then the image is processed to obtain the first image area and the second image area, and the second image area is subjected to motion blur processing to obtain a blurred image area, and then the first image area and the blurred image area are combined to obtain a processed motion image, such as Figure 11-c .

[0094] When the original image is detected as a partially blurred image, the original image is deblurred to obtain a fully clear image. The partially blurred image includes an image in which part of the object is blurred and the other part is clear. For example, the subject is blurred and the background is clear, or the subject is clear and the background is blurred, or any part is blurred and the other part is clear. The deblurring process can adopt a state-of-the-art deblurring algorithm based on deep learning. Please refer to Figure 12. Figure 12-a Blurred image of moving subject, deblurred by state-of-the-art deep learning based deblurring algorithm Figure 12-b Then, the image is processed to obtain a first image area and a second image area, and the second image area is subjected to motion blur processing to obtain a blurred image area, and then the first image area and the blurred image area are combined to obtain a processed motion image, such as Figure 12-c .

[0095] When the original image is detected as a fully clear image, the original image is determined as the input image. In some embodiments, in order to facilitate unified processing, the images in various blurry situations mentioned above may also be subjected to unified deblurring preprocessing.

[0096] In this way, by de-motion blurring the original images in various blurry situations, the clarity of the input image can be enhanced, so that when distinguishing the first image area and the second image area, the distinction can be improved. At the same time, de-blurring the blurred image can improve the effect of the final dynamic special effects. Furthermore, for any blurred picture, or a picture that the user intuitively "takes a waste", the image processing method of the present application can ultimately generate a picture with dynamic special effects, which effectively enhances the image processing capability and improves the user experience to a certain extent.

[0097] See also Fig.13 In some embodiments, the image processing method of the present application further includes:

[0098] S40: Processing the first image region according to a predetermined strategy to obtain a processed image region;

[0099] Accordingly, S30 includes:

[0100] S32: Combining the processed image area with the blurred image area to obtain a processed motion image.

[0101] Accordingly, see Fig.14 The image processing device 100 further includes a second processing module 150. S40 can be implemented by the second processing module 150. In other words, the second processing module 150 is used to process the first image area according to a predetermined strategy to obtain a processed image area, and the combination module 130 is used to combine the processed image area with the blurred image area to obtain a processed motion image.

[0102] In some embodiments, the processor is configured to process the first image region according to a predetermined strategy to obtain a processed image region and combine the processed image region with the blurred image region to obtain a processed motion image.

[0103] Specifically, the input image is processed according to a predetermined strategy to obtain a first image area and a second image area, and then the second image area is subjected to motion blur processing to obtain a blurred image area. At the same time, the first image area is also processed to obtain a processed image area. The predetermined strategy includes but is not limited to special effect processing such as sharpening, stylization and other filter processing, and de-motion blur processing again. It can be set to automatic system processing or manual interactive adjustment by the user.

[0104] Furthermore, the processed image region and the blurred image region are combined to obtain a processed motion image.

[0105] In this way, according to the special effect requirements, the first image area is processed with special effects, which can further improve the special effect of the final generated motion image. When the setting is user-adjustable, the user has more special effect options, which increases user interaction and improves user experience to a certain extent.

[0106] See also Fig.15 In some embodiments, step S40 further includes:

[0107] S41: when the first image region includes a plurality of moving objects, segmenting the first image region according to the moving objects to obtain a plurality of sub-image regions, wherein the sub-image regions include one or more moving objects;

[0108] S42: processing the plurality of sub-image regions respectively according to a predetermined rule to obtain a plurality of processed sub-image regions;

[0109] S43: Determine a processing image region according to a plurality of processing sub-image regions.

[0110] In some embodiments, S41-S43 may be implemented by the second processing module 150. In other words, the second processing module 150 is used to, when the first image area includes multiple moving objects, segment the first image area according to the moving objects to obtain multiple sub-image areas, and process the multiple sub-image areas respectively according to a predetermined rule to obtain multiple processed sub-image areas, and determine the processed image area according to the multiple processed sub-image areas.

[0111] In some embodiments, the processor is used to, when the first image area includes multiple moving objects, segment the first image area according to the moving objects to obtain multiple sub-image areas, and process the multiple sub-image areas separately according to predetermined rules to obtain multiple processed sub-image areas, and determine the processed image area based on the multiple processed sub-image areas.

[0112] Specifically, the first image area is an image area containing a moving object. When the first image area includes multiple moving objects, the first image area is segmented to obtain multiple sub-image areas, that is, the moving object in the first image area is segmented into multiple moving objects as in the example, and each one or more moving objects is a sub-image area. For example, the content of an image is a passerby A, a passerby B and a dog running, and the moving subjects include three objects: passerby A, passerby B and a dog. It is segmented into three sub-image areas, namely, a passerby A sub-image area, a passerby B sub-image area, and a dog sub-image area.

[0113] Further, the plurality of sub-image regions are processed respectively according to a predetermined rule to obtain a plurality of processed sub-image regions. The predetermined rule includes but is not limited to performing different special effects processing, such as sharpening, stylization, light and shadow filter processing, rendering processing, or different degrees of motion blur removal processing, or different degrees of motion blur addition processing, etc., on the plurality of sub-image regions. The system can adjust automatically, or the user can adjust manually through interactive input.

[0114] In one example, the content of an image includes passerby A, passerby B, passerby C and multiple spectators, wherein passerby A runs at the fastest speed, followed by passerby B and passerby C, and the spectators are almost stationary, then passerby A, passerby B, passerby C are determined as the first image area, and the spectators and the background are the second image area. The first image area is segmented by instance to obtain three sub-image areas including passerby A, passerby B and passerby C. Further, the movement speed of each sub-image area can be calculated, that is, the movement speeds v1, v2, and v3 of passerby A, passerby B and passerby C. It can be seen that v1>v2>v3, and the preset predetermined rule is that the system automatically performs special effects rendering at different speeds to different degrees, such as retaining a clear image of passerby A with the fastest speed v1, and performing a lesser degree of motion blur processing on passerby B with a speed of v2 and passerby C with a speed of v3, wherein the blur degree of passerby C is greater than that of passerby B, and the motion blur degrees of passerby C and passerby B are both less than the motion blur degree of the second image area. In this way, the obtained motion effect is to highlight the fastest moving subject, and the other moving subjects are blurred to a lesser extent.

[0115] In another example, the image content is the same as the above embodiment, namely, passerby A, passerby B, passerby C and multiple spectators, among which passerby A runs at the fastest speed, followed by passerby B and passerby C, and the spectators are almost stationary. The preset predetermined rule is to make corresponding adjustments according to the manual interactive input of the user. Then, the multiple sub-image areas obtained after segmenting the first image area are provided to the user, and the user can perform special effects processing on each sub-image area. After the manual interactive input adjustment of the user is completed, multiple processed sub-image areas are obtained, and the multiple processed sub-image areas are processed image areas. Further, the processed image area is combined with the second blurred image processing area to generate a motion image.

[0116] Thus, when the first image area includes multiple moving objects, one or more moving objects are determined as sub-image areas according to the moving objects, and special effects and other processing can be performed on each sub-image area. Compared with only performing blur motion processing on the second image area, on the one hand, in certain application scenarios, special effects processing such as highlighting, rendering, etc. are performed on the sub-image areas at the same time, which can improve the visual effect of motion special effects. On the other hand, by interactively adjusting the sub-image areas by users, user interaction can be enhanced and user experience can be improved.

[0117] Preferably, see Fig.16 In some embodiments, step S30 further includes:

[0118] S31: performing edge feathering transition processing on the processed image area and the blurred image area to obtain a processed motion image.

[0119] In some implementations, S31 may be implemented by the combination module 130. In other words, the combination module 130 is used to perform edge feathering transition processing on the processed image region and the blurred image region to obtain a processed motion image.

[0120] In some embodiments, the processor is used to perform edge feathering transition processing on the processed image area and the blurred image area to obtain a processed motion image.

[0121] In this way, the edge feathering transition processing can make the processed image area and the blurred image area transition, splice and combine more naturally, thereby improving the intuitive effect of the motion special effects of the motion image to a certain extent.

[0122] In summary, in the image processing method, image processing device, electronic device and storage medium of the embodiment of the present application, the first image area and the second image area are obtained by processing the input image, and then the second image area is subjected to motion blur processing to obtain a blurred image area, and the first image area and the blurred image area are combined to obtain a motion image. At least the following beneficial effects are achieved:

[0123] 1. A dynamic special effect that can make a first image area containing a moving subject clear and a second image area containing a remaining portion of the image area motion blurred, such as a clear subject and a blurred background.

[0124] Second, users can interactively adjust the required blur level according to their needs to obtain appropriate motion effects, which increases user interaction and improves user experience to a certain extent.

[0125] 3. The edge feathering transition processing can make the processed image area and the blurred image area transition and combine more naturally, thereby improving the intuitive effect of the motion special effects of the motion image to a certain extent.

[0126] Fourth, users can manually adjust the blur level interactively, so that users can obtain appropriate motion effects according to the required blur level, which increases user interaction and improves the user experience to a certain extent.

[0127] 5. By de-motion blurring the original images in various blurry situations, the clarity of the input image can be enhanced, so that when distinguishing the first image area and the second image area, the distinction can be improved. At the same time, de-blurring the blurred image can improve the final dynamic special effects. Furthermore, for any blurred picture, or a picture that the user intuitively "takes a waste", the image processing method of the present application can eventually generate a picture with dynamic special effects, which effectively enhances the image processing capability and improves the user experience to a certain extent.

[0128] 6. When the first image area includes multiple objects, one or more objects are determined as sub-image areas, and special effects and other processing can be performed on each sub-image area. Compared with only blurring and motion processing on the second image area, on the one hand, in some application scenarios, special effects processing such as highlighting and rendering can be performed on the sub-image areas at the same time, which can improve the visual effect of motion special effects. On the other hand, by interactively adjusting the sub-image area by the user, user interaction can be enhanced and user experience can be improved.

[0129] The present application also provides a computer-readable storage medium, one or more non-volatile computer-readable storage media storing a computer program, which implements the image processing method of any of the above embodiments when the computer program is executed by one or more processors.

[0130] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related software through a computer program. The program can be stored in a non-volatile computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM), etc.

[0131] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. An image processing method, characterized in that: include: Processing the input image to obtain a first image region and a second image region; Performing motion blur processing on the second image area to obtain a blurred image area; When the first image area includes a plurality of moving objects, segmenting the first image area according to the moving objects to obtain a plurality of sub-image areas, wherein the sub-image areas include one or more of the moving objects; Processing the plurality of sub-image regions respectively according to a predetermined rule to obtain a plurality of processed sub-image regions, wherein the predetermined rule includes de-motion blur processing to different degrees, or adding motion blur processing to different degrees; Determining a processed image region according to the multiple processed sub-image regions; The processed image region is combined with the blurred image region to obtain a processed motion image.

2. The image processing method according to claim 1, characterized in that: Before the input image is processed to obtain the first image region and the second image region, the method further includes: Acquire an original image and detect the image quality of the original image; When the detection result of the original image is blurred, the original image is subjected to a motion blur removal process to obtain the input image with full clarity; When the detection result of the original image is that the entire image is clear, the original image is used as the input image.

3. The image processing method according to claim 1, characterized in that: The processing of the input image to obtain the first image area and the second image area comprises: Performing semantic segmentation on the input image to obtain a first subject and a first background, wherein the first subject includes one or more moving objects; The area where the first subject is located is determined as the first image area, and the first background area is determined as the second image area.

4. The image processing method according to claim 1, characterized in that: The step of processing the input image to obtain the first image region and the second image region further comprises: Extracting motion pixels from the input image to obtain a second subject, wherein the second subject includes one or more moving objects; determining the area where the second subject is located as the first image area; and A second background area excluding the second subject in the input image is determined as the second image area.

5. The image processing method according to claim 3, characterized in that: The performing motion blur processing on the second image area to obtain a blurred image area comprises: Acquire multiple frames of images corresponding to the input image, wherein the multiple frames of images are multiple frames of images taken before and after the input image; Determine the moving direction and moving speed of the moving object in the first subject according to the multiple frames of images; Determine a motion blur kernel according to the motion direction and the motion speed; The second image region is convolved according to the motion blur kernel to obtain the blurred image region.

6. The image processing method according to claim 4, characterized in that: The performing motion blur processing on the second image area to obtain a blurred image area comprises: Acquire multiple frames of images corresponding to the input image, wherein the multiple frames of images are multiple frames of images taken before and after the input image; Determine the moving direction and moving speed of the moving object in the second subject according to the multiple frames of images; Determine a motion blur kernel according to the motion direction and the motion speed; The second image region is convolved according to the motion blur kernel to obtain the blurred image region.

7. The image processing method according to claim 5 or 6, characterized in that: The performing motion blur processing on the second image area to obtain a blurred image area further comprises: Get the blur level of user input; Determining the motion blur kernel according to the motion direction and the motion speed includes: The motion blur kernel is determined according to the blur degree, the motion direction and the motion speed.

8. The image processing method according to claim 1, characterized in that: The step of combining the first image area with the blurred image area to obtain a processed motion image comprises: The processed image region and the blurred image region are subjected to edge feathering transition processing to obtain a processed motion image.

9. An image processing device, characterized in that: include: A first processing module, used for processing the input image to obtain a first image area and a second image area; A blur module, used for performing motion blur processing on the second image area to obtain a blurred image area; a second processing module, configured to, when the first image area includes a plurality of moving objects, segment the first image area according to the moving objects to obtain a plurality of sub-image areas, wherein the sub-image areas include one or more of the moving objects; and process the plurality of sub-image areas respectively according to a predetermined rule to obtain a plurality of processed sub-image areas, wherein the predetermined rule includes de-motion blur processing to different degrees or motion blur processing to different degrees; and determining a processing image region according to the plurality of processing sub-image regions; A combining module combines the processed image area with the blurred image area to obtain a processed motion image.

10. An electronic device, characterized in that: The electronic device includes a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the image processing method according to any one of claims 1 to 8 is implemented.

11. A non-volatile computer-readable storage medium for a computer program, characterized in that: When the computer program is executed by one or more processors, the image processing method according to any one of claims 1 to 8 is implemented.

Images