Image blurring processing method and device

Abstract

The invention provides an image blurring processing method and device, and the method comprises the steps: determining a target main body and a non-target main body in a multi-person scene, further determining the depth of each human body, taking the depth of the main body as a reference, and carrying out the corresponding blurring processing according to the depth of each non-target main body. The blurring intensities corresponding to the main bodies with different depths relative to the target main body are different, namely, the target main body is clear, and the blurring intensities of the non-target main bodies with different depths relative to the target main body are different, so that the layering sense of the blurring effect better conforms to the optical blurring effect of a professional camera.

Description

Technical Field

[0001] This application relates to the field of image processing technologies, and in particular, to an image blurring processing method and apparatus. Background Art

[0002] When current electronic devices such as mobile phones and digital cameras perform video shooting, especially when shooting portraits, the background of the portrait can be blurred to achieve the effect of highlighting the portrait. In the current portrait background blurring solution, the portrait segmentation technology is adopted to segment all portraits in the video image as the main subjects, that is, all portraits in the image are regarded as the main subjects, and all portraits remain clear, while other parts in the image are blurred as the background. The blurring effect of this solution has poor layering. Summary of the Invention

[0003] In view of this, this application provides an image blurring processing method and apparatus to solve at least some of the above problems, and the disclosed technical solutions are as follows:

[0004] In a first aspect, this application provides an image blurring processing method applied to an electronic device. The method includes: displaying a first interface, where the first interface includes a preview window and a blurring button, the preview window displays a first image, the image includes a first portrait, a second portrait, and a third portrait, the first portrait is a focusing target, and the depths of the three portraits are different from each other; in response to an operation on the blurring button, the preview window displays a second image, the second image includes a clear first portrait, a second portrait with a first blurring intensity, and a third portrait with a second blurring intensity, and the first blurring intensity is different from the second blurring intensity. It can be seen that with this solution, the blurring intensities corresponding to the subjects with different depths relative to the target subject are also different, that is, the target subject is clear, and the blurring intensities of the non-target subjects with different depths from the target subject are also different, making the layering of the blurring effect more in line with the optical blurring effect of a professional camera.

[0005] In a possible implementation manner of the first aspect, the second blurring intensity is positively correlated with the first relative depth, and the first relative depth is the relative depth between the second portrait and the first portrait; the third blurring intensity is positively correlated with the second relative depth, and the second relative depth is the relative depth between the third portrait and the first portrait.

[0006] In a possible implementation of the first aspect, the method further includes: the preview window displays a third image, the third image includes a first portrait, a second portrait, and a third portrait, and the autofocus target changes from the first portrait to the second portrait; within a first preset duration, the image displayed in the preview window gradually changes from a first blurred effect image with the first portrait as the autofocus target to a second blurred effect image with the second portrait as the autofocus target, the second blurred effect image includes a clear second portrait, a first portrait with a third blur intensity, and a third portrait with a fourth blur intensity, and the third blur intensity is different from the fourth blur intensity. It can be seen that after switching the autofocus target, this solution gradually transitions from the blurred effect with the original autofocus target as the main body to the blurred effect with the new target as the main body, avoiding the abruptness caused by immediately switching the blurred effect after switching the autofocus target.

[0007] In a possible implementation of the first aspect, the third blur intensity is positively correlated with the third relative depth, the fourth blur intensity is positively correlated with the fourth relative depth, the third relative depth is the relative depth between the first portrait and the second portrait, and the fourth relative depth is the relative depth between the third portrait and the second portrait.

[0008] In a possible implementation of the first aspect, in response to an operation on the blur button, the preview window displays a second image, including: in response to an operation on the blur button, based on the first position information of each portrait in the first image, using an AI depth estimation algorithm to determine the depth of each portrait; taking the depth of the first portrait as a reference, based on the depth of the second portrait, performing blur processing on the second person to obtain a second portrait with a first blur intensity; taking the depth of the first portrait as a reference, based on the depth of the third portrait, performing blur processing on the third person to obtain a third portrait with a second blur intensity.

[0009] In a possible implementation of the first aspect, taking the depth of the first portrait as a reference, based on the depth of the second person, performing blur processing on the second person to obtain a second portrait with a first blur intensity, includes: determining the first blur intensity corresponding to the second portrait based on the relative depth between the second portrait and the first portrait; performing blur processing on the second portrait so that the second portrait reaches the first blur intensity.

[0010] In a possible implementation of the first aspect, the process of obtaining the first position information of each portrait in the first image includes: downsampling the first image to obtain a first-resolution image; detecting the human bodies included in the first-resolution image to obtain a human body detection result, where the human body detection result includes the second position information of the human body and the first human body key points, and the second position information includes the first human body frame position, and the first human body frame includes a square area of all body parts included in any portrait in the first image; using a coordinate transformation matrix to perform coordinate transformation on the first human body frame position and the coordinates corresponding to the first key points to obtain a second human body frame position and second key points, where the coordinate transformation matrix is generated during the anti-shake processing of the first image, and the first position information includes the second human body frame position and the second key points. In this way, after performing the same coordinate transformation as the EIS processing on the position coordinates of the human body detected based on the first-resolution image, it can be directly mapped to the first image after EIS processing, and then depth estimation and blurring processing are performed on each subject in the first image, improving the accuracy of the depth estimation result.

[0011] In a possible implementation of the first aspect, based on the first position information of each portrait in the first image, using an AI depth estimation algorithm to determine the depth of each portrait includes: for any portrait, determining the third human body frame position of any portrait based on the second human body frame position corresponding to any portrait and the second human body key points of the preset part, where the third human body frame position includes the head and torso part of any portrait; for any portrait, using the AI depth estimation algorithm to estimate the depth of the pixel points within the third human body frame corresponding to any portrait to obtain the depth of any portrait. In this way, based on the human body frame determined by partial human body key points, more than 90% of the human body frame covers the human body, so that the human body depth information determined based on this human body frame is more accurate, further improving the blurring effect of the human body.

[0012] In a possible implementation of the first aspect, the electronic device includes a focus tracking decision module and an AI depth blurring module;

[0013] The process of determining the first portrait as the focus tracking target includes: the focus tracking decision module, based on the automatic focus tracking strategy and the human body detection result corresponding to the first image, decides that the focus tracking target is the first portrait, and transmits the serial number of the first portrait to the AI depth blurring module;

[0014] In response to an operation on the defocus button, the preview window displays a second image, including: The AI depth defocus module, in response to the operation on the defocus button, determines the first portrait as the focus tracking target based on the serial number of the first portrait, and performs defocus processing on the second portrait and the third portrait respectively with the depth of the first portrait as the reference, obtaining the second portrait with the first defocus intensity and the third portrait with the second defocus intensity. It can be seen that the focus tracking decision module passes the serial number of the focus tracking target (i.e., the unique label of the target) determined by the decision to the AI depth defocus module, so that the AI depth defocus module can know which human body is the focus tracking target, and then perform defocus processing on other subjects based on the depth of the focus tracking target.

[0015] In a possible implementation manner of the first aspect, the electronic device includes a focus tracking decision module; the preview window displays a third image, including: The focus tracking decision module determines the focus tracking target as the second portrait according to the human body detection result corresponding to the fourth image, and the shooting time of the fourth image is later than that of the first image; the focus tracking decision module continuously tracks the focus tracking target, and if the focus tracking target is always the second portrait within the second preset duration, switches the focus tracking target from the first portrait to the second portrait. In this way, the focus tracking target is switched only after it is determined to be stable, avoiding the focus tracking target from jumping back and forth, and avoiding the focus tracking target from jumping due to a face that briefly enters, improving the stability of the focus tracking target.

[0016] In a possible implementation manner of the first aspect, the electronic device includes an AI depth defocus module; after the focus tracking decision module switches the focus tracking target from the first portrait to the second portrait, it transmits the serial number of the second portrait to the AI depth defocus module; within the first preset duration, the AI depth defocus module gradually changes the first defocus effect image with the first portrait as the focus tracking target to the second defocus effect image with the second portrait as the focus tracking target.

[0017] In a possible implementation manner of the first aspect, within the first preset duration, the image displayed in the preview window gradually changes from the first defocus effect image with the first portrait as the focus tracking target to the second defocus effect image with the second portrait as the focus tracking target, including: After receiving the information that the focus tracking target is switched to the second portrait, determining the depth of each object in the current image frame, where the object includes the portrait object and other shooting objects in the image; determining the target defocus intensity corresponding to the depth of other objects in the image based on the depth of the second portrait; within the first preset duration, controlling any other object to gradually change from the current defocus intensity corresponding to any other object to the target defocus intensity corresponding to any other object, and controlling the second portrait to gradually change from the current defocus intensity corresponding to the second portrait to clear.

[0018] In a possible implementation of the first aspect, within a first preset duration, controlling any other object to gradually change from the current virtualization intensity corresponding to any other object to the target virtualization intensity corresponding to any other object includes: taking the start moment of the first preset duration as the start moment of the Bézier curve, taking the end moment of the first preset duration as the end moment of the Bézier curve, and taking the current virtualization intensity of any other object as the starting point and the target virtualization intensity of any other object as the ending point to generate a smooth Bézier curve corresponding to any other object; at any moment within the first preset duration, finding the target virtualization intensity corresponding to the moment based on the smooth Bézier curve, and performing virtualization processing on any other object to the target virtualization intensity corresponding to any moment.

[0019] In a second aspect, the present application further provides a chip system, including at least one processor and an interface. The interface is used to receive code instructions and transmit them to at least one processor; the at least one processor runs the code instructions to implement the image virtualization processing method of any item in the first aspect.

[0020] In a third aspect, the present application further provides an electronic device, which includes: one or more processors, a memory, and a touch screen; the memory is used to store program code; the processor is used to run the program code so that the electronic device implements the image virtualization processing method of any item in the first aspect.

[0021] In a fourth aspect, the present application further provides a computer-readable storage medium, on which instructions are stored. When the instructions are run on an electronic device, the electronic device is caused to execute the image virtualization processing method of any item in the first aspect.

[0022] In a fifth aspect, the present application further provides a computer program product, on which an execution is stored. When the computer program product is run on an electronic device, the electronic device is caused to implement the image virtualization processing method of any item in the first aspect. Description of the Drawings

[0023] Figure 1 is a schematic diagram of a video recording interface provided by an embodiment of the present application;

[0024] Figure 2 is a schematic diagram of an interface for shooting a movie provided by an embodiment of the present application;

[0025] Figure 3 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application;

[0026] Figure 4 is a schematic diagram of the structure of an image virtualization processing system provided by an embodiment of the present application;

[0027] Figure 5It is a flowchart of an image blurring processing method provided by an embodiment of the present application;

[0028] Figure 6 It is a schematic diagram of recording a video containing a portrait provided by an embodiment of the present application;

[0029] Figure 7 It is another schematic diagram of recording a video containing a portrait provided by an embodiment of the present application;

[0030] Figure 8 It is yet another schematic diagram of recording a video containing a portrait provided by an embodiment of the present application;

[0031] Figure 9 It is a schematic diagram of a human body frame provided by an embodiment of the present application;

[0032] Figure 10 It is a flowchart of a focus tracking target switching process provided by an embodiment of the present application;

[0033] Figure 11 It is a curve schematic diagram of a blurring effect transition process provided by an embodiment of the present application. Detailed implementation manners

[0034] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application.

[0035] An embodiment of the present application provides an image blurring processing method, which can be applied to electronic devices such as digital cameras, mobile phones, and tablet computers that have cameras and can provide shooting functions.

[0036] In the scenario of an electronic device recording a video, after detecting an operation of the user turning on the portrait blurring button, the depth values of each object in the video image are calculated through an AI depth estimation algorithm. Then, blurring processing is performed according to the background depth value of the video image, and the blurring degrees corresponding to different depths are different, so that such a blurring effect has a stronger sense of hierarchy. Moreover, in a multi-person scenario, only the target subject remains clear, and other subjects are blurred accordingly according to their depth values, achieving different blurring degrees corresponding to portraits at different depths, which is more in line with the blurring effect of a real depth-of-field lens.

[0037] In an embodiment of the present application, the portrait background blurring function can be turned on in the video recording mode or movie mode of the camera application. The user interfaces for turning on the portrait background blurring in these two scenarios will be introduced separately below.

[0038] As Figure 1 shown, taking the electronic device as a mobile phone as an example, the user interface for turning on the portrait background blurring in the video recording scenario is introduced.

[0039] Figure 1(1) is a schematic diagram of a mobile phone desktop. The status bar 101 is located at the top of the desktop 100. The general program tray 102 is located below the status bar 101, and the frequently used program tray 103 is located below the general program tray 102.

[0040] The status bar is used to display the current status of the mobile phone. For example, information such as the cellular network signal quality indicator, the wireless network signal quality indicator, the battery status indicator, and the real-time time.

[0041] Both the general application program tray 102 and the frequently used application program tray 103 are used to carry application program icons. The icon of any application program can be placed in the frequently used application program tray 103 or the general application program tray 102. The user can enable the application program corresponding to the icon by clicking on the application program icon. For example, in the embodiment of the present application, the user can click on the camera application icon 104 in the general application program tray 102, and the mobile phone runs the camera application, and the main interface of the camera application is displayed on the mobile phone screen, that is Figure 1 the interface shown in (2).

[0042] As Figure 1 shown in (2), the main interface of the camera application may include a setting bar 110 located at the top, a preview window 113 with the largest area located below the setting bar, a menu bar 117 located below the preview window 113, and a control area located below the menu bar. For example, it may include a shooting control 119, a review button 118, and a camera inversion control 120.

[0043] The setting bar 110 may include multiple setting controls for adjusting shooting parameters. For example, a flash control 111, a setting control 112. In addition, it may also include a filter control, an auto-follow control, etc. The present application does not make special limitations on the types of controls included in the setting bar.

[0044] The menu bar 117 may include multiple shooting mode options, such as "portrait", "photo", "video", "movie" and other options. For example, Figure 1 the interface shown in (2) is the main interface of the video mode.

[0045] The number of options that the menu bar 117 can display is limited, and there are still some mode options that cannot be displayed. The menu bar can be swiped left and right to display other mode options.

[0046] The preview window 113 is used to display the image captured by the camera in real time. In the embodiment of the present application, the bottom of the preview window 113 also includes a blur button 114, a zoom magnification selection button 115, and a beauty button 116. After the user clicks the blur button 114, the background blur function is enabled and it jumps to Figure 1 the interface shown in (3).

[0047] AsFigure 1 As shown in (3) of FIG. [Reference numeral not provided], the background 131 other than the target subject 130 in the captured image is blurred to highlight the portrait. In addition, a blur degree scale 132 can be displayed at the bottom of the preview window 113. Sliding the blur degree scale 132 left and right can adjust the background blur degree. For example, in this embodiment, the blur degree scale 132 indicates that the blur degree gradually increases from left to right. The stronger the blur degree, the higher the blur level, that is, the more blurred. Conversely, the weaker the blur degree, the lower the blur level, that is, the clearer.

[0048] In another scenario, the portrait blur function can also be enabled when the camera application of the mobile phone is in the movie mode.

[0049] As Figure 2 shown in (1) of FIG. [Reference numeral not provided], after the user clicks the camera application icon 102 on the desktop 101, it jumps to the main interface of the camera application. For example, in this embodiment, as Figure 2 shown in (2) of FIG. [Reference numeral not provided], the main interface of the movie mode.

[0050] The electronic device of the present application is not limited to mobile phones, tablet computers, and digital cameras, but can also be other electronic devices with a camera, such as desktop computers, laptop computers, handheld computers, notebook computers, ultra-mobile personal computers (UMPCs), netbooks, and personal digital assistants (PDAs), augmented reality (AR) devices, virtual reality (VR) devices, artificial intelligence (AI) devices, wearable devices, in-vehicle devices, smart home devices, and / or smart city devices. The specific type of the electronic device in the embodiments of the present application is not particularly limited.

[0051] As Figure 2 shown in (2) of FIG. [Reference numeral not provided], the movie mode interface includes a settings bar 147 located at the top, a preview window 140, a menu bar 148, a review button 144, a movie shooting button 145, and a lens reversal button 146. Among them, the bottom of the preview window 140 includes a blur button 141, a zoom magnification selection button 142, and a beauty button 143.

[0052] After the user clicks the blur button 141, the background blur function is enabled, and it jumps to Figure 2 the interface shown in (3) of FIG. [Reference numeral not provided], that is, the portrait remains clear and the background is blurred. A blur degree scale 149 is displayed at the bottom of the preview window. The blur degree scale 149 is the same as the blur degree scale 132 in (3) of Figure 1 FIG. [Reference numeral not provided], and will not be elaborated here. Adjusting the blur degree scale 149 can adjust the background blur degree.

[0053] As Figure 3 shown, the electronic device provided in the embodiment of the present application may include a processor 201, a memory 202, a camera 203, and a display screen 204.

[0054] The processor 201 may include one or more processing units. Different processing units may be independent devices or integrated in one or more processors.

[0055] The memory 202 may be used to store computer-executable program code, and the executable program code includes instructions. The processor 201 executes various functional applications and data processing of the terminal device by running the instructions stored in the memory. For example, in this embodiment, the processor may perform an image blurring processing method by executing the instructions in the memory.

[0056] The camera 203 is used to capture still images or videos. In some embodiments, the electronic device may include one or N cameras, where N is a positive integer greater than 1.

[0057] The display screen 204 is used to display images, videos, etc., and may also display a series of graphical user interfaces (GUIs), and these GUIs are the main screens of the electronic device. In the embodiment of the present application, the display screen may adopt a touch screen. In addition, the electronic device may include one or N display screens.

[0058] Next, in combination with Figure 4 introduce the structural schematic diagram of the image blurring processing system provided in the embodiment of the present application.

[0059] As Figure 4 shown, the system may include a camera, a perception detection module, a focus tracking decision module, a camera application, an autofocus module, an anti-shake module, an AI depth blurring module, and a beauty module.

[0060] In some embodiments, an operating system runs in the electronic device, and a layered architecture, an event-driven architecture, a microkernel architecture, a microservices architecture, or a cloud architecture may be adopted. In the embodiment of the present application, taking the system of the layered architecture as an example, the layered architecture divides the software into several layers, and each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system may include an application layer, an application framework layer, a hardware abstraction layer, and a kernel layer arranged in sequence from top to bottom.

[0061] The above camera application belongs to the application of the electronic device, and the camera application can call the camera to capture images or videos.

[0062] The above-mentioned perception detection module, focus tracking decision module, autofocus module, anti-shake module, AI depth-of-field virtualization module, and beauty module all belong to the algorithm modules in the camera algorithm library of the hardware abstraction layer.

[0063] The hardware abstraction layer is an interface layer between the application framework layer and the kernel layer, providing a virtual hardware platform for the operating system. In this embodiment, the hardware abstraction layer includes a camera hardware abstraction layer and a camera algorithm library. The camera hardware abstraction layer can provide one or more virtual camera devices, and the virtual camera device is the virtual hardware of the camera. The operating system calls the corresponding camera through the virtual hardware of the camera.

[0064] In an exemplary embodiment, after the camera application starts and enters the video recording mode, it calls the camera in the hardware layer through the virtual camera device in the camera hardware abstraction layer to collect image electrical signals in real time, and transmits the image electrical signals to the Image Signal Processor (ISP) for processing and conversion into visible images to continuously generate an image stream, that is, a video, also known as a preview video stream (such as a resolution of 1080P, 4K, etc.). At the same time, the ISP downsamples the preprocessed image to obtain a Tiny stream with a small resolution (such as a resolution of 640P).

[0065] As Figure 4 shown, the preview video stream is used for anti-shake processing, virtualization algorithms, beauty algorithms, etc. The Tiny stream is used for perception detection algorithms, such as face detection, human body detection, etc.

[0066] The processing processes of the Tiny stream and the preview video stream can be executed in parallel. For example, different threads can be used to process the two video streams in parallel respectively.

[0067] After the Tiny stream is subjected to perception detection by the perception detection module, the human body detection result of the main body (i.e., the person) included in the image is obtained. The human body detection result can include the position of the human body frame, the key points of the human body, and the position of the face frame, and the human body detection result is transmitted to the focus tracking decision module.

[0068] In the autofocus tracking mode, the focus tracking decision module decides the target subject based on the received human body detection result according to the focus tracking strategy, and transmits the ID of the target subject and the position information of each subject (i.e., the position of the face frame, the position of the human body frame) to the AI depth-of-field virtualization module.

[0069] In another embodiment, the perception detection module can also directly transmit the human body detection result to the AI depth-of-field virtualization module.

[0070] The anti-shake module performs anti-shake processing on the preview video stream, such as adjusting the position and orientation of the image through image deformation and cropping to keep it stable. This process can also be called electronic image stabilization (EIS) processing. The anti-shake module transfers the anti-shake processed image to the AI depth blurring module, and at the same time transfers the coordinate transformation matrix used in the anti-shake processing process to the AI depth blurring module.

[0071] In an exemplary embodiment, the AI depth blurring module may include a frame coordinate transformation module, a human body frame transformation module, and an AI depth blurring algorithm module.

[0072] The pixels in the EIS processing process are not simple linear transformations, which will cause the coordinates of the video image after anti-shake processing and the Tiny stream image to not be directly and simply mapped. The frame coordinate transformation module is used to perform the same EIS processing on the positions of the human body frame and the face frame detected by the perception detection module to obtain the transformed human body frame coordinates and face frame coordinates, so as to ensure that the transformed human body frame coordinates or face frame coordinates correspond to the main body position in the video image.

[0073] The AI depth blurring algorithm module needs to estimate the depth information of each human body based on the pixels within each human body frame in the image. If the human body frame covers a large area of hollow areas other than the human body, this will lead to inaccurate human body depth estimation, and further lead to poor blurring effects. The human body frame transformation module is used to transform the human body frame containing a large area of hollow areas detected by the perception detection module into a small human body frame, and the human body area covered by this human body frame can reach more than 90%, that is, the pixels other than the human body contained in the human body frame are less than 10%. Further, the human body frame transformation module transfers the positions of the transformed human body frames to the AI depth blurring algorithm.

[0074] In an exemplary embodiment, the human body frame transformation module re-determines the position of the human body frame according to the human body key points corresponding to the human body trunk part, such as the head key point, the shoulder key point, and the trunk key point.

[0075] The AI depth blurring algorithm determines the depth value of the target subject and the depth values of each non-target subject based on the human body frame coordinates of each subject received and the ID of the target subject. Based on the relative depth between each non-target subject and the target subject with the depth value of the target subject as the reference, each non-target subject (or called the portrait background) is blurred, so that the target subject remains clear, and the blurring intensity of non-target subjects with different relative depths from the target subject is also different.

[0076] The AI depth blurring module can transfer the blurred video image to the beauty module for beauty processing, send it to the camera application for display as a preview video stream. At the same time, the beauty processed video stream is saved to the media library, that is, the video recording is saved.

[0077] In addition, the focus tracking decision module can also transmit the position of the face frame or the body frame of the target subject to the camera application, and the camera application displays the face frame or the body frame of the current target subject on the preview interface, so that the user can know which person the current target subject is according to the face frame or the body frame displayed on the interface.

[0078] Furthermore, the user can select other people as the target subject on the preview interface of the camera application. In this scenario, after the camera application detects the click position, it is transmitted to the focus tracking decision module. The focus tracking decision module enters the manual focus tracking mode, and determines which body frame the click position is located in according to the position information of each subject, that is, determines which subject the target subject manually selected by the user is. After determining the subject to which the position belongs, the automatic focus tracking logic judgment is no longer performed, and the subject manually selected by the user is continuously tracked until the subject disappears or is lost.

[0079] If the click position of the user does not belong to any subject, the automatic focus tracking mode is restored.

[0080] In addition, the focus tracking decision module can also transmit the face frame or the body frame corresponding to the determined target subject to the autofocus module, so that the autofocus module can adjust the focus of the lens to ensure that the target subject is within the focus range.

[0081] It can be seen that by using the image blurring processing system provided in this embodiment, in a multi-person scenario, the target subject and the non-target subjects are determined, the depth positions of each subject are further determined, and based on the relative depth distances between each non-target subject and the target subject, corresponding blurring processing is performed on each non-target subject, so that the blurring intensities corresponding to the subjects with different depths relative to the target subject are also different, that is, the target subject is clear, and the blurring intensities of the non-target subjects with different depths from the target subject are also different, making the layering of the blurring effect more in line with the optical blurring effect of a professional camera.

[0082] Next, the image blurring processing flow provided in the embodiments of the present application will be introduced in detail in conjunction with Figure 5 As shown in the figure, the image blurring processing method may include the following steps:

[0083] As Figure 5 shown, the image blurring processing method may include the following steps:

[0084] S100, after the camera application receives the operation of the portrait blurring button in the video recording scenario, it calls the camera to record a video.

[0085] S101, the camera transmits the collected image electrical signal to the ISP.

[0086] The camera is used to capture static images or videos. An object generates an optical image through a lens and projects it onto a photosensitive element. The photosensitive element converts the optical signal into an electrical signal, i.e., an image electrical signal. Then, the image electrical signal is transmitted to the ISP for processing.

[0087] In some embodiments, the electronic device may include one or N cameras. For example, a front camera located on the plane where the screen of the electronic device is located, and a rear camera located on the back of the electronic device. The number of rear cameras can be one or more, and there is no special limitation here. The image blurring processing method of this application is applicable to processing videos recorded by the front camera or the rear camera.

[0088] S102, the ISP processes the received image electrical signal to obtain a preview video image and transmits it to the anti-shake module.

[0089] The ISP converts the image electrical signal into an image visible to the naked eye. In addition, the ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image, etc. The continuously generated images form an image stream, i.e., a video, which is called a preview video stream here.

[0090] Furthermore, the ISP can transmit the processed image to the anti-shake module for anti-shake processing.

[0091] S103, the anti-shake module performs anti-shake processing on the preview video image.

[0092] When shooting a video or taking a photo, the original image is usually post-edited and processed, and the position and orientation of the image are adjusted by image warping and cropping to keep it stable.

[0093] In some embodiments, the anti-shake processing process depends on the capabilities of the ISP, that is, the anti-shake module needs to call the processing capabilities of the ISP to complete the anti-shake processing of the image.

[0094] S104, the anti-shake module transmits the video image after anti-shake processing, and the coordinate transformation matrix corresponding to the anti-shake processing to the AI deep blurring module.

[0095] The preview video image is transmitted to the AI deep blurring module after being processed by the anti-shake module. At the same time, the coordinate transformation matrix corresponding to the crop and warp processes is also transmitted to the AI deep blurring module. So that the AI deep blurring module can perform the same coordinate transformation on the position of the face frame or the body frame detected based on the Tiny stream as the anti-shake processing using this coordinate transformation matrix, so that the coordinates of the face frame (or the coordinates of the body frame) correspond to the coordinates in the preview video image.

[0096] S105, the ISP downsamples each image frame in the preview video stream to obtain a Tiny stream and transmits the Tiny stream to the perception detection module.

[0097] Among them, after the ISP finishes executing S102, it executes S105, that is, the image electrical signal collected by the camera is processed by the ISP to obtain two video streams with different resolutions, namely, a high-resolution preview video stream and a low-resolution Tiny stream.

[0098] S106, the perception detection module detects each image frame in the Tiny stream to obtain a human detection result.

[0099] The human detection result includes the position of the face frame, the position of the first human body frame, and the position of the human body key points.

[0100] For example, the position of all faces included in the video image (i.e., the position of the face frame) can be detected through a face detection algorithm. The position of all human bodies in the video image (i.e., the position of the human body frame) is detected by using a human body detection algorithm, and the information of the human body key points included in all human bodies is detected by using a human body key point detection algorithm.

[0101] In some embodiments, different detection algorithms perform detection at different times. For example, different detection algorithms respectively detect different image frames to obtain corresponding detection results. For example, the face detection algorithm is used to detect whether the first to second frames of images contain faces, and the human body detection algorithm is used to detect the human bodies and human body key points included in the third to fourth frames of images.

[0102] S107, the perception detection module transmits the human detection result to the focus tracking decision module.

[0103] The perception detection module transmits the human detection result to the focus tracking decision module so that the focus tracking decision module can make a decision on the target subject for focus tracking.

[0104] S108, in the automatic focus tracking mode, the focus tracking decision module decides the target subject based on the human detection result by using a focus tracking strategy.

[0105] The focus tracking decision module decides which shooting object in the image is the current shooting focus based on the focus tracking strategy, and further adjusts the focus of the camera lens to ensure that the shooting object is always within the focus range, so that the shooting object is clearer.

[0106] For example, as Figure 4 shown, after the focus tracking decision module decides the target subject, it transmits the position of the target subject (i.e., the position of the human body frame) to the autofocus module, so that the autofocus module adjusts the focus of the camera lens to ensure that the shooting object is always within the focus range, so that the shooting object is clearer.

[0107] The target subject decision strategy for a multi-person scenario in the automatic focus tracking mode is as follows:

[0108] ① Priority of different types of frames: face frame > body frame. That is, in the same image with multiple people, the priority of the person with a face is higher than that of the person without a face (such as facing away from the camera).

[0109] For example, as Figure 6 shown, the preview window 150 shows a picture including three people A, B, and C. Among them, person A is facing the camera lens, while person B and C are facing away from the camera. The perception detection module can detect the face frame of person A, but cannot detect the face frames of person B and C. That is, person A has a face frame, while person B and C only have body frames. In this scenario, the focus tracking decision module decides that person A is the target subject. Subsequently, the AI depth blurring module will perform corresponding blurring processing on person B and C.

[0110] ② Priority of the same type of frames: Sort according to the size of the face frame / body frame from large to small, and the largest frame has the highest priority.

[0111] For example, as Figure 7 shown, the preview window picture includes three people A, B, and C, and all three are facing the camera, that is, the perception detection module can detect the face frames of all three people. Since the distances between the three people and the camera are different, that is, the depths of field are different, the sizes of the faces of the three people are different. The face of the person with the smallest depth of field is the largest, and the face of the person with the largest depth of field is the smallest. In this example, the face sizes of the three people are A < B < C in sequence. In this scenario, the focus tracking decision module decides that person C is the target subject. Subsequently, the AI depth blurring module will perform blurring processing on person A and B.

[0112] In another scenario, if multiple people in the preview picture are all facing away from the camera and the depths of field of the three people are different. In this scenario, the perception detection module can only detect the body frames of the three people, and the sizes of the body frames of the three people are different. Similar to the scenario with all face frames, the focus tracking decision module decides that the person with the largest body frame is the target subject.

[0113] In addition, the user can manually select any person as the target subject in the preview window of the camera application. For example, Figure 8 shown in the scenario, the focus tracking decision module automatically decides that user C is the target subject. After the user clicks on the area where person B is located, the focus tracking decision module manually selects the focus tracking state, that is, confirms that person B is the target subject and no longer performs automatic focus tracking logic judgment.

[0114] S109, the focus tracking decision module transmits the body detection results of all subjects and the serial number of the target subject to the AI depth blurring module.

[0115] In a multi-person scenario, the AI depth blurring module needs to know the accurate positions of all subjects in the image and the serial number of the target subject in order to distinguish different subjects and perform blurring processing according to the depth information of different subjects to obtain different blurring effects.

[0116] In an exemplary embodiment, the focus tracking decision module can transmit the human detection results of all subjects to the AI depth blurring module, and transmit the serial number of the determined target subject to the AI depth blurring module, so that the AI depth blurring module can obtain information such as the positions and key points of all subjects in the picture.

[0117] In another exemplary embodiment, the AI depth blurring module can directly obtain the human detection results of all subjects from the output interface of the perception detection module. In this case, the focus tracking decision module only needs to transmit the ID of the target subject to the AI depth blurring module, so that the AI depth blurring module can distinguish the target subject from the non-target subjects.

[0118] In an exemplary embodiment, the focus tracking decision module can directly transmit the ID of the determined target subject to the AI depth blurring algorithm.

[0119] In addition, the focus tracking decision module can also transmit the position information of the target subject (or the position information of all subjects) to the camera application, so that the camera application can display the position of the target subject or other subjects on the video recording interface. Here, the position information can be the face box position or the human body box position of the subject.

[0120] S110. The AI depth blurring module uses the coordinate transformation matrix transmitted by the anti-shake module to perform coordinate transformation on the first coordinate box positions of each subject to obtain the corresponding second human body box positions of each subject.

[0121] To ensure that the subject coordinates detected in the Tiny stream can be correctly mapped to the preview video stream, it is necessary to perform coordinate transformation on the subject position coordinates based on the coordinate matrix of the crop and warp processing, that is, use the coordinate transformation matrix obtained by the EIS module to perform the EIS process on the coordinate box (i.e., the human body box) in the Tiny stream to ensure that the converted human body box or face box corresponds to the subject position in the video stream.

[0122] As Figure 4 shown, the AI depth blurring module includes a box coordinate transformation module, a human body box transformation module, and an AI depth blurring algorithm. The box coordinate transformation module uses the coordinate transformation matrix transmitted by the anti-shake module (such as the crop and warp transformation matrix) to transform the face box or human body box corresponding to each subject, as well as the coordinate positions of the human body key points, to obtain the transformed human body key point coordinates, face box coordinates or human body box coordinates. Further, the box coordinate transformation module transmits the transformed human body key point coordinates, face box coordinates or human body box coordinates to the human body box transformation module.

[0123] S111. The AI depth blurring module re-determines the third human body box position of each subject according to some human body key points, and the area of the third human body box is smaller than that of the first human body box.

[0124] The AI depth blurring algorithm performs depth estimation on the pixel points within the human body frame to obtain the depth information of the subject, which is also called the main depth of field. If the human body frame includes a large area of hollowed-out regions, the hollowed-out regions may be the areas where the scenery / objects outside the human body are located, which may lead to inaccurate subject depth values obtained.

[0125] For example, as Figure 9 shown in (1) of, the human body is in a state where both arms are extended horizontally to the sides. The human body detection algorithm detects the regions of the human head, torso, and limbs and determines them as the regions where the human body is located, that is, the region where the human body frame 160 is located. As shown in the figure, in addition to the human body, the human body frame 160 includes many background regions. When further matching the depth value of the human body based on the pixel points of the human body frame, the depth values of the background pixel points will be introduced, resulting in inaccurate estimation of the human body's depth value.

[0126] To ensure that the human body frame covers more than 90% of the subject area, the embodiment of the present application converts the human body frame into a human body frame with a smaller area that only includes the head and torso parts. In one embodiment, the position of the human body frame can be re-determined by using the human key points corresponding to the human torso part, such as the head key point, shoulder key point, and torso key point.

[0127] As Figure 4 shown, the human body frame conversion module receives the converted human key point coordinates and the human body frame coordinates transmitted by the frame coordinate conversion module, and re-determines the human body frame coordinates of the human body according to the coordinates of the partial human key points corresponding to the human body (such as the head, shoulders, and torso), and transmits the new human body frame coordinates to the AI depth blurring algorithm.

[0128] For example, as Figure 9 shown in (1) of, the human body frame 160 is converted into a human body frame 161 with a smaller area that includes the head and torso regions. Another example, as Figure 9 shown in (2) of, the human body is in a state where both arms are extended forward towards the body. In this posture, the human body frame detected by the human body detection algorithm is 162, and the human body frame 163 is obtained by using the key points of the head, shoulders, and torso.

[0129] The converted human body frame covers more than 90% of the subject area, so that the AI depth blurring algorithm can obtain a more accurate subject depth value based on the depth information of the pixel points of the converted human body frame.

[0130] S112. The AI depth blurring module determines the depth values of each subject based on the third human body frame positions of each subject by using the AI depth estimation algorithm.

[0131] The AI depth blurring algorithm performs depth estimation on the pixel points within the human body frame to obtain the depth information of the subject, which is also called the main depth of field.

[0132] S113. The AI depth blurring module blurs each non-target subject based on the depth of the target subject, such that non-target subjects with different depths relative to the target subject have different blurring intensities.

[0133] The AI depth blurring algorithm determines the depth value of the target subject and the depth values of each non-target subject according to the human body frame coordinates of each received subject and the ID of the target subject. Based on the depth value of the target subject and the relative depth between each non-target subject and the target subject, each non-target subject (or called the portrait background) is blurred, such that the target subject remains clear and non-target subjects with different relative depths from the target subject also have different blurring intensities.

[0134] For example, in the scenario shown in Figure 7 , person C is the target subject, and persons A and B are non-target subjects. Moreover, the distances between person A, B and person C are all different, and the distance between person A and person C is greater than the distance between person B and person C. In this scenario, the blurring intensity of person A is greater than that of person B.

[0135] Another example, as shown in Figure 8 , person B is the target subject, and both persons A and C are non-target subjects (i.e., the background of the person type). If the distances between person A and C and person B are the same, then the blurring intensities of person A and C are the same.

[0136] The above content focuses on introducing the blurring process of the portrait background. This method is also applicable to the blurring process of non-portrait backgrounds. For example, for other types of backgrounds located at different depths of field, different blurring intensity effects can also be achieved, that is, the blurring process provided in this application is applicable not only to the background of the person type but also to other types of backgrounds.

[0137] S114. The AI depth blurring module transmits the blurred image frame to the camera application.

[0138] S115. The camera application displays the preview video stream composed of the continuously received blurred image frames in the preview window.

[0139] In this embodiment, S102 - S104 and S110 - S113 are the processing processes of the preview video stream, and S105 - S109 are the processing processes of the Tiny stream. The preview video stream processing process and the Tiny stream processing process can be executed in parallel.

[0140] In addition, the embodiments of this application are all described by taking the portrait background blurring process in the video recording scenario as an example. The image blurring method of this application is also applicable to the photographing scenario. The blurring process is similar to the video scenario and will not be elaborated here.

[0141] The image blurring processing method provided in this embodiment determines the target subject and non-target subjects in a multi-person scenario, further determines the depth positions of each subject, and performs corresponding blurring processing on each non-target subject based on the relative depth distances between each non-target subject and the target subject, so that the blurring intensities corresponding to the subjects with different depths relative to the target subject are also different, that is, the target subject is clear, and the blurring intensities of the non-target subjects with different depths from the target subject are also different, making the layering of the blurring effect more in line with the optical blurring effect of a professional camera.

[0142] In some embodiments, the camera application is in the automatic focus tracking mode. As the recording time progresses, the focus tracking target may change. To avoid the focus tracking target jumping back and forth when multiple faces or multiple human bodies are of similar size, and to avoid the focus tracking target jumping due to a face briefly entering the frame, the image blurring processing method provided in this application further includes a focus stability processing process, that is, when the target subject changes, the focus tracking target is not immediately switched, but rather, after determining that the new focus tracking target is stable, the focus is switched to the new target subject.

[0143] As Figure 10 shown, the focus switching stability processing process may include the following steps:

[0144] S201, in the automatic focus tracking mode, the focus tracking decision module continuously receives the position information of all subjects in the captured image.

[0145] In some embodiments, the position information may be the face frame position or the human body frame position of the subject.

[0146] For example, in one scenario, the subject is facing the camera and the captured image includes all body parts of the subject. In this scenario, the face frame and the human body frame of the subject can be detected. Therefore, the position information of the subject includes the face frame position and the human body frame position.

[0147] In another scenario, the subject is facing away from the camera. In this scenario, only the human body frame of the subject can be detected. Therefore, the position information of the subject is the human body frame position.

[0148] In yet another scenario, the subject is facing the camera and the captured image does not include body parts. In this scenario, only the face frame of the subject can be detected. Therefore, the position information of the subject is the face frame position.

[0149] S202, the focus tracking decision module determines that the target subject of the first image frame is the first person according to the focus tracking strategy.

[0150] Exemplarily, based on the focus tracking strategy, it is determined that the current target subject is the first person. For example, the captured image is as Figure 6As shown, that is, person A is facing the camera directly, while person B and person C are both facing away from the camera. According to the focusing strategy that the face has a higher priority than the body, person A is determined as the current focusing target, that is, the target subject.

[0151] S203. The focusing decision module determines that the target subject in the second image frame is the second person according to the focusing strategy.

[0152] As the video recording time progresses, the postures of person B and person C change. For example, from the Figure 6 posture shown to the Figure 7 posture shown, that is, person B and person C change from facing away from the camera to facing the camera directly. According to the focusing strategy that the larger the area of the same type of frame, the higher the priority, user C is determined as the new focusing target, that is, the new target subject.

[0153] S204. The focusing decision module continuously tracks for the first time period. After determining that the target subject is stable as the second person, the focusing target switches to the second person, and the ID of the second person is transmitted to the AI depth blurring module.

[0154] The first time period can be set according to actual needs. In addition, the time period here can be measured by the number of image frames. For example, the first time period can be 8 - 10 frames of images.

[0155] When the focusing decision module detects a change in the target subject, it does not immediately switch the focusing target, but judges the stability of the new focusing target. For example, continuously track the target subject in 8 - 10 frames of images. If the target subject in these image frames is the second person, it is determined that the new focusing target is stable. If the target subject changes again during the continuous tracking process, start continuous tracking again from the image frame where the target subject changes.

[0156] For example, starting from the decision that the target subject changes from the first person to the second person, after continuously tracking for 4 frames, it is detected that the target subject changes from the second person to another person different from the first and second persons (such as the third person). Then start continuous tracking again from the next frame. If the target subject is stable as the third person after continuously tracking for 8 - 10 frames, it is determined that the new target subject is the third person.

[0157] After the stability processing of the focusing process, it can avoid the focusing target from jumping back and forth caused by the similar sizes of the face frame or body frame, and also avoid the focusing target from jumping caused by a face that briefly enters (for example, the face of a passerby who suddenly enters the picture during video recording), ensuring the stable switching of the focusing target.

[0158] S205. The AI depth blurring module determines the depth of each subject and determines the blurring intensity corresponding to other persons in the image based on the depth distance between other persons and the second person.

[0159] The AI depth blurring module receives the ID of the new target subject passed by the focus tracking decision module and the converted human body frame position corresponding to the ID to determine the depth information of the target subject, and determines the depth information of other persons based on the converted human body frame positions of other persons. Further, based on the depth relationship between other persons and the target subject, the blurring intensity of other persons is determined. For example, the greater the depth from the target subject, the stronger the blurring intensity; conversely, the smaller the depth from the target subject, the weaker the corresponding blurring intensity.

[0160] S206, within the second time period, the AI depth blurring module gradually changes the image blurring effect with the first person as the target subject to the image blurring effect corresponding to the second person as the target subject.

[0161] After the target subject changes, the blurring effect will also change accordingly. The AI depth blurring algorithm does not immediately switch the blurring focus to achieve smooth blurring processing. The AI depth blurring algorithm will synchronously track the positions of multiple subjects. After receiving the detection result that the target subject has changed, it will complete the transition of the blurring effect within a certain period of time (such as 10 frames). For example, the original target subject gradually becomes blurred, the new target gradually becomes clear, and the blurring of other subjects or scenes is reprocessed based on the depth of the new target. This avoids the abrupt change of the recorded video image caused by the immediate change of the blurring focus.

[0162] In some embodiments, the transition process of the blurring effect conforms to the Bezier curve model. As Figure 11 shown, it is the Bezier curve of the blurring effect transition process. Within the time from T1 to T2, the blurring intensity of the subject changes from the initial blurring intensity to the target blurring intensity. Specifically, in the initial stage, that is, at the beginning of T1, the change rate of the curve is small, which means the change of the blurring effect is slow. In the middle stage, the change rate of the curve is large, that is, the change of the blurring effect is fast. In the last stage, the change rate of the curve becomes small again, that is, the change of the blurring effect is slow.

[0163] The above process is the process of smooth transition of the blurring effect when the focus tracking target changes in the automatic focus tracking mode. This process also applies to the manual focus tracking mode. For example, after the focus tracking decision module determines the new target subject manually selected by the user, it passes the serial number of the new target subject to the AI depth blurring module. Within the second time period, the AI depth blurring module gradually changes the image blurring effect with the original target to the image blurring effect corresponding to the new target subject.

[0164] The focus target switching process provided in this embodiment does not switch the focus target when a change in the focus target is detected. Instead, the focus target is switched to the new target subject after determining that the target is stable. In this way, the focus target is avoided from jumping back and forth due to the similar size of the face frame or the body frame, and the focus target is avoided from jumping due to the short-term entry of the face (for example, the face of a passerby who suddenly breaks into the picture during the video recording), thereby ensuring the stable switching of the focus target. Moreover, the blurring process after the focus target is switched in this embodiment also adopts smoothing processing, that is, the transition of the blurring effect after the focus target is switched is completed within a period of time, for example, the original target subject gradually becomes blurred from clear, the new target gradually becomes clear from blurred, and other subjects or scenes are blurred again based on the depth of the new target. In this way, abrupt changes in the video picture caused by the immediate change of the blurred focus are avoided.

[0165] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform all or part of the steps of the method described in each embodiment. The aforementioned storage medium includes: flash memory, mobile hard disk, read-only memory, random access memory, disk or optical disk and other media that can store program codes.

[0166] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An image blurring processing method, characterized in that, applied to an electronic device, the method includes: displaying a first interface, the first interface includes a preview window and a blurring button, the preview window displays a first image, the image includes a first portrait, a second portrait and a third portrait, the first portrait is the focusing target, and the depths of the three portraits are different from each other; in response to an operation on the blurring button, the preview window displays a second image, the second image includes a clear first portrait, a second portrait with a first blurring intensity, and a third portrait with a second blurring intensity, the first blurring intensity is different from the second blurring intensity.

2. The method according to claim 1, characterized in that, the second blurring intensity is positively correlated with the first relative depth, the first relative depth is the relative depth between the second portrait and the first portrait; the third blurring intensity is positively correlated with the second relative depth, the second relative depth is the relative depth between the third portrait and the first portrait.

3. The method according to claim 1 or 2, characterized in that, the method further includes: the preview window displays a third image, the third image includes a first portrait, a second portrait and a third portrait, and the focusing target changes from the first portrait to the second portrait; within a first preset time period, the image displayed in the preview window gradually changes from a first blurring effect image with the first portrait as the focusing target to a second blurring effect image with the second portrait as the focusing target, the second blurring effect image includes a clear second portrait, a first portrait with a third blurring intensity and a third portrait with a fourth blurring intensity, the third blurring intensity is different from the fourth blurring intensity.

4. The method according to claim 3, characterized in that, the third blurring intensity is positively correlated with the third relative depth, the fourth blurring intensity is positively correlated with the fourth relative depth, the third relative depth is the relative depth between the first portrait and the second portrait, and the fourth relative depth is the relative depth between the third portrait and the second portrait.

5. The method according to any one of claims 1-4, characterized in that, the response to the operation on the blurring button, and the preview window displays the second image, including: in response to the operation on the blurring button, based on the first position information of each portrait in the first image, using an AI depth estimation algorithm to determine the depth of each portrait; taking the depth of the first portrait as a reference, based on the depth of the second person, blurring the second person to obtain a second portrait with a first blurring intensity; taking the depth of the first portrait as a reference, based on the depth of the third person, blurring the third person to obtain a third portrait with a second blurring intensity.

6. The method according to claim 5, characterized in that, taking the depth of the first portrait as a reference, based on the depth of the second person, blurring the second person to obtain a second portrait with a first blurring intensity, including: determining the first blurring intensity corresponding to the second portrait according to the relative depth between the second portrait and the first portrait; Blur the second portrait so that the second portrait reaches the first blurring intensity.

7. The method according to claim 5, wherein, The process of obtaining the first position information of each portrait in the first image includes: Downsample the first image to obtain a first-resolution image; Detect the human bodies included in the first-resolution image to obtain a human body detection result, the human body detection result includes the second position information of the human body and the first human body key points, the second position information includes the position of the first human body frame, and the first human body frame includes a square area of all body parts included in any portrait in the first image; Use a coordinate transformation matrix to perform coordinate transformation on the position of the first human body frame and the coordinates corresponding to the first key points to obtain the position of the second human body frame and the second key points. The coordinate transformation matrix is generated during the anti-shake process of the first image. The first position information includes the position of the second human body frame and the second key points.

8. The method according to claim 7, wherein, Based on the first position information of each portrait in the first image, use an AI depth estimation algorithm to determine the depth of each portrait, including: For any portrait, determine the position of the third human body frame of the portrait based on the position of the second human body frame corresponding to the portrait and the second human body key points of the preset part. The position of the third human body frame includes the head and torso parts of the portrait; For any portrait, use an AI depth estimation algorithm to estimate the depth of the pixel points within the third human body frame corresponding to the portrait to obtain the depth of the portrait.

9. The method according to any one of claims 1-8, wherein, The electronic device includes a focus tracking decision module and an AI depth blurring module; The process of determining the first portrait as the focus tracking target includes: The focus tracking decision module determines, based on an automatic focus tracking strategy and the human body detection result corresponding to the first image, that the focus tracking target is the first portrait, and transmits the serial number of the first portrait to the AI depth blurring module; The preview window displays a second image in response to an operation on the blurring button, including: The AI depth blurring module, in response to an operation on the blurring button, determines the first portrait as the focus tracking target based on the serial number of the first portrait, and blurs the second portrait and the third portrait respectively with the depth of the first portrait as a reference to obtain a second portrait with the first blurring intensity and a third portrait with the second blurring intensity.

10. The method according to claim 3, wherein, The electronic device includes a focus tracking decision module; The preview window displays a third image, including: The focus tracking decision module determines, based on the human body detection result corresponding to the fourth image, that the focus tracking target is the second portrait. The shooting time of the fourth image is later than that of the first image; The focus tracking decision module continuously tracks the focus tracking target. If the focus tracking target is always the second portrait within the second preset time period, the focus tracking target is switched from the first portrait to the second portrait.

11. The method according to claim 10, wherein, The electronic device includes an AI depth blurring module; After the focus tracking decision module switches the focus tracking target from the first portrait to the second portrait, it transmits the serial number of the second portrait to the AI depth blurring module; Within a first preset time period, the AI depth blurring module gradually changes the first blurring effect image with the first portrait as the focus tracking target to the second blurring effect image with the second portrait as the focus tracking target.

12. The method according to claim 3, 4 or 11, wherein, within the first preset time period, the image displayed in the preview window gradually changes from the first blurring effect image with the first portrait as the focus tracking target to the second blurring effect image with the second portrait as the focus tracking target, including: After receiving the information that the focus tracking target is switched to the second portrait, determining the depth of each object in the current image frame, where the object includes the portrait object and other shooting objects in the image; Determining the target blurring intensity corresponding to the depth of other objects in the image based on the depth of the second portrait; Within the first preset time period, controlling any other object to gradually change from the current blurring intensity corresponding to the any other object to the target blurring intensity corresponding to the any other object, and controlling the second portrait to gradually change from the current blurring intensity corresponding to the second portrait to clear.

13. The method according to claim 12, wherein, within the first preset time period, controlling any other object to gradually change from the current blurring intensity corresponding to the any other object to the target blurring intensity corresponding to the any other object, includes: Taking the start moment of the first preset time period as the start moment of the Bézier curve, taking the end moment of the first preset time period as the end moment of the Bézier curve, and taking the current blurring intensity of the any other object as the start point and the target blurring intensity of the any other object as the end point to generate a smooth Bézier curve corresponding to the any other object; At any moment within the first preset time period, searching for the target blurring intensity corresponding to the moment based on the smooth Bézier curve, and performing blurring processing on the any other object to the target blurring intensity corresponding to the any moment.

14. A chip system, wherein, it includes: At least one processor and an interface, where the interface is used to receive code instructions and transmit them to the at least one processor; The at least one processor runs the code instructions to implement the image blurring processing method according to any one of claims 1-13.

15. An electronic device, wherein, the electronic device includes: one or more processors, a memory, and a touch screen; the memory is used to store program code; the processor is used to run the program code so that the electronic device implements the image blurring processing method according to any one of claims 1 to 13.

16. A computer-readable storage medium, wherein, instructions are stored thereon, and when the instructions are run on an electronic device, the electronic device is caused to execute the image blurring processing method according to any one of claims 1 to 13.

Images