Image blurring processing method and device

Abstract

The application provides an image blurring processing method and device. In a multi-person scene, a target subject and a non-target subject are determined, the depth of each person is further determined, and the depth of each non-target subject is used as a reference to perform corresponding blurring processing, so that the blurring intensity corresponding to the depth of the target subject is different, that is, the target subject is clear, and the blurring intensity of the non-target subject with different depths from the target subject is also different, so that the level of the blurring effect is more consistent with the optical blurring effect of a professional camera.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image blurring processing method and apparatus. Background Technology

[0002] Current electronic devices such as mobile phones and digital cameras can blur the background of portraits during video recording, especially when shooting people, to make the subjects stand out. Current portrait background blurring solutions use portrait segmentation technology, dividing all portraits in the video image into subjects. That is, all portraits in the image are treated as subjects, remaining sharp, while other parts of the image are blurred as background. This solution produces a blurring effect with poor depth of field. Summary of the Invention

[0003] In view of this, this application provides an image blurring processing method and apparatus to solve at least some of the above-mentioned problems, and the disclosed technical solution is as follows:

[0004] Firstly, this application provides an image blurring processing method applied to an electronic device. The method includes: displaying a first interface, the first interface including a preview window and a blur button; the preview window displays a first image, the image including a first portrait, a second portrait, and a third portrait, the first portrait being the focus target, and the three portraits having different depths; in response to an operation of the blur button, the preview window displays a second image, the second image including a clear first portrait, a second portrait with a first blur intensity, and a third portrait with a second blur intensity, the first blur intensity being different from the second blur intensity. It is evident that this solution achieves different blur intensities for subjects with different depths relative to the target subject, i.e., the target subject is clear, and the blur intensities for non-target subjects with different depths relative to the target subject are also different, making the layering of the blur effect more consistent with the optical blur effect of a professional camera.

[0005] In one possible implementation of the first aspect, the second blur intensity is positively correlated with the first relative depth, which is the relative depth between the second portrait and the first portrait; the third blur intensity is positively correlated with the second relative depth, which is the relative depth between the third portrait and the first portrait.

[0006] In one possible implementation of the first aspect, the method further includes: displaying a third image in a preview window, the third image including a first portrait, a second portrait, and a third portrait, with the focus target changing from the first portrait to the second portrait; within a first preset time period, the image displayed in the preview window gradually changes from a first bokeh effect image with the first portrait as the focus target to a second bokeh effect image with the second portrait as the focus target, the second bokeh effect image including a clear second portrait, a first portrait with a third bokeh intensity, and a third portrait with a fourth bokeh intensity, the third bokeh intensity being different from the fourth bokeh intensity. It can be seen that this scheme, after switching the focus target, gradually transitions from a bokeh effect dominated by the original focus target to a bokeh effect dominated by the new target, avoiding the abruptness of immediately switching the bokeh effect after switching the focus target.

[0007] In one possible implementation of the first aspect, the third blur intensity is positively correlated with the third relative depth, and the fourth blur intensity is positively correlated with the fourth relative depth. The third relative depth is the relative depth between the first portrait and the second portrait, and the fourth relative depth is the relative depth between the third portrait and the second portrait.

[0008] In one possible implementation of the first aspect, in response to an operation on the blur button, a preview window displays a second image, including: in response to an operation on the blur button, determining the depth of each portrait using an AI depth estimation algorithm based on the first position information of each portrait in the first image; blurring the second portrait based on the depth of the first portrait and the depth of the second portrait to obtain a second portrait with a first blur intensity; and blurring the third portrait based on the depth of the first portrait and the depth of the third portrait to obtain a third portrait with a second blur intensity.

[0009] In one possible implementation of the first aspect, the second person is blurred based on the depth of the first person and the depth of the second person to obtain a second person with a first blur intensity, including: determining the first blur intensity corresponding to the second person based on the relative depth between the second person and the first person; blurring the second person so that the second person reaches the first blur intensity.

[0010] In one possible implementation of the first aspect, the process of acquiring the first position information of each human figure in the first image includes: downsampling the first image to obtain a first resolution image; detecting human figures contained in the first resolution image to obtain human detection results, the human detection results including second position information of the human figure and first human key points, the second position information including the position of the first human bounding box, the first human bounding box including a square region containing all body parts contained in any human figure in the first image; and using a coordinate transformation matrix to perform coordinate transformation on the coordinates corresponding to the first human bounding box position and the first key point to obtain the second human bounding box position and the second key point, the coordinate transformation matrix being generated during the image stabilization process of the first image, the first position information including the second human bounding box position and the second key point. In this way, after performing the same coordinate transformation as EIS processing on the human figure position coordinates obtained based on the first resolution image detection, it can be directly mapped to the first image after EIS processing, thereby performing depth estimation and blurring processing on each subject in the first image, improving the accuracy of the depth estimation results.

[0011] In one possible implementation of the first aspect, based on the first position information of each portrait in the first image, the depth of each portrait is determined using an AI depth estimation algorithm. This includes: for any portrait, determining the position of a third portrait based on the position of a second human body frame corresponding to that portrait and a preset portion of second human body key points, wherein the third human body frame position includes the head and torso of the portrait; and for any portrait, estimating the depth of the pixels within the third human body frame corresponding to that portrait using the AI ​​depth estimation algorithm to obtain the depth of that portrait. In this way, the human body frame determined based on a subset of human body key points covers more than 90% of the human body, thereby making the human body depth information determined based on this human body frame more accurate and further improving the blurring effect of the human body.

[0012] In one possible implementation of the first aspect, the electronic device includes a focus decision module and an AI deep bokeh module;

[0013] The process of determining the first portrait as the focus target includes: the focus decision module determines the focus target as the first portrait based on the automatic focus strategy and the human body detection results corresponding to the first image, and transmits the sequence number of the first portrait to the AI ​​deep blur module;

[0014] In response to the operation of the blur button, the preview window displays a second image, including: The AI ​​depth blur module, responding to the operation of the blur button, determines the first portrait as the focus target based on its sequence number, and performs blur processing on the second and third portraits respectively, using the depth of the first portrait as a reference, to obtain the second portrait with a first blur intensity and the third portrait with a second blur intensity. It can be seen that the focus decision module transmits the sequence number of the decided focus target (i.e., the target's unique identifier) ​​to the AI ​​depth blur module, enabling the AI ​​depth blur module to know which human body is the focus target, and then blur other subjects based on the depth of the focus target.

[0015] In one possible implementation of the first aspect, the electronic device includes a focus decision module; a preview window displays a third image, and the focus decision module determines that the focus target is a second human image based on the human detection result corresponding to the fourth image, wherein the fourth image was captured later than the first image; the focus decision module continuously tracks the focus target, and if the focus target remains the second human image for a second preset time period, the focus target is switched from the first human image to the second human image. This ensures that the focus target is switched only after it is determined to be stable, avoiding frequent changes in the focus target and preventing brief appearances of faces from causing the focus target to change, thus improving the stability of the focus target.

[0016] In one possible implementation of the first aspect, the electronic device includes an AI deep bokeh module; after the focus tracking decision module switches the focus target from the first portrait to the second portrait, it transmits the sequence number of the second portrait to the AI ​​deep bokeh module; within a first preset time period, the AI ​​deep bokeh module gradually transforms the first bokeh effect image with the first portrait as the focus target into a second bokeh effect image with the second portrait as the focus target.

[0017] In one possible implementation of the first aspect, within a first preset duration, the image displayed in the preview window gradually changes from a first bokeh effect image with a first portrait as the focus target to a second bokeh effect image with a second portrait as the focus target, including: after receiving information that the focus target has switched to the second portrait, determining the depth of each object in the current image frame, the objects including portrait objects and other shooting objects in the image; determining the target bokeh intensity corresponding to the depth of other objects in the image based on the depth of the second portrait; within the first preset duration, controlling any other object to gradually change from the current bokeh intensity corresponding to any other object to the target bokeh intensity corresponding to any other object, and controlling the second portrait to gradually change from the current bokeh intensity corresponding to the second portrait to clear.

[0018] In one possible implementation of the first aspect, within a first preset duration, controlling any other object to gradually change from its current blur intensity to its target blur intensity includes: using the start time of the first preset duration as the start time of the Bézier curve, the end time of the first preset duration as the end time of the Bézier curve, and using the current blur intensity of any other object as the start point and the target blur intensity of any other object as the end point, generating a smooth Bézier curve corresponding to any other object; at any time within the first preset duration, finding the target blur intensity corresponding to that time based on the smooth Bézier curve, and blurring any other object to the target blur intensity corresponding to that time.

[0019] Secondly, this application also provides a chip system including at least one processor and an interface, the interface being used to receive code instructions and transmit them to at least one processor; the at least one processor executes the code instructions to implement the image blurring processing method of any of the first aspects.

[0020] Thirdly, this application also provides an electronic device comprising: one or more processors, a memory, and a touch screen; the memory for storing program code; and the processor for running the program code, thereby enabling the electronic device to implement the image blurring processing method of any one of the first aspects.

[0021] Fourthly, this application also provides a computer-readable storage medium having instructions stored thereon, which, when executed on an electronic device, cause the electronic device to perform the image blurring processing method as described in any of the first aspects.

[0022] Fifthly, this application also provides a computer program product having stored on it an executable that, when run on an electronic device, causes the electronic device to implement the image blurring processing method as described in any of the first aspects. Attached Figure Description

[0023] Figure 1 This is a schematic diagram of a video recording interface provided in an embodiment of this application;

[0024] Figure 2 This is a schematic diagram of an interface for shooting a movie, provided in an embodiment of this application;

[0025] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0026] Figure 4 This is a schematic diagram of the structure of an image blurring processing system provided in an embodiment of this application;

[0027] Figure 5This is a flowchart of an image blurring processing method provided in an embodiment of this application;

[0028] Figure 6 This is a schematic diagram illustrating the recording of a video containing a human image, provided in an embodiment of this application.

[0029] Figure 7 This is a schematic diagram illustrating another method for recording video containing human images, provided in an embodiment of this application.

[0030] Figure 8 This is a schematic diagram illustrating another method of recording video containing human images, provided in an embodiment of this application.

[0031] Figure 9 This is a schematic diagram of a human body frame provided in an embodiment of this application;

[0032] Figure 10 This is a flowchart illustrating a target switching process provided in an embodiment of this application;

[0033] Figure 11 This is a schematic diagram of the transition process of a blurring effect provided in an embodiment of this application. Detailed Implementation

[0034] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be a limitation of this application.

[0035] This application provides an image blurring processing method, which can be applied to electronic devices with cameras that can provide shooting functions, such as digital cameras, mobile phones, and tablet computers.

[0036] When an electronic device detects a user activating the portrait blur button while recording video, it uses an AI depth estimation algorithm to calculate the depth values ​​of objects in the video image. The blurring is then applied according to the background depth values, with different degrees of blur corresponding to different depths, resulting in a more layered blurring effect. Furthermore, in multi-person scenes, only the main subject remains sharp, while other subjects are blurred according to their depth values, achieving different degrees of blur for people at different depths, which more closely resembles the blurring effect of a realistic depth-of-field lens.

[0037] In one embodiment of this application, the portrait background blur function can be enabled in the video recording mode or movie mode of the camera application. The user interface for enabling portrait background blur in these two scenarios will be described below.

[0038] like Figure 1 As shown, taking a mobile phone as an example, this document introduces the user interface for enabling portrait background blur in a video recording scenario.

[0039] Figure 1(1) is a schematic diagram of the mobile phone desktop. The status bar 101 is located at the top of the desktop 100, the general program tray 102 is located below the status bar 101, and the frequently used program tray 103 is located below the general program tray 102.

[0040] The status bar displays the phone's current status, such as cellular signal quality, wireless network signal quality, battery status, and real-time information.

[0041] Both the general application tray 102 and the frequently used application tray 103 are used to hold application icons. The icon of any application can be placed in either the frequently used application tray 103 or the general application tray 102. Users can activate the application corresponding to that icon by clicking it. For example, in this embodiment, a user can click the camera application icon 104 in the general application tray 102, the phone will run the camera application, and the phone screen will display the main interface of the camera application. Figure 1 The interface shown in (2) is as follows.

[0042] like Figure 1 As shown in (2), the main interface of the camera application may include a settings bar 110 at the top, a preview window 113 below the settings bar and the largest in area, a menu bar 117 below the preview window 113, and a control area below the menu bar, such as a shooting control 119, a playback button 118 and a camera flip control 120.

[0043] The settings panel 110 may include multiple settings controls for adjusting shooting parameters. For example, flash control 111 and settings control 112. In addition, it may include filter controls, auto-follow controls, etc. This application does not specifically limit the types of controls included in the settings panel.

[0044] Menu bar 117 may include multiple shooting mode options, such as "Portrait," "Photo," "Video," and "Movie." For example, Figure 1 The interface shown in (2) is the main interface of the recording mode.

[0045] The menu bar 117 can only display a limited number of options, and some mode options cannot be displayed. You can slide the menu bar left and right to display other mode options.

[0046] The preview window 113 is used to display images captured by the camera in real time. In this embodiment, the bottom of the preview window 113 also includes a background blur button 114, a zoom selection button 115, and a beauty button 116. After the user clicks the background blur button 114, the background blur function is activated, and the user is redirected to... Figure 1 The interface shown in (3) is as follows.

[0047] like Figure 1 As shown in (3), the background 131 in the captured image, excluding the target subject 130, is blurred to highlight the portrait. Furthermore, a blur degree scale 132 can be displayed at the bottom of the preview window 113. Sliding the blur degree scale 132 left or right adjusts the degree of background blur. For example, in this embodiment, the blur degree scale 132 indicates that the blur degree gradually increases from weak to strong from left to right. A stronger blur degree indicates a higher degree of blur, i.e., a more blurred image. Conversely, a weaker blur degree indicates a lower degree of blur, i.e., a clearer image.

[0048] In another scenario, the portrait blurring function can also be enabled when the phone's camera app is in movie mode.

[0049] like Figure 2 As shown in (1), after the user clicks the camera application icon 102 on the desktop 101, they are redirected to the main interface of the camera application. For example, in this embodiment, as shown in (1), the user is redirected to the main interface of the camera application. Figure 2 The main interface of the movie mode shown in (2) is as follows.

[0050] The electronic devices covered by this application are not limited to mobile phones, tablets, and digital cameras, but can also be other electronic devices with cameras, such as desktop computers, laptops, handheld computers, notebook computers, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), augmented reality (AR) devices, virtual reality (VR) devices, artificial intelligence (AI) devices, wearable devices, in-vehicle devices, smart home devices, and / or smart city devices. The embodiments of this application do not impose any special restrictions on the specific types of electronic devices.

[0051] like Figure 2 As shown in (2), the movie mode interface includes a settings bar 147 at the top, a preview window 140, a menu bar 148, a playback button 144, a movie shooting button 145, and a lens reversal button 146. The bottom of the preview window 140 includes a bokeh button 141, a zoom selection button 142, and a beauty button 143.

[0052] After the user clicks the blur button 141, the background blur function is enabled, and the user is redirected to... Figure 2 The interface shown in (3) is such that the portrait remains sharp while the background is blurred. A blur scale 149 is displayed at the bottom of the preview window. The blur scale 149 and... Figure 1 The blur scale 132 of (3) is the same, so it will not be repeated here. Adjusting the blur scale 149 can adjust the degree of background blur.

[0053] like Figure 3 As shown, the electronic device provided in this application embodiment may include a processor 201, a memory 202, a camera 203, and a display screen 204.

[0054] The processor 201 may include one or more processing units. Different processing units may be independent devices or integrated into one or more processors.

[0055] The memory 202 can be used to store computer executable program code, which includes instructions. The processor 201 executes various functional applications and data processing of the terminal device by running the instructions stored in the memory. For example, in this embodiment, the processor can perform an image blurring process by executing instructions in the memory.

[0056] Camera 203 is used to capture still images or videos. In some embodiments, the electronic device may include one or N cameras, where N is a positive integer greater than 1.

[0057] The display screen 204 is used to display images, videos, etc., and can also display a series of graphical user interfaces (GUIs), which serve as the main screen of the electronic device. In this embodiment, the display screen can be a touchscreen. Furthermore, the electronic device may include one or more displays.

[0058] The following is combined with Figure 4 This application provides a schematic diagram illustrating the structure of an image blurring processing system according to an embodiment.

[0059] like Figure 4 As shown, the system may include a camera, a perception detection module, a focus tracking decision module, a camera application, an autofocus module, an image stabilization module, an AI deep bokeh module, and a beautification module.

[0060] In some embodiments, the electronic device runs an operating system and may employ a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application's embodiments use a layered architecture... Taking the system as an example, the layered architecture divides the software into several layers, each with a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments, the Android system may include, from top to bottom, an application layer, an application framework layer, a hardware abstraction layer, and a kernel layer.

[0061] The aforementioned camera applications are applications for electronic devices, which can call upon the camera to capture images or videos.

[0062] The aforementioned perception detection module, focus tracking decision module, autofocus module, image stabilization module, AI deep bokeh module, and beautification module all belong to the algorithm modules in the camera algorithm library within the hardware abstraction layer.

[0063] The hardware abstraction layer (HAL) is located at the interface layer between the application framework layer and the kernel layer, providing a virtual hardware platform for the operating system. In this embodiment, the HAL includes a camera hardware abstraction layer and a camera algorithm library. The camera hardware abstraction layer can provide one or more virtual camera devices, which are the virtual hardware of the camera. The operating system calls the corresponding camera through the camera's virtual hardware.

[0064] In one exemplary embodiment, after the camera application starts and enters recording mode, it calls the camera in the hardware layer through the virtual camera device in the camera hardware abstraction layer to acquire image electrical signals in real time. The image electrical signals are then transmitted to the image signal processor (ISP) for processing and conversion into images visible to the naked eye. The continuously generated images constitute an image stream, i.e., video, also known as a preview video stream (e.g., with a resolution of 1080P, 4K, etc.). At the same time, the ISP downsamples the preprocessed images to obtain a low-resolution Tiny stream (e.g., with a resolution of 640P).

[0065] like Figure 4 As shown, the preview video stream is used for image stabilization, bokeh algorithms, and beautification algorithms. The Tiny stream is used for perception and detection algorithms, such as face detection and human detection.

[0066] The processing of the Tiny stream and the preview video stream can be performed in parallel. For example, the two video streams can be processed in parallel by different threads.

[0067] After the Tiny stream passes through the perception detection module, it obtains the human body detection results of the subject (i.e., person) contained in the image. The human body detection results may include the human body bounding box position, human body key points, and face bounding box position. The human body detection results are then passed to the focus tracking decision module.

[0068] In automatic focus tracking mode, the focus tracking decision module determines the target subject based on the received human detection results and the focus tracking strategy, and transmits the target subject's ID and the position information of each subject (i.e., the face frame position and the human body frame position) to the AI ​​deep blurring module.

[0069] In another embodiment, the perception detection module can also directly transmit the human body detection results to the AI ​​deep blurring module.

[0070] The image stabilization module performs image stabilization processing on the preview video stream, such as adjusting the position and orientation of the image through image deformation and cropping to keep it stable. This process can also be called electronic image stabilization (EIS). The image stabilization module then passes the stabilized image to the AI ​​deep bokeh module, along with the coordinate transformation matrix used in the image stabilization process.

[0071] In one exemplary embodiment, the AI ​​deep blurring module may include a bounding box coordinate transformation module, a human body bounding box transformation module, and an AI deep blurring algorithm module.

[0072] The pixel transformation during EIS processing is not a simple linear transformation, which means that the coordinates of the video image after image stabilization cannot be directly and easily mapped to the coordinates of the Tiny stream image. The bounding box coordinate transformation module is used to perform the same EIS processing on the human body bounding box position and face bounding box position detected by the perception detection module to obtain the transformed human body bounding box coordinates and face bounding box coordinates, thereby ensuring that the transformed human body bounding box coordinates or face bounding box coordinates correspond to the position of the subject in the video image.

[0073] The AI ​​deep blurring algorithm module needs to estimate the depth information of each human body based on the pixels within each human body bounding box in the image. If the human body bounding box covers a large area of ​​open space excluding the human body, this will lead to inaccurate depth estimation and poor blurring effect. The human body bounding box conversion module is used to convert the human body bounding boxes containing large open areas detected by the perception detection module into smaller human body bounding boxes. These smaller bounding boxes can cover more than 90% of the human body area, meaning that the pixels outside the human body within the bounding box are less than 10%. Furthermore, the human body bounding box conversion module transmits the positions of the converted human body bounding boxes to the AI ​​deep blurring algorithm.

[0074] In an exemplary embodiment, the human body frame conversion module redetermines the position of the human body frame based on the human body key points corresponding to the human torso, such as head key points, shoulder key points and torso key points.

[0075] The AI-powered deep blurring algorithm determines the depth value of the target subject and the depth value of each non-target subject based on the received human bounding box coordinates and the target subject's ID. Using the target subject's depth value as a benchmark, and based on the relative depth between each non-target subject and the target subject, the algorithm blurs each non-target subject (or portrait background) to keep the target subject sharp, and the blurring intensity varies for non-target subjects with different relative depths to the target subject.

[0076] The AI ​​deep bokeh module can pass the bokeh-processed video image to the beautification module for beautification, send it to the camera application as a preview video stream for display, and at the same time save the beautified video stream to the media library, that is, save the recording.

[0077] In addition, the focus tracking decision module can also transmit the face or body frame position of the target subject to the camera application. The camera application displays the face or body frame of the current target subject in the preview interface, so that the user can know which person the current target subject is based on the face or body frame displayed on the interface.

[0078] Furthermore, users can select other people as the target subject in the camera app's preview interface. In this scenario, the camera app detects the user's click location and transmits it to the focus tracking decision module. The focus tracking decision module enters manual focus tracking mode and determines which subject's frame the clicked location falls within based on the position information of each subject, thus identifying the target subject manually selected by the user. Once the subject to which the location belongs is determined, automatic focus tracking logic is no longer performed, and the module continues to track the manually selected subject until the subject disappears or is lost.

[0079] If the user clicks on a location that does not belong to any subject, the system will revert to autofocus mode.

[0080] In addition, the focus tracking decision module can also transmit the face or body frame corresponding to the target subject to the autofocus module so that the autofocus module can adjust the lens focus to ensure that the target subject is within the focus range.

[0081] As can be seen, by using the image blurring processing system provided in this embodiment, in a multi-person scene, the target subject and non-target subjects are identified, the depth position of each subject is further determined, and based on the relative depth distance between each non-target subject and the target subject, the corresponding blurring processing is performed on each non-target subject. This achieves that the blurring intensity of subjects with different depths relative to the target subject is also different. That is, the target subject is clear, and the blurring intensity of non-target subjects with different depths relative to the target subject is also different, making the sense of layering of the blurring effect more in line with the optical blurring effect of a professional camera.

[0082] The following will combine Figure 5 This application provides a detailed description of the image blurring process provided in its embodiments.

[0083] like Figure 5 As shown, the image blurring method may include the following steps:

[0084] When the camera app receives an operation to enable the portrait blur function in a video recording scenario, it calls the camera to record video.

[0085] S101, the camera transmits the acquired image electrical signals to the ISP.

[0086] A camera is used to capture still images or videos. An object passes through a lens, generating an optical image that is projected onto a photosensitive element. The photosensitive element converts the light signal into an electrical signal, i.e., an image electrical signal. This image electrical signal is then transmitted to an image processing unit (ISP) for processing.

[0087] In some embodiments, an electronic device may include one or more cameras, such as a front-facing camera located on the side where the screen of the electronic device is located, and a rear-facing camera located on the back of the electronic device. The number of rear-facing cameras may be one or more, without special limitation. The image blurring processing method of this application is applicable to processing videos recorded by the front-facing camera or the rear-facing camera.

[0088] S102, the ISP processes the received image electrical signal to obtain a preview video image and transmits it to the image stabilization module.

[0089] The image ISP converts the image's electrical signals into a visible image. Furthermore, the ISP can perform algorithmic optimizations on image noise, brightness, skin tone, and other aspects. The continuously generated images constitute an image stream, i.e., video, referred to here as a preview video stream.

[0090] Furthermore, the ISP can transmit the processed image to the image stabilization module for image stabilization.

[0091] S103, the image stabilization module performs image stabilization processing on the preview video image.

[0092] When shooting videos or taking photos, the original images are usually edited and processed in post-production. The position and orientation of the image are adjusted by warping and cropping to keep it stable.

[0093] In some embodiments, the image stabilization process depends on the capabilities of the ISP, meaning that the image stabilization module needs to utilize the processing capabilities of the ISP to complete the image stabilization process.

[0094] S104, the image stabilization module transmits the image after image stabilization and the coordinate transformation matrix corresponding to the image stabilization to the AI ​​deep bokeh module.

[0095] The preview video image, after being processed by the image stabilization module, is passed to the AI ​​deep blurring module. Simultaneously, the coordinate transformation matrix corresponding to the cropping and warping processes is also passed to the AI ​​deep blurring module. This allows the AI ​​deep blurring module to use this coordinate transformation matrix to perform the same coordinate transformation on the face or body bounding box positions detected based on the Tiny stream, ensuring that the coordinates of the face bounding box (or body bounding box) correspond to the coordinates in the preview video image.

[0096] S105, the ISP downsamples each image frame in the preview video stream to obtain the Tiny stream, and then passes the Tiny stream to the perception detection module.

[0097] After S102 is executed, the ISP executes S105, which means that the image electrical signal captured by the camera is processed by the ISP to obtain two video streams with different resolutions: a high-resolution preview video stream and a low-resolution Tiny stream.

[0098] S106, the perception detection module detects each image frame in the Tiny stream to obtain human detection results.

[0099] The human body detection results include the position of the face frame, the position of the first human body frame, and the positions of key points on the human body.

[0100] For example, face detection algorithms can be used to detect the location of all faces (i.e., face bounding boxes) in a video image. Human detection algorithms can be used to detect the location of all human bodies (i.e., human bounding boxes) in a video image, and human keypoint detection algorithms can be used to detect the keypoint information of all human bodies.

[0101] In some embodiments, different detection algorithms are performed in a time-sharing manner. For example, different detection algorithms detect different image frames to obtain corresponding detection results. For instance, a face detection algorithm is used to detect whether a face is contained in the first and second frames, and a human body detection algorithm is used to detect the human body and human body key points contained in the third and fourth frames.

[0102] S107, the perception and detection module transmits the human body detection results to the focus tracking decision module.

[0103] The perception and detection module transmits the human detection results to the focus tracking decision module, so that the focus tracking decision module can decide on the target subject to be tracked.

[0104] S108, the focus tracking decision module, in automatic focus tracking mode, uses a focus tracking strategy to determine the target subject based on the human body detection results.

[0105] The focus tracking decision module determines which subject in the image is currently in focus based on the focus tracking strategy, and further adjusts the focus of the camera lens to ensure that the subject is always in focus, thereby making the subject clearer.

[0106] For example, such as Figure 4 As shown, after the focus tracking decision module identifies the target subject, it transmits the position of the target subject (i.e., the position of the human body frame) to the autofocus module, thereby enabling the autofocus module to adjust the focus of the camera lens to ensure that the subject is always in focus, thus making the subject clearer.

[0107] The target subject decision-making strategy in multi-person scenes under autofocus mode is as follows:

[0108] ① Priority of different types of frames: Face frame > Body frame. That is, if an image contains multiple people, the person with a face has a higher priority than the person without a face (such as a person with their back to the camera).

[0109] For example, such as Figure 6 As shown, the preview window 150 displays three figures: A, B, and C. Figure A is facing the camera lens, while figures B and C are facing away from the lens. The perception detection module can detect the face outline of figure A, but cannot detect the face outlines of figures B and C. That is, figure A has a face outline, while figures B and C only have body outlines. In this scenario, the focus tracking decision module determines that figure A is the target subject. Subsequently, the AI ​​deep blurring module will perform corresponding blurring processing on figures B and C.

[0110] ② Priority of similar bounding boxes: Sort by face / body bounding boxes from largest to smallest, with the largest bounding box having the highest priority.

[0111] For example, such as Figure 7 As shown, the preview window displays three figures, A, B, and C, all facing the camera, meaning the perception detection module can detect their face outlines. Due to the different distances between the three figures and the camera (i.e., different depths of field), their faces are of different sizes; the figure with the smallest depth of field has the largest face, and the figure with the largest depth of field has the smallest face. In this example, the face sizes are in the order A < B < C. In this scenario, the focus tracking decision module identifies figure C as the target subject. The subsequent AI depth blur module will then blur figures A and B.

[0112] In another scenario, if multiple figures in the preview image are facing away from the camera, and their depths of field differ, the perception detection module can only detect the bounding boxes of all three figures, and these bounding boxes are of different sizes. Similar to the scenario where all figures are faces, the focus tracking decision module determines that the figure with the largest bounding box is the target subject.

[0113] In addition, users can manually select any person as the subject within the camera app's preview window. For example, Figure 8 The scene shown illustrates how the focus tracking decision module automatically identifies user C as the target subject. After the user clicks on the area where person B is located, the focus tracking decision module manually selects the focus state, confirming person B as the target subject and ceasing automatic focus tracking logic checks.

[0114] S109, the focus tracking decision module transmits the human detection results of all subjects, as well as the serial number of the target subject, to the AI ​​deep blurring module.

[0115] In multi-person scenarios, the AI ​​deep blurring module needs to know the exact location of all subjects in the image, as well as the index of the target subject, in order to distinguish different subjects and blur them according to the depth information of different subjects to obtain different blurring effects.

[0116] In an exemplary embodiment, the focus tracking decision module can transmit the human detection results of all subjects to the AI ​​deep blur module, and transmit the sequence number of the determined target subject to the AI ​​deep blur module, so that the AI ​​deep blur module can obtain information such as the position and key points of all subjects in the picture.

[0117] In another exemplary embodiment, the AI ​​deep blurring module can directly obtain the human detection results of all subjects from the output interface of the perception detection module. In this case, the focus tracking decision module only needs to pass the ID of the target subject to the AI ​​deep blurring module, thereby enabling the AI ​​deep blurring module to distinguish between the target subject and non-target subjects.

[0118] In one exemplary embodiment, the focus tracking decision module can directly pass the ID of the target subject determined to the AI ​​deep blurring algorithm.

[0119] In addition, the focus tracking decision module can also transmit the location information of the target subject (or the location information of all subjects) to the camera application, allowing the camera application to display the location of the target subject or other subjects in the recording interface. This location information can be the location of the subject's face outline or body outline.

[0120] S110, the AI ​​deep blurring module uses the coordinate transformation matrix transmitted by the anti-shake module to perform coordinate transformation on the first coordinate frame position of each subject to obtain the corresponding second human body frame position.

[0121] To ensure that the coordinates of the subject detected in the Tiny stream can be correctly mapped to the preview video stream, the coordinate matrix of the subject's position coordinates needs to be transformed based on cropping and warping. That is, the coordinate transformation matrix obtained by the EIS module is used to perform EIS processing on the coordinate boxes (i.e. human bounding boxes) in the Tiny stream to ensure that the transformed human bounding boxes or face bounding boxes correspond to the subject's position in the video stream.

[0122] like Figure 4 As shown, the AI ​​deep blurring module includes a bounding box coordinate transformation module, a human body bounding box transformation module, and an AI deep blurring algorithm. The bounding box coordinate transformation module uses the coordinate transformation matrix (e.g., crop and warp transformation matrix) transmitted by the image stabilization module to transform the coordinate positions of the face or body bounding boxes and key points of each subject, obtaining the transformed human key point coordinates, face bounding box coordinates, or body bounding box coordinates. Further, the bounding box coordinate transformation module transmits the transformed human key point coordinates, face bounding box coordinates, or body bounding box coordinates to the body bounding box transformation module.

[0123] S111, the AI ​​deep blurring module redetermines the position of the third human body frame of each subject based on some human body key points, and the area of ​​the third human body frame is smaller than that of the first human body frame.

[0124] AI depth blurring algorithms estimate the depth of a subject by analyzing the pixels within its bounding box; this depth is also known as the principal depth of field. If the bounding box includes a large area of ​​open space, this open space may represent scenery or objects outside the subject, leading to inaccurate depth values ​​based on the obtained subject.

[0125] For example, such as Figure 9 As shown in (1), the human body is in a state of outstretched arms. The human body detection algorithm detects the area of ​​the human head, torso, and limbs and determines it as the area where the human body is located, that is, the area where the human body bounding box 160 is located. As shown in the figure, the human body bounding box 160 includes many background areas in addition to the human body. When further matching the depth value of the human body based on the pixel points of the human body bounding box, the depth value of the background pixels will be introduced, which will lead to inaccurate estimation of the depth value of the human body.

[0126] To ensure that the human body frame covers more than 90% of the main body area, this embodiment converts the human body frame into a smaller frame that only includes the head and torso. In one embodiment, the position of the human body frame can be redefined using key points corresponding to the torso, such as head key points, shoulder key points, and torso key points.

[0127] like Figure 4 As shown, the human body bounding box conversion module receives the converted human body key point coordinates and human body bounding box coordinates from the bounding box coordinate conversion module, and redetermines the human body bounding box coordinates based on the coordinates of the corresponding human body key points (such as head, shoulders, and torso), and then passes the new human body bounding box coordinates to the AI ​​deep blurring algorithm.

[0128] For example, such as Figure 9 As shown in (1), the human body frame 160 is transformed into a smaller human body frame 161 that includes the head and torso regions. For example, as... Figure 9 As shown in (2), the human body is in a state where both arms are extended forward. In this posture, the human body bounding box detected by the human body detection algorithm is 162. The human body bounding box 163 is obtained by using the key points of the head, shoulders and torso.

[0129] The converted human body bounding box covers more than 90% of the subject area, enabling the AI ​​depth blurring algorithm to obtain a more accurate subject depth value based on the depth information of the pixels in the converted human body bounding box.

[0130] S112, the AI ​​depth blurring module determines the depth value of each subject based on the third human body frame position of each subject using an AI depth estimation algorithm.

[0131] AI-powered depth blurring algorithms estimate the depth of a subject by analyzing the pixels within the bounding box of the human body. This depth information is also known as the primary depth of field.

[0132] S113, the AI ​​depth blurring module uses the depth of the target subject as a reference to blur various non-target subjects, resulting in different blurring intensities for non-target subjects with different depths relative to the target subject.

[0133] The AI-powered deep blurring algorithm determines the depth value of the target subject and the depth value of each non-target subject based on the received human bounding box coordinates and the target subject's ID. Using the target subject's depth value as a benchmark, and based on the relative depth between each non-target subject and the target subject, the algorithm blurs each non-target subject (or portrait background) to keep the target subject sharp, and the blurring intensity varies for non-target subjects with different relative depths to the target subject.

[0134] For example, such as Figure 7 In the scene shown, character C is the target subject, while characters A and B are non-target subjects. Moreover, the distances between characters A, B and C are different, with the distance between characters A and C being greater than the distance between characters B and C. In this scene, the blurring intensity of character A is greater than that of character B.

[0135] For example, such as Figure 8 As shown, character B is the target subject, while characters A and C are non-target subjects (i.e., backgrounds of the character type). If the distance between characters A and C and character B is the same, then the blurring intensity of characters A and C is the same.

[0136] The above content focuses on the blurring process of portrait backgrounds. This method is also applicable to the blurring process of non-portrait backgrounds. For example, it can achieve different blurring intensities for other types of backgrounds located at different depths of field. That is, the blurring process provided in this application is not only applicable to backgrounds of people but also to other types of backgrounds.

[0137] S114, the AI ​​deep bokeh module transmits the bokeh-processed image frames to the camera application.

[0138] S115, the camera application displays a preview video stream consisting of continuously received, blurred image frames in the preview window.

[0139] In this embodiment, S102-S104 and S110-S113 are the processing procedures for the preview video stream, and S105-S109 are the processing procedures for the Tiny stream. The preview video stream processing procedure and the Tiny stream processing procedure can be executed in parallel.

[0140] Furthermore, the embodiments of this application all use the process of blurring the background of a person in a video recording scene as an example for illustration. The image blurring method of this application is also applicable to the photography scene. The blurring process is similar to that of the video scene, and will not be described again here.

[0141] The image blurring method provided in this embodiment, in a multi-person scene, identifies the target subject and non-target subjects, further determines the depth position of each subject, and performs corresponding blurring processing on each non-target subject based on the relative depth distance between each non-target subject and the target subject. This achieves different blurring intensities for subjects with different depths relative to the target subject. In other words, the target subject is clear, and the blurring intensity of non-target subjects with different depths relative to the target subject is also different, making the layering of the blurring effect more in line with the optical blurring effect of a professional camera.

[0142] In some embodiments, the camera application is in autofocus mode. As the recording time progresses, the focus target may change. In order to avoid the focus target jumping back and forth when there are multiple faces or multiple people of similar size, as well as the focus target jumping caused by a face that briefly enters the frame, the image blurring processing method provided in this application also includes a focus stabilization processing process. That is, when the target subject changes, the focus target is not switched immediately, but the focus is switched to the new target subject only after the new focus target is stabilized.

[0143] like Figure 10 As shown, the focus switching stability processing can include the following steps:

[0144] In S201, in auto focus tracking mode, the focus tracking decision module continuously receives the position information of all subjects in the shooting scene.

[0145] In some embodiments, the location information may be the location of the subject's face frame or body frame.

[0146] For example, in a scenario where the subject is facing the camera and the image contains all of the subject's body parts, the subject's face frame and body frame can be detected. Therefore, the subject's position information includes the face frame position and the body frame position.

[0147] In another scenario, the subject is facing away from the camera. In this scenario, only the outline of the subject's body can be detected. Therefore, the subject's position information is the position of the outline of the body.

[0148] In another scenario, the subject is facing the camera and the image does not contain any body parts. In this scenario, only the subject's face frame can be detected. Therefore, the subject's position information is the position of the face frame.

[0149] S202, the focus tracking decision module determines that the target subject of the first image frame is the first person based on the focus tracking strategy.

[0150] For example, based on the focus tracking strategy, it is determined that the current target subject is the first person, for example, in the captured image such as... Figure 6As shown, person A is facing the camera, while people B and C are facing away from the camera. Based on the focus tracking strategy that prioritizes faces over bodies, person A is identified as the current focus target, i.e., the main subject.

[0151] S203, the focus tracking decision module determines that the target subject of the second image frame is the second person based on the focus tracking strategy.

[0152] As the recording progressed, the postures of figures B and C changed, such as from... Figure 6 The posture shown becomes Figure 7 As shown in the pose, characters B and C changed from facing away from the camera to facing the camera directly. Based on the focus tracking strategy that prioritizes larger frame areas, user C was identified as the new focus target, i.e., the new subject.

[0153] S204, the focus tracking decision module continuously tracks the first time period. After confirming that the target subject is stable as the second person, the focus is switched to the second person, and the ID of the second person is transmitted to the AI ​​deep bokeh module.

[0154] The first time period can be set according to actual needs. Additionally, this time period can be measured in image frame count; for example, the first time period could be 8-10 frames.

[0155] When the focus tracking decision module detects a change in the target subject, it does not immediately switch the focus target. Instead, it performs a stability assessment on the new focus target. For example, it continuously tracks the target subject for 8-10 frames. If the target subject in all these frames is a second person, then the new focus target is considered stable. If the target subject changes again during continuous tracking, the tracking resumes from the frame where the target subject changed.

[0156] For example, starting from the decision that the target subject changes from the first person to the second person, after tracking for 4 frames, if the target subject changes from the second person to another person different from the first and second persons (such as the third person), then the tracking will continue from the next frame. If the target subject stabilizes as the third person after tracking for 8-10 frames, then the new target subject is determined to be the third person.

[0157] After the focus tracking process is stabilized, it can avoid the focus target jumping back and forth caused by the face frame or body frame being of similar size, and also avoid the focus target jumping caused by the face that enters the frame briefly (for example, the face of a passerby who suddenly enters the frame during the recording process), thus ensuring stable switching of the focus target.

[0158] S205, the AI ​​depth blur module determines the depth of each subject and determines the blur intensity of other subjects in the image based on the depth distance between other subjects and the second subject.

[0159] The AI ​​depth blurring module receives the ID of the new target subject and the corresponding converted human bounding box position from the focus tracking decision module to determine the depth information of the target subject. It also determines the depth information of other figures based on their converted human bounding box positions. Furthermore, based on the depth relationship between other figures and the target subject, the blurring intensity of the other figures is determined. For example, the greater the depth between them and the target subject, the stronger the blurring intensity; conversely, the smaller the depth between them and the target subject, the weaker the blurring intensity.

[0160] S206, the AI ​​deep blurring module gradually changes the image blurring effect of the first person as the target subject to the image blurring effect of the second person as the target subject during the second time period.

[0161] When the subject changes, the blurring effect also changes accordingly. The AI ​​deep blurring algorithm does not immediately switch the blurring focus, achieving smooth blurring processing. The AI ​​deep blurring algorithm simultaneously tracks the positions of multiple subjects. When it receives a detection result indicating a change in the subject, it completes the transition of the blurring effect within a certain period of time (e.g., 10 frames). For example, the original subject gradually becomes blurred, the new subject gradually becomes clear, and other subjects or scenes are re-blurred based on the depth of the new subject. This avoids abrupt changes in the recorded image caused by immediate changes in the blurring focus.

[0162] In some embodiments, the transition process of the blurring effect conforms to the Bézier curve model, such as... Figure 11 The diagram shows the Bézier curve for the blurring effect transition. From T1 to T2, the blurring intensity of the subject changes from the initial blurring intensity to the target blurring intensity. Specifically, in the initial stage, i.e., the beginning of T1, the curve's rate of change is small, meaning the blurring effect changes slowly. In the middle stage, the curve's rate of change is large, meaning the blurring effect changes quickly. In the final stage, the curve's rate of change decreases again, meaning the blurring effect changes slowly.

[0163] The above process describes the smooth transition of the bokeh effect when the focus target changes in automatic focus tracking mode. This process also applies to manual focus tracking mode. For example, after the focus tracking decision module determines a new target subject manually selected by the user, it transmits the sequence number of the new target subject to the AI ​​deep bokeh module. In the second time period, the AI ​​deep bokeh module gradually changes the image bokeh effect of the original target to the image bokeh effect corresponding to the new target subject.

[0164] The focus target switching process provided in this embodiment does not switch focus targets when a change in the focus target is detected. Instead, it determines that the target is stable before switching the focus target to the new subject. This avoids the focus target jumping back and forth caused by similarly sized face or body frames, and also avoids the focus target jumping caused by faces that briefly enter the frame (e.g., the face of a passerby suddenly entering the frame during recording), ensuring stable focus target switching. Furthermore, this embodiment also employs a smoothing process in the blurring process after focus target switching. That is, the blurring effect transition after focus target switching is completed over a period of time. For example, the original subject gradually becomes blurred, the new target gradually becomes clear, and other subjects or objects are re-blurred based on the depth of the new target. This avoids abrupt changes in the recorded image caused by immediate changes in the focus.

[0165] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments. The aforementioned storage medium includes various media capable of storing program code, such as flash memory, portable hard disk, read-only memory, random access memory, magnetic disk, or optical disk.

[0166] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An image blurring processing method characterized by comprising: The method is applied to an electronic device equipped with a camera, the electronic device including a perception detection module, a focus tracking decision module, and an AI deep bokeh module, and includes: The first interface is displayed, which includes a preview window and a blur button; In response to the recording operation, video is captured by the camera to obtain a preview video stream and a low-resolution video stream, and a first image in the preview video stream is displayed in the preview window. The first image includes a first portrait, a second portrait, and a third portrait. The low-resolution video stream is obtained by downsampling the preview video stream. The perception detection module performs human perception detection on the second image in the low-resolution video stream to obtain a first human detection result. The first human detection result includes first position information corresponding to the first human image, the second human image, and the third human image, respectively. The tracking focus decision module determines the first portrait as the tracking focus target based on the automatic tracking focus strategy, and transmits the ID of the first portrait and the first human body detection result to the AI ​​deep blurring module. The AI ​​depth blurring module determines a third human body frame corresponding to the first portrait, the second portrait, and the third portrait respectively, and includes only the head and torso of the portrait, based on the first position information. It also determines the depth of each portrait based on the depth of the pixels within the third human body frame corresponding to each portrait in the first image, determines the first portrait as the focus target based on the ID of the first portrait, determines the first relative depth of the second portrait relative to the first portrait based on the depth of the first portrait and the depth of the second portrait, and determines the second relative depth of the third portrait relative to the first portrait based on the depth of the first portrait and the depth of the third portrait. In response to the operation of the blur button, a first blur effect image after blurring the first image through the AI ​​depth blur module is displayed in the preview window. The first blur effect image includes a clear first portrait, a second portrait with a first blur intensity, and a third portrait with a second blur intensity, wherein the first blur intensity is positively correlated with the first relative depth, and the second blur intensity is positively correlated with the second relative depth. The first location information includes the location information of the first human body key point and the location information corresponding to the first human body frame containing all body parts in the portrait; the automatic focus tracking strategy includes: portraits in the image containing face frames have higher priority than portraits without faces; when all portraits in the image contain face frames, the priority is positively correlated with the size of the face frames; when all portraits in the image contain human body frames, the priority is positively correlated with the size of the human body frames.

2. The method according to claim 1, characterized in that, The method further includes: When the focus tracking decision module determines that the focus target is switched to the second portrait based on the second human detection result of the third image in the low-resolution video stream, it transmits the ID of the second portrait and the second human detection result to the AI ​​depth blurring module. The AI ​​depth blurring module switches the first blurring effect image to the second blurring effect image within a first preset time period. The second blurring effect image includes a clear second portrait, a first portrait with a third blur intensity, and a third portrait with a fourth blur intensity. The third blur intensity is positively correlated with the third relative depth, and the fourth blur intensity is positively correlated with the fourth relative depth. The third relative depth is the relative depth between the first portrait and the second portrait, and the fourth relative depth is the relative depth between the third portrait and the second portrait.

3. The method according to claim 2, characterized in that, The AI ​​deep blurring module switches the first blurred image to a second blurred image within a first preset time period, including: After receiving the information that the focus target has switched to the second portrait, the AI ​​depth blurring module determines the depth of each object in the current image frame. The objects include the portrait objects in the image and other shooting objects. The target blur intensity corresponding to the relative depth of other objects in the image is determined based on the depth of the second portrait, wherein the other objects include the first portrait and the third portrait; Within the first preset time period, the first portrait is controlled to gradually change from clear to the third blur intensity, the third portrait is controlled to gradually change from the second blur intensity to the fourth blur intensity, and the second portrait is controlled to gradually change from the first blur intensity to clear.

4. The method according to claim 3, characterized in that, Within the first preset time period, controlling the first portrait to gradually change from clear to the third blur intensity, the third portrait to gradually change from the second blur intensity to the fourth blur intensity, and controlling the second portrait to gradually change from the first blur intensity to clear, includes: Using the start time of the first preset duration as the start time of the Bézier curve and the end time of the first preset duration as the end time of the Bézier curve, with clarity as the starting point and the third blur intensity as the ending point, a first smooth Bézier curve corresponding to the first portrait is generated; at any time within the first preset duration, the target blur intensity corresponding to the first smooth Bézier curve is found, and the first portrait is blurred to the target blur intensity corresponding to the first portrait; Using the start time of the first preset duration as the start time of the Bézier curve, the end time of the first preset duration as the end time of the Bézier curve, and the first blur intensity as the start point and the clear point as the end point, a second smooth Bézier curve corresponding to the second portrait is generated; at any time within the first preset duration, the target blur intensity corresponding to the second smooth Bézier curve is found, and the second portrait is blurred to the target blur intensity corresponding to the second portrait; Using the start time of the first preset duration as the start time of the Bézier curve, the end time of the first preset duration as the end time of the Bézier curve, the second blur intensity as the start point, and the fourth blur intensity as the end point, a third smooth Bézier curve corresponding to the third portrait is generated; at any time within the first preset duration, the target blur intensity corresponding to the third smooth Bézier curve is found, and the third portrait is blurred to the target blur intensity corresponding to the third portrait.

5. The method according to any one of claims 1-4, characterized in that, The first location information also includes the location of the first face frame; the electronic device also includes an image stabilization module, and the method further includes: The image stabilization module performs image stabilization processing on the first image and transmits the image stabilization processed first image and coordinate transformation matrix to the AI ​​deep blurring module. The AI ​​deep blurring module converts the coordinates of each position in the first position information into second position information based on the coordinate transformation matrix. The second position information includes the position information of the second human body frame, the position information of the second face frame, and the position information of the second human body key points.

6. The method according to claim 5, characterized in that, The process of obtaining the position information of the third-person body frame corresponding to each portrait includes: For any human figure in the first image, the position information of the third human figure is determined based on the position information of the second human figure frame corresponding to the human figure and the position information of the second human key points of the preset part.

7. The method according to claim 2, characterized in that, The focus tracking decision module determines that the focus target is switched to the second human image based on the second human detection result of the third image in the low-resolution video stream, including: The focus tracking decision module determines that the focus tracking target is the second human image based on the second human detection result corresponding to the third image, and the third image was captured later than the second image; If the focus target remains the second portrait within the second preset time period, it is determined that the focus target has switched from the first portrait to the second portrait.

8. A chip system, characterized in that, include: At least one processor and an interface, the interface being used to receive code instructions and transmit them to the at least one processor; The at least one processor executes the code instructions to implement the image blurring processing method according to any one of claims 1-7.

9. An electronic device, characterized in that, The electronic device includes: one or more processors, a memory, and a touch screen; the memory is used to store program code; the processor is used to run the program code, causing the electronic device to implement the image blurring processing method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, It stores instructions that, when executed on an electronic device, cause the electronic device to perform the image blurring process as described in any one of claims 1 to 7.

Images