Focus tracking method, device, terminal device and computer-readable storage medium

Abstract

The present application is applicable to the field of computer application technology and provides a focus tracking method, apparatus, terminal device, and computer-readable storage medium. The method includes: identifying and processing each object in the camera preview interface to determine the object type of each object; determining the focus frame type and the first focus tracking priority of each object based on the object type and preset priority strategy of each object; generating and displaying the focus frame corresponding to each object based on the focus frame type and coordinates of each object; and determining any object with the highest first focus tracking priority as the target focus tracking object based on the first focus tracking priority of each object, and performing focus tracking on the target focus tracking object. Thus, by selecting the object that the user is most concerned about in the current shooting scene during the preview process for focus tracking, and different focus frames for different types of objects, the shooting effect of the object that the user is most concerned about is improved, as well as the flexibility and convenience of focus tracking.

Description

Technical Field

[0001] The present application belongs to the field of computer application technology, and in particular relates to a focus tracking method, apparatus, terminal device and computer-readable storage medium. Background Art

[0002] With the widespread popularity and rapid development of terminal devices such as smartphones, their functions are becoming increasingly sophisticated. For example, users can take photos with their smartphones. In many shooting scenarios, in order to obtain a clear image, it is necessary to focus on the subject to meet users' higher shooting needs.

[0003] In the related art, the center area of the camera preview interface is usually used as the focus area for focusing, or the area manually selected by the user is focused. However, these focusing methods cannot adaptively adjust the focus object according to the actual shooting scene, resulting in not only poor focusing effect, but also poor flexibility and ease of use, affecting the user's shooting experience. Summary of the Invention

[0004] The embodiments of the present application provide a focus tracking method, apparatus, terminal device, and computer-readable storage medium, which can solve the problem that focusing methods such as focusing on the center area of the camera preview interface as the focus area, or focusing on an area manually selected by the user, cannot adaptively adjust the focus object according to the actual shooting scene, resulting in not only poor focusing effect, but also poor flexibility and ease of use, affecting the user's shooting experience.

[0005] In a first aspect, an embodiment of the present application provides a focus tracking method, including: displaying a camera preview interface in response to a user's first operation; performing identification processing on each object displayed in the camera preview interface to determine the object type of each object; determining the focus tracking frame type of each object based on the object type of each object; determining the first focus tracking priority of each object based on the object type of each object and a preset priority strategy, wherein the preset priority strategy is set based on the user's attention level to each object type; generating a focus tracking frame corresponding to each object based on the focus tracking frame type of each object, and displaying the focus tracking frame corresponding to each object in the camera preview interface based on the coordinates of each object; determining any object with the highest first focus tracking priority as the target focus tracking object based on the first focus tracking priority of each object, and performing focus tracking processing on the target focus tracking object.

[0006] In a possible implementation of the first aspect, the object types include the following types: portraits ranked in the top N in the image gallery, portraits ranked in the top N in the non-image gallery, and non-face objects of a preset type, where N is a positive integer greater than or equal to 1;

[0007] The focus frame type for the portraits ranked in the top N in the gallery is the first or second focus frame type, while the focus frame type for portraits not ranked in the top N in the gallery and for non-face objects of a preset type is the third focus frame type. The first, second, and third focus frame types are all different.

[0008] The above-mentioned preset priority strategy is: the first focus tracking priority of the portraits ranked in the top N in the gallery is higher than the first focus tracking priority of the portraits ranked in the top N outside the gallery, and the first focus tracking priority of the portraits ranked in the top N outside the gallery is higher than the first focus priority of the preset type of non-face objects.

[0009] Optionally, in another possible implementation of the first aspect, generating a focus frame corresponding to each object based on the focus frame type of each object, and displaying the focus frame corresponding to each object in the camera preview interface based on the coordinates of each object, includes:

[0010] When the objects include a first object whose object type is a portrait ranked in the top N in the gallery, generating a first tracking focus frame corresponding to the first object according to the first tracking focus frame type, and displaying the first tracking focus frame corresponding to the first object in the camera preview interface according to the coordinates of the first object;

[0011] When the objects include a first object whose object type is a portrait ranked in the top N in the gallery and at least one second object, a first tracking frame corresponding to the first object is generated according to the first tracking frame type, and a second tracking frame corresponding to each second object is generated according to the second tracking frame type, and the first tracking frame corresponding to the first object and the second tracking frame corresponding to each second object are displayed in the camera preview interface according to the coordinates of the first object and each second object, wherein the size of the first object is larger than the size of each second object.

[0012] Optionally, in another possible implementation of the first aspect, after generating the first tracking focus frame corresponding to the first object according to the first tracking focus frame type, generating the second tracking focus frames corresponding to each second object according to the second tracking focus frame type, and displaying the first tracking focus frame corresponding to the first object and the second tracking frames corresponding to each second object on the camera preview interface based on the coordinates of the first object and each second object, the method further includes:

[0013] If it is determined that the size of any second object is larger than the size of the first object, then any second object is determined as a new first object, and the first object is determined as a new second object, and the focus frame corresponding to the new first object is changed from the second focus frame to the first focus frame, and the focus frame corresponding to the new second object is changed from the first focus frame to the second focus frame.

[0014] Optionally, in another possible implementation of the first aspect, after generating a focus frame corresponding to each object based on the focus frame type of each object and displaying the focus frame corresponding to each object on the camera preview interface based on the coordinates of each object, the method further includes:

[0015] When the object type is not a portrait ranked in the top N in the image library or a non-face object of a preset type, determining the display duration of the focus tracking frame corresponding to the object;

[0016] When the display time of the focus tracking frame corresponding to the object reaches a time threshold, the focus tracking frame corresponding to the object stops being displayed.

[0017] Optionally, in another possible implementation of the first aspect, determining, based on the first focus tracking priorities of the respective objects, any object with the highest first focus tracking priority as the target focus tracking object includes:

[0018] Determine whether the objects include the first object whose object type is a portrait ranked in the top N in the image library;

[0019] If the objects include at least one first object, the first object is determined as the object with the highest first priority, and any first object is determined as the target focus tracking object;

[0020] If the first object is not included in the objects, determining whether the objects include a second object whose object type is not a portrait ranked in the top N in the image library;

[0021] If the objects include at least one second object, the second object is determined as the object with the highest first priority, and any second object is determined as the target tracking focus object;

[0022] If the second object is not included in the objects, determining whether the objects include a third object whose object type is a non-face object of a preset type;

[0023] If the objects include at least one third object, the third object is determined as the object with the highest first priority, and any third object is determined as the target tracking focus object;

[0024] If the objects do not include the third object, center focus processing is performed in the preview interface.

[0025] Optionally, in another possible implementation of the first aspect, after determining, based on the first focus tracking priorities of the respective objects, any object with the highest first focus tracking priority as the target focus tracking object and performing focus tracking on the target focus tracking object, the method further includes:

[0026] In response to a user's click operation on any object, determining the any object as a designated focus tracking object;

[0027] If the specified focus object is the target focus object, the focus process will continue to be performed on the specified focus object until the user clicks on any object;

[0028] If the specified focus object is not the target focus object, the focus processing of the target focus object is suspended, and the focus processing of the specified focus object is continued until the user clicks on any object.

[0029] Optionally, in another possible implementation of the first aspect, after determining any object as a designated focus target in response to a user's click operation on any object, the method further includes:

[0030] If the designated focus object is the target focus object and its object type is one of the top N portraits in the gallery, the focus frame type of the designated focus object remains unchanged, and the focus frame of the designated focus object continues to be displayed, while the focus frames of all objects other than the designated focus object are temporarily displayed.

[0031] If the designated focus object is not the target focus object, and the object type of the designated focus object is a portrait ranked in the top N in the gallery, then the focus frame type of the designated focus object is changed to the first focus frame type, and a first focus frame corresponding to the designated focus object is generated and continuously displayed based on the first focus frame type, while the focus frames of all objects other than the designated focus object are temporarily displayed;

[0032] If the object type of the specified focus object is not a portrait ranked in the top N in the gallery, the focus frame type of the specified focus object will be changed to the fourth focus frame type, and the fourth focus frame corresponding to the specified focus object will be generated and continuously displayed based on the fourth focus frame type, and the focus frames of all objects except the specified focus object will be temporarily displayed.

[0033] Optionally, in another possible implementation of the first aspect, after determining any object as a designated focus target in response to a user's click operation on any object, the method further includes:

[0034] In response to the user's click operation on the specified focus object, the focus processing of the specified focus object is paused, and according to the first focus priority of each object, any object with the highest first focus priority is determined as the target focus object, and the focus processing is performed on the target focus object.

[0035] Optionally, in another possible implementation of the first aspect, the objects include multiple objects with the highest first focus tracking priority; accordingly, determining any object with the highest first focus tracking priority as the target focus tracking object based on the first focus tracking priority of each object includes:

[0036] Determine a second focus tracking priority for each object having the highest first focus tracking priority;

[0037] The object with the second highest tracking priority is determined as the target tracking object.

[0038] Optionally, in another possible implementation of the first aspect, determining the second focus priority of each object having the highest first focus priority includes:

[0039] Obtain the preset priority of each object with the highest first focus tracking priority;

[0040] Determine the preset priority of each object as the second focus tracking priority of each object;

[0041] or,

[0042] Determine the size of each object with the highest priority for first focus tracking;

[0043] Determine the second focus tracking priority of each object based on the size of each object.

[0044] In a second aspect, an embodiment of the present application provides a focus tracking device, comprising: a first display module, for displaying a camera preview interface in response to a first operation of a user; a first determination module, for identifying and processing each object displayed in the camera preview interface to determine the object type of each object; a second determination module, for determining the focus tracking frame type of each object based on the object type of each object; a third determination module, for determining the first focus tracking priority of each object based on the object type of each object and a preset priority strategy, wherein the preset priority strategy is set according to the user's attention level to each object type; the second display module, for generating a focus tracking frame corresponding to each object according to the focus tracking frame type of each object, and displaying the focus tracking frame corresponding to each object in the camera preview interface according to the coordinates of each object; the first focus tracking module, for determining any object with the highest first focus tracking priority as the target focus tracking object according to the first focus tracking priority of each object, and performing focus tracking processing on the target focus tracking object.

[0045] In a possible implementation of the second aspect, the object types include the following types: portraits ranked in the top N in the image gallery, portraits ranked in the top N in the non-image gallery, and non-face objects of a preset type, where N is a positive integer greater than or equal to 1;

[0046] The focus frame type for the portraits ranked in the top N in the gallery is the first or second focus frame type, while the focus frame type for portraits not ranked in the top N in the gallery and for non-face objects of a preset type is the third focus frame type. The first, second, and third focus frame types are all different.

[0047] The preset priority strategy is: the first focus tracking priority of the portraits ranked in the top N in the gallery is higher than the first focus tracking priority of the portraits ranked in the top N outside the gallery, and the first focus tracking priority of the portraits ranked in the top N outside the gallery is higher than the first focus priority of the preset type of non-face objects.

[0048] Optionally, in another possible implementation of the second aspect, the second display module includes:

[0049] A first display unit is configured to generate a first tracking focus frame corresponding to the first object according to the first tracking focus frame type when the objects include a first object whose object type is a portrait ranked in the top N in the gallery, and to display the first tracking focus frame corresponding to the first object in the camera preview interface according to the coordinates of the first object;

[0050] The second display unit is used to generate a first tracking frame corresponding to the first object according to the first tracking frame type, and generate a second tracking frame corresponding to each second object according to the second tracking frame type, when the objects include a first object whose object type is a portrait ranked in the top N in the gallery and at least one second object, and display the first tracking frame corresponding to the first object and the second tracking frame corresponding to each second object in the camera preview interface according to the coordinates of the first object and each second object, wherein the size of the first object is larger than the size of each second object.

[0051] Optionally, in another possible implementation of the second aspect, the second display module further includes:

[0052] The first changing unit is used to determine any second object as a new first object if it is determined that the size of any second object is larger than the size of the first object, and to determine the first object as a new second object, and to change the focusing frame corresponding to the new first object from the second focusing frame to the first focusing frame, and to change the focusing frame corresponding to the new second object from the first focusing frame to the second focusing frame.

[0053] Optionally, in another possible implementation of the second aspect, the apparatus further includes:

[0054] A fourth determining module is configured to determine a display duration of a focus tracking frame corresponding to an object when the object type is a non-human face object that is ranked in the top N in the image gallery or a non-human face object of a preset type;

[0055] The first display stopping module is configured to stop displaying the focus tracking frame corresponding to the object when the display duration of the focus tracking frame corresponding to the object reaches a duration threshold.

[0056] Optionally, in another possible implementation of the second aspect, the first focus tracking module includes:

[0057] A first judging unit is configured to judge whether the objects include a first object whose object type is a portrait ranked in the top N in the image library;

[0058] a first determining unit configured to, if the objects include at least one first object, determine the first object as the object with the highest first priority, and determine any first object as a target focus tracking object;

[0059] a second determining unit configured to determine whether the objects include a second object whose object type is not a portrait ranked in the top N in the image library if the objects do not include the first object;

[0060] a second determining unit, configured to, if the objects include at least one second object, determine the second object as the object with the highest first priority, and determine any second object as a target focus tracking object;

[0061] a third determining unit configured to determine whether the objects include a third object of a preset non-face type if the objects do not include the second object;

[0062] a third determining unit, configured to, if the objects include at least one third object, determine the third object as the object with the highest first priority, and determine any third object as a target focus tracking object;

[0063] The first focusing unit is configured to perform center focusing processing in the preview interface if the objects do not include the third object.

[0064] Optionally, in another possible implementation of the second aspect, the apparatus further includes:

[0065] a fifth determining module, configured to determine any object as a designated focus tracking object in response to a user's click operation on any object;

[0066] The second focus tracking module is configured to, if the designated focus tracking object is the target focus tracking object, continue to focus on the designated focus tracking object until a user clicks on any object;

[0067] The third focusing module is used to suspend focusing on the target focusing object if the designated focusing object is not the target focusing object, and continue focusing on the designated focusing object until a user clicks on any object.

[0068] Optionally, in another possible implementation of the second aspect, the apparatus further includes:

[0069] A third display module is configured to maintain the focus frame type of the designated focus object unchanged and continuously display the focus frame of the designated focus object, and to pause displaying the focus frames of all objects other than the designated focus object, if the designated focus object is the target focus object and the object type of the designated focus object is a portrait ranked in the top N in the gallery;

[0070] a fourth display module configured to, if the designated focus object is not the target focus object and the object type of the designated focus object is a portrait ranked in the top N in the gallery, change the focus frame type of the designated focus object to a first focus frame type, generate and continuously display a first focus frame corresponding to the designated focus object based on the first focus frame type, and pause display of focus frames for objects other than the designated focus object;

[0071] The fifth display module is used to change the focus frame type of the specified focus object to the fourth focus frame type if the object type of the specified focus object is not a portrait ranked in the top N in the gallery, and to generate and continuously display the fourth focus frame corresponding to the specified focus object according to the fourth focus frame type, and to pause displaying the focus frames of all objects except the specified focus object.

[0072] Optionally, in another possible implementation of the second aspect, the apparatus further includes:

[0073] The fourth focus tracking module is used to pause the focus tracking process of the specified focus tracking object in response to the user's click operation on the specified focus tracking object, and determine any object with the highest first focus tracking priority as the target focus tracking object according to the first focus tracking priority of each object, and perform focus tracking on the target focus tracking object.

[0074] Optionally, in another possible implementation of the second aspect, the objects include multiple objects with the highest first focus tracking priority; accordingly, the first focus tracking module includes:

[0075] a fourth determining unit, configured to determine a second focus tracking priority for each object having the highest first focus tracking priority;

[0076] The fifth determining unit is configured to determine the object with the highest second focus tracking priority as the target focus tracking object.

[0077] Optionally, in another possible implementation of the second aspect, the fourth determining unit is specifically configured to:

[0078] Obtain the preset priority of each object with the highest first focus tracking priority;

[0079] Determine the preset priority of each object as the second focus tracking priority of each object;

[0080] or,

[0081] Determine the size of each object with the highest priority for first focus tracking;

[0082] Determine the second focus tracking priority of each object based on the size of each object.

[0083] In a third aspect, an embodiment of the present application provides a terminal device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the aforementioned focus tracking method when executing the computer program.

[0084] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium on which a computer program is stored, wherein the computer program implements the focus tracking method as described above when executed by a processor.

[0085] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when executed on a terminal device, enables the terminal device to execute the focus tracking method as described above.

[0086] Compared with the prior art, the beneficial effects of the embodiments of the present application are: during the preview process, the object that the user is most concerned about in the current shooting scene is selected according to the object type of each object in the camera preview interface, and is used as the highest priority focus tracking object, so as to focus on the object that the user is currently most concerned about, and different focus tracking frames are used for displaying different types of objects. This not only can adaptively adjust the focus tracking object according to the real-time shooting scene, thereby improving the shooting effect of the object that the user is most concerned about, but also does not require manual operation by the user, thereby improving the flexibility and ease of use of focus tracking, and improving the user's shooting experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0087] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0088] Figure 1 1 is a flow chart of a focus tracking method provided in one embodiment of the present application;

[0089] Figure 2 This is a schematic diagram of a main interface of a terminal device provided in one embodiment of the present application;

[0090] Figure 3 This is a schematic diagram of a camera preview interface of a terminal device provided by an embodiment of the present application;

[0091] Figure 4 This is a schematic diagram of a camera preview interface of another terminal device provided in an embodiment of the present application;

[0092] Figure 5 This is a schematic diagram of a recording interface of a terminal device provided in one embodiment of the present application;

[0093] Figures 6-10 This is a schematic diagram of a camera preview interface of a group of terminal devices provided by an embodiment of the present application;

[0094] Figure 11 is a flowchart of a focus tracking method provided in another embodiment of the present application;

[0095] Figure 12-14 is a schematic diagram of a camera preview interface of another group of terminal devices provided by an embodiment of the present application;

[0096] Figure 15 This is a software architecture diagram of the focus tracking method provided in one embodiment of the present application;

[0097] Figure 16 is a timing diagram of a focus tracking method provided in one embodiment of the present application;

[0098] Figure 17 Schematic diagram of the structure of the focus tracking device provided in an embodiment of the present application;

[0099] Figure 18 It is a structural diagram of the terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0100] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail.

[0101] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or collections thereof.

[0102] It will also be understood that the term "and / or" used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0103] As used in this specification and the appended claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0104] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0105] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0106] The following describes in detail the focus tracking method, apparatus, terminal device, storage medium and computer program provided in the present application with reference to the accompanying drawings.

[0107] Figure 1 A flow chart of a focus tracking method provided in an embodiment of the present application is shown.

[0108] Step 101: In response to a first operation by a user, a camera preview interface is displayed.

[0109] It should be noted that the focus tracking method of the embodiments of the present application can be performed by the focus tracking device of the embodiments of the present application. The focus tracking device of the embodiments of the present application can be configured in any terminal device to perform the focus tracking method of the embodiments of the present application. For example, the focus tracking device of the embodiments of the present application can be configured in a terminal device such as a mobile phone, a computer, or a wearable device to enable the user to automatically track focus during shooting.

[0110] The first operation may be any operation input by the user on the terminal device that can trigger the opening of the camera preview interface. For example, the first operation may be triggering the camera application icon on the main interface of the terminal device, triggering the shutter control on the camera operation interface, triggering the end recording control during camera recording, triggering the pause recording control during camera recording, etc.

[0111] As a possible implementation, Figure 2 As shown, the main interface 200 of the terminal device displays a page with application icons, which may include multiple application icons (for example, a weather application icon, a calendar application icon, an album application icon, a note application icon, an email application icon, an application store application icon, a settings application icon, etc.). A page indicator may also be displayed below the above-mentioned multiple application icons to indicate the positional relationship between the currently displayed page and other pages. There are multiple application icons (for example, a camera application icon 201, a browser application icon, a message application icon, and a dial application icon) below the page indicator. These application icons remain displayed when the page is switched. It can be understood that the camera application icon 201 is an icon for a camera application (i.e., a camera application). The camera application icon 201 can be used to trigger the launch of a camera application.

[0112] The terminal device can detect the trigger operation acting on the camera application icon 201, and in response to the trigger operation, the electronic device can display the following Figure 3 The camera preview interface 300 is shown. The camera preview interface 300 is a shooting interface of the default shooting mode of the camera application, and the user can preview the image and complete the shooting on this interface.

[0113] like Figure 3 As shown, the camera preview interface 300 may include a preview window 301, a camera mode option 302, an album shortcut control 303, a shutter control 304 and a camera flip control.

[0114] The preview window 301 can be used to display a preview image. The preview image displayed in the preview window 301 is an original image captured by the camera of the terminal device based on the viewing range.

[0115] One or more shooting mode options may be displayed in the camera mode options 302. These one or more shooting mode options may include: a night scene mode option, a smart portrait mode option, a photo mode option, a video mode option 3021, and more options. It is understood that the camera mode options 302 may also include more or fewer shooting mode options.

[0116] The album shortcut control 303 can be used to open the album application. After the user triggers the electronic device to open the album application through the album shortcut control 303, the user can view the images and videos taken. In addition, the album shortcut control 303 can also display thumbnails of the images or videos taken.

[0117] As a possible implementation, the shutter control 304 can be used to trigger the camera to capture an image and complete the photo-taking operation. The terminal device can automatically return to and display the camera preview interface 300 after obtaining the user's triggering operation on the shutter control 304 and completing the photo-taking.

[0118] As a possible implementation, triggering the video mode option 3021 can enter the video mode and display the following: Figure 4 The camera preview interface 400 (video preview interface) shown in FIG. 4 is substantially the same as the controls included in the camera preview interface 300. The difference is that the camera preview interface 400 may further include a video start control 401. The video start control 402 is used to trigger the electronic device to start video recording. The terminal device may display the following after obtaining the user's triggering operation on the video start control 401: Figure 5 The recording interface 500 shown in FIG. The recording interface 500 includes an end recording control 501 and a pause recording control 502. The end recording control can be used to end recording, and the pause recording control 502 can be used to pause recording. The terminal device can end recording and display the following after obtaining the user's trigger operation on the end recording control 501. Figure 4 The camera preview interface 400 shown can also stop recording and display the camera preview interface after obtaining the user's trigger operation on the pause recording control 502.

[0119] It will be understood that the triggering operations mentioned in the embodiments of the present application may include but are not limited to touch (for example, click, etc.), voice control, gestures and other operations, and the present application does not impose any restrictions on this.

[0120] It should be noted that the types of first operations and the methods for triggering the display of the camera preview interface listed above are merely exemplary and should not be considered as limitations of this application. In actual use, the type of first operation that triggers the display of the camera preview interface may be related to the functions of the terminal device and the camera application, and this embodiment of the application does not limit this.

[0121] Step 102 : performing recognition processing on each object displayed in the camera preview interface to determine the object type of each object.

[0122] The object type may include portraits ranked in the top N in the gallery, portraits ranked in the top N in non-gallery, and non-face objects of preset types, where N is a positive integer greater than or equal to 1.

[0123] It should be noted that the gallery application in the terminal device can perform clustering processing based on the images that the user has collected and stored in the gallery database to classify the images that the user has collected according to the objects contained in each image, thereby generating personal albums corresponding to each collected object, and sorting each personal album according to parameters such as the number of images contained in the personal album and whether the personal album has been named. Then, the portraits corresponding to the personal albums ranked in the top N in the gallery can be determined as the portraits in the top N in the gallery, and the facial features of the portraits in the top N in the gallery can be sent to the camera application, so that the camera application can determine whether the object in the preview interface is the portrait in the top N in the gallery based on the sent facial features during the preview stage. In the embodiment of the present application, the portraits ranked in the top N in the gallery are referred to as the protagonist portraits, that is, the portraits ranked in the top N in the gallery mentioned in this application are the same type of portraits as the protagonist portraits. In actual use, the specific value of N can be determined according to actual needs and specific application scenarios, and the embodiment of the present application does not limit this. For example, N can be 5.

[0124] It is understandable that since the portraits ranked higher in the gallery are usually portraits that appear more frequently in the user's gallery, or portraits named by the user, the portraits ranked in the top N in the gallery can usually be considered as objects of high user attention.

[0125] It should be noted that the portraits ranked in the top N outside the image library can be portraits included in the image library but not ranked in the top N, or portraits not included in the image library. In the embodiments of this application, the portraits ranked in the top N outside the image library are referred to as ordinary portraits, that is, the portraits ranked in the top N outside the image library mentioned in this application and the ordinary portraits are the same type of portraits.

[0126] It should be noted that the preset types of non-face objects can be any type of physical object other than facial images. For example, the preset types of non-face objects can include pets such as cats and dogs, vehicles, etc., and this embodiment of the application does not limit this. In actual use, the types of preset non-face objects can be set according to the actual needs of the user, and this embodiment of the application does not limit this. For example, the preset types of non-face objects can be cats and dogs.

[0127] In an embodiment of the present application, after opening the camera application and displaying the camera preview interface, the camera can capture images of the framing range at a certain frame rate and store the captured image frames in a frame buffer. Therefore, after displaying the camera preview interface, the preview frames that the camera has captured can be obtained frame by frame from the frame buffer, and the objects in each preview frame can be identified and processed using the all-object tracking algorithm and the facial feature algorithm in the semantic engine to determine the object type of each object in each preview frame.

[0128] Step 103 : Determine the focus frame type of each object according to the object type of each object.

[0129] In an embodiment of the present application, a focus frame for each object can be displayed in the camera preview interface, making the process of focusing on each object in the camera preview interface visible to the user. Furthermore, different types of focus frames can be used for focusing on different types of objects, so that the user can understand the type of each object based on the focus frame corresponding to each object, and further, whether each object in the camera preview interface is the object of their interest, so that the user can adjust the shooting strategy according to their own shooting needs.

[0130] As a possible implementation method, the correspondence between object type and focus frame type can be pre-set. After determining the object type of each object in the camera preview interface, the focus frame type corresponding to the object type of each object can be determined as the focus frame type of each object according to the preset correspondence between object type and focus frame type.

[0131] As an example, when the object types include portraits ranked in the top N in the gallery (portraits of the protagonist), portraits ranked in the top N outside the gallery (ordinary portraits), and non-face objects of a preset type, the focus frame type of the portraits ranked in the top N in the gallery can be the first focus frame type or the second focus frame type, and the focus frame type of the portraits ranked in the top N outside the gallery and the focus frame type of non-face objects of the preset type can be the third focus frame type, wherein the first focus frame type, the second focus frame type, and the third focus frame type are all different.

[0132] For example, the focus frame type for the protagonist's portrait can be a double-layer yellow frame or a double-layer white frame, that is, the first focus frame type can be a double-layer yellow frame, and the second focus frame type can be a double-layer white frame; the focus frame type for ordinary portraits and preset types of non-face objects can be a single-layer white frame, that is, the third focus frame type can be a single-layer white frame.

[0133] Step 104 : Determine the first focus tracking priority of each object according to the object type and the preset priority strategy of each object.

[0134] The preset priority strategy may be set according to the user's attention level to each object type.

[0135] In an embodiment of the present application, when multiple objects appear in the camera preview interface, such as in a group photo scenario, the user's attention to each object in the camera preview interface may be different. The user usually hopes that the shooting effect of the object to which the user pays more attention will be better. Therefore, in the embodiment of the application, a priority strategy can be pre-set according to the user's attention to each object type. That is, in the preset priority strategy, the higher the object type to which the user pays more attention, the higher the corresponding first focus tracking priority, so that the object to which the user currently pays the most attention can be focused during the shooting process, thereby achieving the purpose of taking the clearest picture of the object to which the user pays the most attention.

[0136] As an example, when the object types include portraits ranked in the top N in the gallery (protagonist portraits), portraits ranked in the top N outside the gallery (ordinary portraits), and non-face objects of a preset type, the preset priority strategy can be: the first focus tracking priority of portraits ranked in the top N in the gallery (protagonist portraits) is higher than the first focus tracking priority of portraits ranked in the top N outside the gallery (ordinary portraits), and the first focus tracking priority of portraits ranked in the top N outside the gallery (ordinary portraits) is higher than the first focus priority of non-face objects of a preset type.

[0137] It should be noted that, since the protagonist portrait is a portrait that appears frequently in the gallery application or has been named by the user, it can be determined that the protagonist portrait is the portrait that the user pays the most attention to. Therefore, the first focus priority of the protagonist portrait can be set to the highest, so that when the protagonist portrait exists in the camera preview interface, the protagonist portrait is given priority to focus. In addition to the protagonist portrait, when taking pictures, compared with other types of physical objects, users usually pay more attention to facial objects. Therefore, the first focus priority of ordinary portraits can be set to be lower than the protagonist portrait and higher than the preset type of non-face objects, so that when the protagonist portrait is not included in the camera preview interface, ordinary faces are given priority to focus. Since the preset type of non-face objects is an object type that the user pays more attention to other than facial objects according to the user's usage needs, the first focus priority of the preset type of non-face objects can be set to be lower than the first focus priority of ordinary faces, so that when the protagonist face and ordinary faces do not exist in the camera preview interface, the preset type of non-face objects are given priority to focus.

[0138] As a possible implementation method, when determining the first focus tracking priority of each object in the camera preview interface, if the camera preview interface contains a protagonist portrait, an ordinary portrait, and a preset type of non-face object, the first focus tracking priority of the protagonist portrait can be determined as the highest priority, the first focus tracking priority of the ordinary portrait can be determined as the second highest priority, and the first focus tracking priority of the preset type of non-face object can be determined as the lowest priority; if there is no protagonist portrait in the camera preview interface, and there are both ordinary portraits and a preset type of non-face object, the first focus tracking priority of the ordinary portrait can be determined as the highest priority, and the first focus tracking priority of the preset type of non-face object can be determined as the second highest priority; if there is no ordinary portrait in the camera preview interface, and there are both protagonist portraits and a preset type of non-face object, the first focus tracking priority of the protagonist portrait can be determined as the highest priority, and the first focus tracking priority of the preset type of non-face object can be determined as the second highest priority; if the camera preview interface contains only any one of the protagonist face, ordinary portrait, and preset type of object, the first focus priority of the type of object in the camera preview interface can be directly determined as the highest priority.

[0139] Step 105 : Generate a focus frame corresponding to each object according to the focus frame type of each object, and display the focus frame corresponding to each object in the camera preview interface according to the coordinates of each object.

[0140] In an embodiment of the present application, after determining the focus frame type of each object in the camera preview interface, the coordinates and focus frame type of each object can be sent to the Everything Focus Plug-in in the camera application, so that the Everything Focus Plug-in can draw the corresponding type of focus frame according to the focus frame type of each object, and display the drawn focus frame in the camera preview interface according to the coordinates of each object, so that the focus frame corresponding to each object in the camera preview interface follows the real-time position of the corresponding object.

[0141] It should be noted that the coordinates of an object may include the coordinates of the object's center point and the size (height and width) of the area corresponding to the object in the camera preview interface. For a face object, the coordinates of the object's center point may be the coordinates of the center point of the object's head area, and the corresponding size of the area in the camera preview interface may be the size corresponding to the object's head area. When drawing a focus frame, the size of the area corresponding to the object in the camera preview interface can be used as the size of the corresponding focus frame. When displaying the focus frame, the coordinates of the object's center point can be used as the center point coordinates of the corresponding focus frame to display the focus frame, so that the display position of the focus frame is consistent with the display position of the corresponding object.

[0142] For example, suppose the first tracking frame type corresponding to the protagonist portrait is a double-layer yellow frame, the second tracking frame type is a double-layer white frame, and the third tracking frame type corresponding to ordinary portraits and preset non-face objects is a single-layer white frame. Figure 6 As shown, when there is only a preset type of non-face object (cat) 601 in the camera preview interface 600, a single layer white frame 602 can be drawn and displayed in the display area corresponding to the cat; Figure 7 As shown, when a preset type of non-face object (cat) 701 and ordinary portraits 702, 703, and 704 exist in the camera preview interface 700 at the same time, a single-layer white frame 7011 can be drawn and displayed in the display area corresponding to the cat, and single-layer white frames 7021, 7031, and 7041 can be displayed in the head areas corresponding to the ordinary portraits 702, 703, and 704, respectively; when there is a protagonist portrait in the camera preview interface, a double-layer yellow frame or a double-layer white frame can be displayed in the head area of the protagonist portrait.

[0143] As a possible implementation, a focus frame of the first focus frame type can be used to represent the subject portrait that is locked in focus. Therefore, when the camera preview interface includes multiple subject portraits, the focus frame of one of the subject portraits can be determined as the first focus frame type, and the focus frames of the remaining subject portraits can be determined as the second focus frame type. That is, in a possible implementation of this embodiment of the application, the above step 105 can include:

[0144] When the objects include a first object whose object type is a portrait (portrait of the protagonist) ranked in the top N in the gallery, generating a first tracking focus frame corresponding to the first object according to the first tracking focus frame type, and displaying the first tracking focus frame corresponding to the first object in the camera preview interface according to the coordinates of the first object;

[0145] When the objects include a first object whose object type is a portrait (portrait of the protagonist) ranked in the top N in the gallery and at least one second object, a first tracking frame corresponding to the first object is generated according to the first tracking frame type, and a second tracking frame corresponding to each second object is generated according to the second tracking frame type, and the first tracking frame corresponding to the first object and the second tracking frame corresponding to each second object are displayed in the camera preview interface according to the coordinates of the first object and each second object, wherein the size of the first object is larger than the size of each second object.

[0146] In an embodiment of the present application, if the camera preview interface only includes a first object which is a portrait of a protagonist, the focus frame type of the first portrait can be directly determined as the first focus frame type, and a first focus frame of the first focus frame type can be drawn and displayed in the head area of the first object; if the camera preview interface includes multiple protagonist portraits, the protagonist portrait with the largest size can be determined as the first object based on the size of each protagonist object, and the other protagonist portraits can be determined as the second object, and then the focus frame type of the first object can be determined as the first focus frame type, and the focus frame type of the second object can be determined as the second focus frame type, and then the first focus frame of the first focus frame type can be drawn and displayed in the head area of the first object, and the second focus frame of the second focus frame type can be drawn and displayed in the head area of the second object.

[0147] For example, suppose the first focus frame type corresponding to the protagonist's portrait is a double-layer yellow frame, and the second focus frame type is a double-layer white frame. Figure 8 As shown, the camera preview interface 800 includes ordinary portraits 801, 802 and a protagonist portrait 803, then the protagonist portrait 803 can be determined as the first object, and a double-layer yellow frame 8031 can be drawn and displayed in the head area of the protagonist portrait 803, and single-layer white frames 8011 and 8021 can be drawn and displayed in the head areas of the ordinary portraits 801 and 802 respectively.

[0148] like Figure 9 As shown, the camera preview interface 900 includes ordinary portraits 901, 902 and protagonist portraits 903, 904, and the size of the protagonist portrait 904 is larger than the size of the protagonist portrait 903, then the protagonist portrait 904 can be determined as the first object, and the protagonist portrait 903 can be determined as the second object, and a double-layer yellow frame 9041 is drawn and displayed in the head area of the protagonist portrait 904, a double-layer white frame 9031 is drawn and displayed in the head area of the protagonist portrait 903, and single-layer white frames 9011 and 9021 are drawn and displayed in the head areas of the ordinary portraits 901 and 902 respectively.

[0149] Furthermore, if, during the preview process, the size of multiple portraits of the main characters changes due to positional changes, the focus frames of the portraits of the main characters can be redrawn based on the size changes of the portraits to further improve the accuracy and flexibility of focus tracking. That is, in one possible implementation of the embodiment of the present application, after generating the first focus frame corresponding to the first object according to the first focus frame type and generating the second focus frames corresponding to the second objects according to the second focus frame type, and displaying the first focus frame corresponding to the first object and the second focus frames corresponding to the second objects in the camera preview interface based on the coordinates of the first object and the second objects, the following may also be included:

[0150] If it is determined that the size of any second object is larger than the size of the first object, then any second object is determined as a new first object, and the first object is determined as a new second object, and the focus frame corresponding to the new first object is changed from the second focus frame to the first focus frame, and the focus frame corresponding to the new second object is changed from the first focus frame to the second focus frame.

[0151] In an embodiment of the present application, when the camera preview interface includes multiple portraits of the main characters, the focus frame type of each portrait can be switched in real time based on the position changes of each portrait. During the preview process, if the size of any second object is larger than the size of the first object due to the change in the position of the main character portrait, the second object can be determined as the new first object, and the focus frame of the new first object can be changed from the second focus frame to the first focus frame. The first object can also be determined as the new second object, and the focus frame of the new second object can be changed from the first focus frame to the second focus frame.

[0152] For example, if Figure 9 As shown, if the protagonist portrait 903 moves to the front of the protagonist portrait 904, the size of the protagonist portrait 903 is larger than the size of the protagonist portrait 904. For example, the positional relationship between the protagonist portrait 903 and the protagonist portrait 904 becomes Figure 10 When the positional relationship between the protagonist portrait 1003 (the same object as the protagonist portrait 103) and the protagonist portrait 1004 (the same portrait as the protagonist portrait 904) is concerned, since the size of the protagonist portrait 1003 is larger than the size of the protagonist portrait 1004, the focus frame of the protagonist portrait 1003 can be switched from the double-layer white frame 9031 to the double-layer yellow frame 10031, and the focus frame of the protagonist portrait 1004 can be switched from the double-layer yellow frame 9041 to the double-layer white frame 10041.

[0153] As a possible implementation method, when there is no interaction in the camera preview interface, the focus frame corresponding to each protagonist portrait can be continuously displayed.

[0154] Furthermore, when there is no interaction in the camera preview interface, the focus frame for normal portraits and pre-set non-face objects such as cats and dogs can automatically stop displaying after a period of time. That is, in a possible implementation of the embodiment of the present application, after the above step 105, the following steps may also be included:

[0155] When the object type is not a portrait (normal portrait) ranked in the top N in the image library or a non-face object of a preset type, determine the display duration of the focus tracking frame corresponding to the object;

[0156] When the display time of the focus tracking frame corresponding to the object reaches a time threshold, the focus tracking frame corresponding to the object stops being displayed.

[0157] As a possible implementation, during the camera preview process without interaction, the focus frames corresponding to ordinary portraits and preset non-face types can be stopped from being displayed in the camera preview interface when the display time of these frames reaches a threshold. In other words, the focus frames corresponding to ordinary portraits and preset non-face types are only displayed in the camera preview interface for a short period of time. This prevents the camera preview interface from displaying too many focus frames for a long time when there are many objects, resulting in a cluttered display and affecting the user's visual experience when shooting.

[0158] It should be noted that in actual use, the specific value of the time threshold can be set according to actual needs and specific application scenarios, and the embodiment of the present application does not limit this. For example, the time threshold can be 3 seconds, 5 seconds, 10 seconds, etc.

[0159] Step 106 : According to the first focus tracking priorities of the objects, any object with the highest first focus tracking priority is determined as a target focus tracking object, and focus tracking processing is performed on the target focus tracking object.

[0160] In an embodiment of the present application, after determining the first focus priority for each object, the object with the highest first focus priority can be determined as the target focus object, so that the target focus object can be focused during the preview process and after the user triggers shooting. It should be noted that when there are multiple objects with the highest first focus priority, any object with the highest first focus priority can be determined as the target focus object, or an object that meets the preset rules can be selected from the objects with the highest first focus priority according to the preset rules as the target focus object. This embodiment of the present application does not limit this.

[0161] As a possible implementation method, after determining the target focusing object, an autofocus region of interest command can be generated according to the coordinates of the target focusing object, and the autofocus region of interest command can be sent to the autofocus algorithm to perform focusing processing on the target focusing object through the autofocus algorithm.

[0162] It should be noted that when the object types include portraits ranked in the top N in the gallery (protagonist portraits), portraits ranked in the top N in non-gallery (ordinary portraits), and non-face objects of preset types, if the target tracking object is finally determined to be the protagonist portrait, the tracking frame type of the target tracking object can be determined as the first tracking frame type (such as a double-layer yellow frame), and when there is no interaction in the camera preview interface, the first tracking frame (double-layer yellow frame) corresponding to the target tracking object is continuously displayed in the camera preview interface; if the target tracking object is finally determined to be an ordinary portrait or a non-face object of preset type, the tracking frame display strategy of the target tracking object is the same as the above steps, that is, the tracking frame type of the target tracking object is determined to be the third tracking frame type (such as a single-layer white frame), and when there is no interaction in the camera preview interface, the display time of the tracking frame of the target tracking object reaches the time threshold, that is, the process of automatic tracking of ordinary faces and non-face objects of preset types can be not explicitly displayed to the user.

[0163] Furthermore, when the object types include portraits ranked in the top N in the gallery (portraits of the main characters), portraits not ranked in the top N in the gallery (ordinary portraits), and non-face objects of a preset type, the target focus tracking object can be determined by the following method:

[0164] Determine whether the objects include the first object whose object type is a portrait (portrait of the protagonist) ranked in the top N in the gallery;

[0165] If the objects include at least one first object, the first object is determined as the object with the highest first priority, and any first object is determined as the target focus tracking object;

[0166] If the first object is not included in the objects, determining whether the objects include a second object whose object type is not a portrait ranked in the top N in the image library;

[0167] If the objects include at least one second object, determining the second object as the object with the highest first priority, and determining any second object as the target tracking focus object;

[0168] If the second object is not included in the objects, determining whether the objects include a third object whose object type is a non-face object of a preset type;

[0169] If the objects include at least one third object, the third object is determined as the object with the highest first priority, and any third object is determined as the target tracking focus object;

[0170] If the objects do not include the third object, center focus processing is performed in the preview interface.

[0171] As a possible implementation method, when the camera preview interface includes a portrait of the protagonist, any object can be selected from the portrait of the protagonist as the target focus tracking object; when the camera preview interface does not include a portrait of the protagonist and includes ordinary portraits, any object can be selected from the ordinary portraits as the target focus tracking object; when the camera preview interface only includes preset types of non-face objects, any object can be selected from the preset types of non-face objects as the target focus tracking object.

[0172] Furthermore, for objects of the same object type, their first focus priority may also be the same. Therefore, when there are multiple objects with the highest first focus priority, the focus priorities of the objects with the highest first focus priority can be re-sorted to ensure that the object automatically focused on is the object that the user is currently most interested in, further improving the accuracy of automatic focus. That is, in a possible implementation of the embodiment of the present application, the above step 106 may include:

[0173] Determine a second focus tracking priority for each object having the highest first focus tracking priority;

[0174] The object with the second highest tracking priority is determined as the target tracking object.

[0175] In an embodiment of the present application, when there are multiple objects with the highest first focus tracking priority, the second focus tracking priority of each object with the highest first focus tracking priority can be further determined, and the object with the highest second focus tracking priority can be determined as the target focus tracking object, so that the target focus tracking object is the object that the user is currently most concerned about, thereby improving the accuracy of automatic focus tracking.

[0176] As a possible implementation, the second focus tracking priority of each object can be determined as follows:

[0177] Obtain the preset priority of each object with the highest first focus tracking priority;

[0178] Determine the preset priority of each object as the second focus tracking priority of each object;

[0179] or,

[0180] Determine the size of each object with the highest priority for first focus tracking;

[0181] Determine the second focus tracking priority of each object based on the size of each object.

[0182] As an example, when the object types include portraits ranked in the top N in the gallery (protagonist portraits), portraits ranked in the top N in the non-gallery (ordinary portraits), and non-face objects of a preset type, since the protagonist portrait is a high-frequency image in the gallery determined by the gallery application based on the images collected by the user, after the gallery application determines the protagonist portrait, it can also determine the preset priority of each protagonist portrait based on the number of images corresponding to the protagonist portrait in the gallery, that is, the more images corresponding to the protagonist portrait, the higher the preset priority of the protagonist portrait, and then each protagonist portrait and the preset priority of each protagonist portrait are simultaneously sent to the camera application. Therefore, when it is determined that the object with the highest first focus tracking priority is the protagonist portrait and there are multiple protagonist portraits, the preset priority of each protagonist portrait can be obtained as the second focus tracking priority of each protagonist portrait, and then the protagonist portrait with the highest second focus tracking priority can be determined as the target focus tracking object.

[0183] Alternatively, when the subject with the highest first focus tracking priority is a portrait of the main character, the second focus tracking priority of each portrait can be determined based on the size of the portrait. That is, the larger the size of the portrait, the higher the second focus tracking priority of the portrait. The portrait with the highest second focus tracking priority can then be determined as the target focus tracking subject. In other words, the portrait with the largest size can be determined as the target focus tracking subject.

[0184] It should be noted that when the object with the highest first focus tracking priority is the portrait of the protagonist and there are multiple portraits of the protagonist, any one of the above methods can be selected to determine the second focus tracking priority of the protagonist portrait according to actual needs and specific application scenarios. This embodiment of the present application does not limit this.

[0185] As an example, when the object types include portraits ranked in the top N in the gallery (portraits of the protagonist), portraits ranked in the top N in non-gallery (ordinary portraits), and non-face objects of preset types, since ordinary portraits and non-face objects of preset types appear randomly, their corresponding priorities cannot be preset. Therefore, if the object with the highest first focus tracking priority is an ordinary portrait or a non-face object of preset type, and there are multiple objects, the second focus tracking priority of each object can be determined based on the size of each object with the highest first focus tracking priority, and the object with the highest second focus tracking priority (i.e., the largest object) can be determined as the target focus tracking object.

[0186] The focus tracking method provided in the embodiment of the present application selects the object that the user is most interested in in the current shooting scene according to the object type of each object in the camera preview interface during the preview process, and uses it as the highest priority focus tracking object to focus on the object that the user is currently most interested in. Different focus tracking frames are used for displaying different types of objects. This not only allows the focus tracking object to be adaptively adjusted according to the real-time shooting scene, thereby improving the shooting effect of the object that the user is most interested in, but also eliminates the need for manual operation by the user, thereby improving the flexibility and ease of use of focus tracking and improving the user's shooting experience.

[0187] In one possible implementation form of the present application, users may be allowed to manually select corresponding objects for focus tracking based on their own shooting needs, so as to further enhance the flexibility and interactivity of automatic focus tracking and further enhance the user's shooting experience.

[0188] The following combination Figure 11 , the focus tracking method provided in the embodiment of the present application is further explained.

[0189] Figure 11 A flow chart of another focus tracking method provided in an embodiment of the present application is shown.

[0190] like Figure 11 As shown, the focus tracking method includes the following steps:

[0191] Step 1101: In response to a first operation by the user, display a camera preview interface.

[0192] Step 1102 : performing recognition processing on each object displayed in the camera preview interface to determine the object type of each object.

[0193] Step 1103 : Determine the focus frame type of each object according to the object type of each object.

[0194] Step 1104 : Determine the first focus tracking priority of each object according to the object type and the preset priority strategy of each object.

[0195] The preset priority strategy is set according to the user's attention level to each object type.

[0196] Step 1105 : Generate a focus frame corresponding to each object according to the focus frame type of each object, and display the focus frame corresponding to each object in the camera preview interface according to the coordinates of each object.

[0197] Step 1106: According to the first focus tracking priority of each object, any object with the highest first focus tracking priority is determined as the target focus tracking object, and the target focus tracking object is subjected to focus tracking processing.

[0198] The specific implementation process and principles of the above steps 1101-1106 can be referred to the detailed description of the above embodiment and will not be repeated here.

[0199] Step 1107: In response to a user's click operation on any object, the object is determined as a designated focus tracking object.

[0200] In an embodiment of the present application, during the preview process, not only can the object with the highest user attention be automatically selected as the target focus object based on the object type, focus priority and other parameters of each object in the camera preview interface, and the target focus object can be continuously focused during the subsequent preview process and shooting process without subsequent user interaction, but the user can also be allowed to select any object in the camera preview interface as the designated focus object according to the current actual shooting needs, and the designated focus object selected by the user can be continuously focused during the preview and subsequent shooting process without further interaction from the user.

[0201] Step 1108: If the designated focus object is the target focus object, the focus process is continued on the designated focus object until a click operation on any object by the user is obtained.

[0202] In an embodiment of the present application, since the target focus object is an automatic focus object determined according to the focus priority of each object in the camera preview interface, the target focus object has been focused on before the user clicks on any object in the camera preview interface. Therefore, if the designated focus object selected by the user is the target focus object, the designated focus object (that is, the previous target focus object) can continue to be focused on until the user selects another object as the designated focus object, then the focus on the target focus object is stopped, and the newly selected designated focus object is focused on.

[0203] Step 1109: If the designated focus object is not the target focus object, the focus processing on the target focus object is suspended, and the focus processing on the designated focus object is continued until a user click operation on any object is obtained.

[0204] In an embodiment of the present application, since the automatically selected target focus object has been focused on before the user manually selects the designated focus object, if the designated focus object selected by the user supplements the target focus object, the focus processing of the target focus object can be paused, and the focus processing can be switched to continuous focus processing on the designated focus object until the user selects another object as the designated focus object, and then the focus processing is switched to the new designated focus object selected by the user.

[0205] Furthermore, since after the user clicks to select a designated focus target, it can be determined that the currently locked focus target is the designated focus target, only the focus frame of the designated focus target can be displayed, making the camera display interface more concise and reducing the impact of displaying too many focus frames on the user's shooting experience. That is, in a possible implementation of the embodiment of the present application, after the above step 1107, the following may also be included:

[0206] If the designated focus object is the target focus object and its object type is a portrait ranked in the top N in the gallery (portraits of the main character), the focus frame type of the designated focus object remains unchanged, and the focus frame of the designated focus object continues to be displayed, while the focus frames of all objects except the designated focus object are temporarily displayed.

[0207] If the designated focus object is not the target focus object, and the object type of the designated focus object is a portrait ranked in the top N in the gallery (portrait of the protagonist), the focus frame type of the designated focus object is changed to the first focus frame type, and the first focus frame corresponding to the designated focus object is generated and continuously displayed based on the first focus frame type, while the focus frames of all objects other than the designated focus object are temporarily displayed;

[0208] If the object type of the specified focus object is not a portrait ranked in the top N in the gallery (portrait of the protagonist), the focus frame type of the specified focus object will be changed to the fourth focus frame type, and the fourth focus frame corresponding to the specified focus object will be generated and continuously displayed according to the fourth focus frame type, and the focus frames of all objects except the specified focus object will be temporarily displayed.

[0209] In the embodiment of the present application, according to the description of the aforementioned embodiment, when the object types include a protagonist portrait, an ordinary portrait, and a preset type of non-face object, if the determined target focus object is the protagonist portrait, the focus frame type of the target focus object can be a first focus frame type (such as a double-layer yellow frame), so as to indicate through the first focus frame type that the target focus object is currently being focused on, so as to explicitly display the currently focused protagonist portrait to the user. Therefore, when the designated focus object selected by the user is the target focus object and the target focus object is the protagonist portrait, that is, when the user clicks on the designated focus object, the focus frame of the designated focus object is a focus frame of the first focus frame type, which can be used to indicate the protagonist portrait being focused on. Therefore, the focus frame of the designated focus object can be kept unchanged, and the focus frame of the designated focus object can be continuously displayed, and the focus frames of the remaining objects in the camera preview interface can be temporarily displayed.

[0210] For example, if Figure 8The camera preview interface 800 shown includes two ordinary portraits 801 and 802, whose corresponding focus frames 8011 and 8021 are single-layer white frames (third focus frame type), and a protagonist portrait 803, and the protagonist portrait 803 is the target focus object, and its corresponding focus frame 8031 is a double-layer yellow frame (first focus frame type). If the protagonist portrait 803 is clicked in the camera preview interface 800, the protagonist portrait 803 can be determined as the designated focus object, and the focus frame 8031 remains unchanged, and the focus frames 8011 and 8021 corresponding to the ordinary portraits 801 and 802 are paused. Figure 12 shown.

[0211] In an embodiment of the present application, if the designated focus object is not the target focus object, and the object type of the designated focus object is a portrait of the protagonist, it can be determined that before the designated focus object is clicked by the user, its corresponding focus frame type is the second focus frame type (such as a double-layer white frame). The second focus frame type can be used to indicate that the object is the protagonist object, but is not currently being focused on. After the object is determined to be the designated focus object, the designated focus object needs to be focused, so that the focus frame type of the designated focus object can be changed from the second focus frame type to the first focus frame type (such as a double-layer yellow frame) to indicate that the designated focus object is the portrait of the protagonist currently being focused on.

[0212] For example, if Figure 9 The camera preview interface 900 shown includes two ordinary portraits 901 and 902, whose corresponding focus frames 9011 and 9021 are single-layer white frames (the third focus frame type), and two protagonist portraits 903 and 904, among which the protagonist portrait 904 is the target focus object, and its corresponding focus frame 9041 is a double-layer yellow frame (the first focus frame type), and the protagonist portrait 903 is not the target focus object, and its corresponding focus frame 9031 is a double-layer white frame (the second focus frame type). If the user clicks on the protagonist portrait 903 in the camera preview interface 900, the protagonist portrait 903 can be determined as the designated focus object, and the focus frame 9031 can be changed to a double-layer yellow frame, and the focus frames 9011 and 9021 corresponding to the ordinary portraits 901 and 902 can be paused, as well as the focus frame 9041 corresponding to the protagonist portrait 904 can be paused. Figure 13 shown.

[0213] In an embodiment of the present application, if the designated focus object is not the protagonist portrait, that is, the designated focus object is an ordinary portrait or a preset type of non-face object, since the focus frame type corresponding to these two types of objects is the third focus frame type (such as a single-layer white frame), this type of focus frame will not be displayed continuously and cannot indicate that the focus object has been locked. Therefore, when the designated focus object is an ordinary portrait or a preset type of non-face object, the focus frame type of the designated focus object can be changed to the fourth focus frame type (such as a single-layer yellow frame), and the fourth focus frame of the designated focus object is generated and continuously displayed according to the fourth focus frame type, and the focus frames of the remaining objects in the camera preview interface are temporarily displayed, so that the designated focus object that has been locked in focus can be explicitly displayed to the user through the fourth focus frame.

[0214] For example, if Figure 9 The camera preview interface 900 shown includes two ordinary portraits 901 and 902, whose corresponding focus frames 9011 and 9021 are single-layer white frames (third focus frame type), and two protagonist portraits 903 and 904, among which the protagonist portrait 904 is the target focus object, and its corresponding focus frame 9041 is a double-layer yellow frame (first focus frame type), and the protagonist portrait 903 is not the target focus object, and its corresponding focus frame 9031 is a double-layer white frame (second focus frame type). If the user clicks on the ordinary portrait 901 in the camera preview interface 900, the protagonist portrait 901 can be determined as the designated focus object, and the focus frame 9011 can be changed to a single-layer yellow frame, and the focus frame 9021 corresponding to the ordinary portrait 902 can be paused, as well as the focus frames 9031 and 9041 corresponding to the protagonist portraits 903 and 904 can be paused. Figure 14 shown.

[0215] Furthermore, after the user clicks to select any object in the camera preview interface as the designated focus target, if the user clicks the designated focus target again, the locked focus on the designated focus target can be canceled and the automatic focus mode can be re-entered. That is, in a possible implementation of the embodiment of the present application, after the above step 1107, the following steps may also be included:

[0216] In response to the user's click operation on the specified focus object, the focus processing of the specified focus object is paused, and according to the first focus priority of each object, any object with the highest first focus priority is determined as the target focus object, and the focus processing is performed on the target focus object.

[0217] In an embodiment of the present application, if the user clicks on a designated focus object that has been selected, that is, the user deselects the designated focus object, the focus processing of the designated focus object can be canceled, and the automatic focus process of the embodiment of the present application can be re-executed, that is, return to execute step 1102 and its subsequent steps of the embodiment of the present application.

[0218] The focus tracking method provided in the embodiment of the present application selects the object that the user is most concerned about in the current shooting scene according to the object type of each object in the camera preview interface during the preview process as the highest priority focus tracking object, so as to focus on the object that the user is currently most concerned about, and uses different focus tracking frames for displaying different types of objects; and allows the user to manually select any object as the designated focus tracking object during the preview process, and continuously focus on the designated focus tracking object, so that not only can the focus tracking object be adaptively adjusted according to the real-time shooting scene, improving the shooting effect of the object that the user is most concerned about, and improving the flexibility and ease of use of focus tracking, but also by allowing the user to select or change the focus tracking object according to actual shooting needs, the flexibility and interactivity of automatic focus tracking are improved, and the user's shooting experience is further improved.

[0219] The following combination Figure 15 , further explains the software architecture of the focus tracking method provided in the embodiment of the present application.

[0220] Figure 15 A software architecture diagram of a focus tracking method provided in an embodiment of the present application is shown.

[0221] like Figure 15 As shown, the software architecture of the focus tracking method includes a camera application layer, a camera framework layer, a camera hardware abstraction layer, and a camera driver layer.

[0222] The camera application layer includes the user interface, plug-in modules, and camera management. The user interface includes functions such as photo, video, and portrait mode, which are used to display control elements for these functions within the camera application's user interface. The plug-in modules include the AutoTrack plug-in, the flash plug-in, and the filter plug-in. Camera management includes device management, session control management, and surface management. It's important to note that the AutoTrack plug-in handles user interaction responses and algorithm result processing.

[0223] The camera framework layer includes the camera service, which includes the CamManager class, session control, camera device, and output.

[0224] The camera hardware abstraction layer includes CamX and CHI. CamX represents a collection of code implementations for general functional interfaces, while CHI represents a collection of code implementations for customizable requirements. CamX includes Camera3Device, session control, output, semantic engine (semanticEngine), SWL, and sensor node (sensornode). The semantic engine includes nodes for the cyberFocus algorithm, faceFeature algorithm, and motionDec algorithm.

[0225] It should be noted that the All Things Focus Tracking algorithm node is the node that carries out the All Things Focus Tracking algorithm and is located on a node in the camera post-processing pipeline; the Facial Feature Algorithm node is the algorithm node that performs the main character portrait recognition (i.e., the portraits in the top N in the sorting of the gallery mentioned in the above embodiment). SWL also includes an autofocus algorithm node for executing the focus service. CHI includes the ChiUseCase class, which is mainly responsible for business processing and resource management in CHI.

[0226] The camera driver layer includes at least one camera driver; the number of drivers corresponds to the number of cameras installed in the terminal device.

[0227] Figure 16 A timing diagram of a focus tracking method provided in an embodiment of the present application is shown.

[0228] like Figure 16 As shown in the figure, the timing diagram of the focus tracking method involves the shooting mode module, the all-object focus tracking plug-in, and the camera management module in the camera application layer in the software architecture diagram, as well as the camera service in the camera framework layer, the Camera3Device in the camera hardware abstraction layer, the semantic engine, the all-object focus tracking algorithm node, the face feature algorithm node, and the autofocus algorithm node. Specifically, it includes the following steps:

[0229] S1601: The shooting mode module receives a request from the user to start the camera application;

[0230] S1602: The shooting mode module sends a camera start request to the camera management module;

[0231] S1603: The camera management module sends a request to obtain the camera service to the camera service;

[0232] S1604: The camera service sends a request to Camera3Device to construct a camera instance;

[0233] S1605: Camera3Device returns the camera instance to the camera management module;

[0234] S1606: The shooting mode module sends a request to the all-things focus tracking plug-in to load the all-things focus tracking plug-in;

[0235] S1607: The all-objects focus tracking plug-in sends a request to the camera management module to activate the all-objects focus tracking algorithm;

[0236] S1608: The camera management module sends a request to the semantic engine to construct an all-object tracking algorithm;

[0237] S1609: Enables semantic perception algorithms for all-object tracking;

[0238] S1610: Camera3Device obtains preview frames from the frame buffer one by one and sends them to the semantic engine;

[0239] S1611: The semantic engine inputs the preview frame into the all-object tracking algorithm node to identify the object in the preview frame;

[0240] S1612: The all-object tracking algorithm node inputs the preview frame into the face feature algorithm node to identify the protagonist portrait in the preview frame (i.e., the portraits ranked in the top N in the image library mentioned in the above embodiment);

[0241] S1613: When the facial feature algorithm node determines that a portrait of the protagonist exists in the preview frame, the facial feature algorithm node returns the coordinates of the portrait of the protagonist to the object tracking algorithm node;

[0242] S1614: When a portrait of a protagonist exists in the All-Purpose Focus Tracking algorithm node, the coordinates of the portrait of the protagonist are returned to the All-Purpose Focus Tracking plug-in;

[0243] S1615: The user interface draws a first type of focus tracking frame (double-layer yellow frame) corresponding to the protagonist's portrait;

[0244] S1616: The all-object tracking algorithm node sends an auto-focus region of interest instruction to the auto-focus algorithm node to automatically focus on the subject portrait;

[0245] S1617: When there is no portrait of the main character, the facial feature algorithm node identifies the largest portrait;

[0246] S1618: When the facial feature algorithm node determines that a normal human figure exists, the facial feature algorithm node returns the coordinates of the normal human figure to the object tracking algorithm node.

[0247] S1619: When a normal portrait exists in the All-Purpose Focus Tracking algorithm node, the coordinates of the normal portrait are returned to the All-Purpose Focus Tracking plug-in.

[0248] S1620: The user interface draws a third type of focus tracking frame (single white frame);

[0249] S1621: Sending an auto-focusing region of interest instruction;

[0250] S1622: The protagonist's portrait is detected;

[0251] S1623: Enter the subject detection algorithm when there is no portrait of the protagonist;

[0252] S1624: When a normal portrait or a non-face object of a preset type exists, the coordinates of the normal portrait or the non-face object of the preset type are returned;

[0253] S1625: The user interface draws a third type of focus tracking frame (single white frame) corresponding to a normal portrait or a preset type of non-face object;

[0254] S1626: The all-object tracking algorithm node sends an auto-focusing region of interest instruction to the auto-focus algorithm node;

[0255] S1627: No entity recognition algorithm;

[0256] S1628: When there is no identifiable entity in the preview frame, the all-object tracking algorithm node sends a center auto-focus instruction to the auto-focus algorithm node;

[0257] S1629: The all-objects focus tracking algorithm node detects that the user manually selects a non-protagonist portrait;

[0258] S1630: Skip the main character portrait detection;

[0259] S1631: The user interface draws a fourth type of focus frame (single yellow frame) corresponding to the specified focus object selected by the user;

[0260] S1632: The all-object tracking algorithm node sends an auto-focusing region of interest instruction to the auto-focus algorithm node;

[0261] S1633: After the user manually selects a specific focus target, the all-object focus tracking algorithm node detects that the portrait of the main character has entered the frame.

[0262] S1634: The user interface displays a fourth type of focus frame (single yellow frame) corresponding to the designated focus object, and maintains the focus frame on the object selected by the user, that is, the designated focus object.

[0263] S1635: The All Focus Tracking plug-in detects that the user has deselected the selected object or locked the focus object;

[0264] S1636: After deselection, the all-object tracking algorithm node switches back to the automatic tracking strategy;

[0265] S1637: The All Focus Tracking plugin detected that the user manually selected the main character portrait;

[0266] S1638: The All Things Focus Tracking plug-in maintains the highest priority for focus tracking.

[0267] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0268] Corresponding to the focus tracking method described in the above embodiment, Figure 17 A structural block diagram of the focus tracking device provided in an embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0269] Reference Figure 17 , the apparatus 1700, comprising:

[0270] A first display module 1701 is configured to display a camera preview interface in response to a first operation by a user;

[0271] A first determining module 1702 is configured to perform recognition processing on each object displayed in the camera preview interface to determine the object type of each object;

[0272] A second determining module 1703 is configured to determine a focus frame type for each object based on the object type of each object;

[0273] A third determining module 1704 is configured to determine a first focus tracking priority for each object based on the object type and a preset priority strategy, wherein the preset priority strategy is set based on the user's level of interest in each object type;

[0274] The second display module 1705 is used to generate a focus frame corresponding to each object according to the focus frame type of each object, and display the focus frame corresponding to each object in the camera preview interface according to the coordinates of each object;

[0275] The first focus tracking module 1706 is configured to determine, based on the first focus tracking priorities of the respective objects, any object with the highest first focus tracking priority as a target focus tracking object, and perform focus tracking processing on the target focus tracking object.

[0276] In actual use, the focus tracking device provided in the embodiment of the present application can be configured in any terminal device to execute the aforementioned focus tracking method.

[0277] The focus tracking device provided in the embodiment of the present application selects the object that the user is most interested in in the current shooting scene according to the object type of each object in the camera preview interface during the preview process, and uses it as the highest priority focus tracking object to focus on the object that the user is currently most interested in. Different focus tracking frames are used for displaying different types of objects. This not only allows the focus tracking object to be adaptively adjusted according to the real-time shooting scene, thereby improving the shooting effect of the object that the user is most interested in, but also eliminates the need for manual operation by the user, thereby improving the flexibility and ease of use of focus tracking and improving the user's shooting experience.

[0278] In one possible implementation of the present application, the object types include the following types: portraits ranked in the top N in the image gallery, portraits ranked in the top N in the non-image gallery, and non-face objects of a preset type, where N is a positive integer greater than or equal to 1;

[0279] The focus frame type for the portraits ranked in the top N in the gallery is the first or second focus frame type, while the focus frame type for portraits not ranked in the top N in the gallery and for non-face objects of a preset type is the third focus frame type. The first, second, and third focus frame types are all different.

[0280] The preset priority strategy is: the first focus tracking priority of the portraits ranked in the top N in the gallery is higher than the first focus tracking priority of the portraits ranked in the top N outside the gallery, and the first focus tracking priority of the portraits ranked in the top N outside the gallery is higher than the first focus priority of the preset type of non-face objects.

[0281] Furthermore, in another possible implementation of the present application, the second display module 1705 includes:

[0282] A first display unit is configured to generate a first tracking focus frame corresponding to the first object according to the second tracking focus frame type when the objects include a first object whose object type is a portrait ranked in the top N in the gallery, and to display the first tracking focus frame corresponding to the first object in the camera preview interface according to the coordinates of the first object;

[0283] The second display unit is used to generate a first tracking frame corresponding to the first object according to the first tracking frame type, and generate a second tracking frame corresponding to each second object according to the second tracking frame type, when the objects include a first object whose object type is a portrait ranked in the top N in the gallery and at least one second object, and display the first tracking frame corresponding to the first object and the second tracking frame corresponding to each second object in the camera preview interface according to the coordinates of the first object and each second object, wherein the size of the first object is larger than the size of each second object.

[0284] Furthermore, in another possible implementation of the present application, the second display module 1705 further includes:

[0285] The first changing unit is used to determine any second object as a new first object if it is determined that the size of any second object is larger than the size of the first object, and to determine the first object as a new second object, and to change the focusing frame corresponding to the new first object from the second focusing frame to the first focusing frame, and to change the focusing frame corresponding to the new second object from the first focusing frame to the second focusing frame.

[0286] Furthermore, in another possible implementation of the present application, the apparatus 1700 further includes:

[0287] A fourth determining module is configured to determine a display duration of a focus tracking frame corresponding to an object when the object type is a non-human face object that is ranked in the top N in the image gallery or a non-human face object of a preset type;

[0288] The first display stopping module is configured to stop displaying the focus tracking frame corresponding to the object when the display duration of the focus tracking frame corresponding to the object reaches a duration threshold.

[0289] Furthermore, in another possible implementation of the present application, the first focus tracking module 1706 includes:

[0290] A first judging unit is configured to judge whether the objects include a first object whose object type is a portrait ranked in the top N in the image library;

[0291] a first determining unit configured to, if the objects include at least one first object, determine the first object as the object with the highest first priority, and determine any first object as a target focus tracking object;

[0292] a second determining unit configured to determine whether the objects include a second object whose object type is not a portrait ranked in the top N in the image library if the objects do not include the first object;

[0293] a second determining unit, configured to, if the objects include at least one second object, determine the second object as the object with the highest first priority, and determine any second object as a target focus tracking object;

[0294] a third determining unit configured to determine whether the objects include a third object of a preset non-face type if the objects do not include the second object;

[0295] a third determining unit, configured to, if the objects include at least one third object, determine the third object as the object with the highest first priority, and determine any third object as a target focus tracking object;

[0296] The first focusing unit is configured to perform center focusing processing in the preview interface if the objects do not include the third object.

[0297] Furthermore, in another possible implementation of the present application, the apparatus 1700 further includes:

[0298] a fifth determining module, configured to determine any object as a designated focus tracking object in response to a user's click operation on any object;

[0299] The second focus tracking module is configured to, if the designated focus tracking object is the target focus tracking object, continue to focus on the designated focus tracking object until a user clicks on any object;

[0300] The third focusing module is used to suspend focusing on the target focusing object if the designated focusing object is not the target focusing object, and continue focusing on the designated focusing object until a user clicks on any object.

[0301] Furthermore, in another possible implementation of the present application, the apparatus 1700 further includes:

[0302] A third display module is configured to maintain the focus frame type of the designated focus object unchanged and continuously display the focus frame of the designated focus object, and to pause displaying the focus frames of all objects other than the designated focus object, if the designated focus object is the target focus object and the object type of the designated focus object is a portrait ranked in the top N in the gallery;

[0303] a fourth display module configured to, if the designated focus object is not the target focus object and the object type of the designated focus object is a portrait ranked in the top N in the gallery, change the focus frame type of the designated focus object to a first focus frame type, generate and continuously display a first focus frame corresponding to the designated focus object based on the first focus frame type, and pause display of focus frames for objects other than the designated focus object;

[0304] The fifth display module is used to change the focus frame type of the specified focus object to the fourth focus frame type if the object type of the specified focus object is not a portrait ranked in the top N in the gallery, and to generate and continuously display the fourth focus frame corresponding to the specified focus object according to the fourth focus frame type, and to pause displaying the focus frames of all objects except the specified focus object.

[0305] Furthermore, in another possible implementation of the present application, the apparatus 1700 further includes:

[0306] The fourth focus tracking module is used to pause the focus tracking process of the specified focus tracking object in response to the user's click operation on the specified focus tracking object, and determine any object with the highest first focus tracking priority as the target focus tracking object according to the first focus tracking priority of each object, and perform focus tracking on the target focus tracking object.

[0307] Furthermore, in another possible implementation of the present application, the objects include multiple objects with the highest first focus tracking priority; accordingly, the first focus tracking module 1706 includes:

[0308] a fourth determining unit, configured to determine a second focus tracking priority for each object having the highest first focus tracking priority;

[0309] The fifth determining unit is configured to determine the object with the highest second focus tracking priority as the target focus tracking object.

[0310] Furthermore, in another possible implementation of the present application, the fourth determining unit is specifically configured to:

[0311] Obtain the preset priority of each object with the highest first focus tracking priority;

[0312] The preset priority of each object is determined as the second focus tracking priority of each object.

[0313] or,

[0314] Determine the size of each object with the highest priority for first focus tracking;

[0315] Determine the second focus tracking priority of each object based on the size of each object.

[0316] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of this application. Their specific functions and technical effects can be found in the method embodiment section and will not be repeated here.

[0317] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0318] In order to implement the above embodiments, the present application also proposes a terminal device.

[0319] Figure 18 This is a schematic diagram of the structure of a terminal device according to an embodiment of the present application.

[0320] like Figure 18 As shown, the terminal device 1800 includes:

[0321] The memory 1810 and at least one processor 1820, a bus 1830 connecting different components (including the memory 1810 and the processor 1820), the memory 1810 stores a computer program, and when the processor 1820 executes the program, the focus tracking method described in the embodiment of the present application is implemented.

[0322] Bus 1830 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures. Examples of these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0323] The terminal device 1800 typically includes a variety of electronic device readable media. These media can be any available media that can be accessed by the terminal device 1800, including volatile and non-volatile media, removable and non-removable media.

[0324] The memory 1810 may also include computer system readable media in the form of volatile memory, such as random access memory (RAM) 1840 and / or cache memory 1850. The terminal device 1800 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 1860 may be used to read and write non-removable, non-volatile magnetic media ( Figure 18 Not shown, often called a "hard drive"). Although Figure 18 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 1830 via one or more data medium interfaces. Memory 1810 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of various embodiments of the present application.

[0325] A program / utility 1880 having a set (at least one) of program modules 1870 may be stored, for example, in memory 1810. Such program modules 1870 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may include an implementation of a network environment. Program modules 1870 generally implement the functions and / or methods of the embodiments described herein.

[0326] Terminal device 1800 can also communicate with one or more external devices 1890 (e.g., a keyboard, pointing device, display 1891, etc.), one or more devices that enable a user to interact with terminal device 1800, and / or any device that enables terminal device 1800 to communicate with one or more other computing devices (e.g., a network card, a modem, etc.). This communication can occur via input / output (I / O) interface 1892. Furthermore, terminal device 1800 can communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network such as the Internet) via network adapter 1893. As shown, network adapter 1893 communicates with other modules of terminal device 1800 via bus 1830. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with terminal device 1800, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0327] The processor 1820 executes the programs stored in the memory 1810 to perform various functional applications and data processing.

[0328] It should be noted that the implementation process and technical principles of the terminal device of this embodiment can be found in the above explanation of the focus tracking method of the embodiment of the present application, and will not be repeated here.

[0329] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned various method embodiments can be implemented.

[0330] An embodiment of the present application provides a computer program product. When the computer program product is run on a terminal device, the terminal device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0331] If the integrated unit is implemented as a software functional unit and sold or used as a standalone product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the process in the above-mentioned method embodiment by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When executed by a processor, the computer program can implement the steps of each of the above-mentioned method embodiments. The computer program includes computer program code, which can be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium can at least include: any entity or device capable of carrying computer program code to a camera / terminal device, a recording medium, computer memory, read-only memory (ROM), random access memory (RAM), an electric carrier signal, a telecommunications signal, and a software distribution medium. Examples include a USB flash drive, a mobile hard drive, a magnetic disk, or an optical disk. In some jurisdictions, based on legislation and patent practice, computer-readable media cannot be electric carrier signals or telecommunications signals.

[0332] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0333] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0334] In the embodiments provided in this application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are merely illustrative. For example, the division of the modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0335] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0336] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A focus tracking method, characterized in that: include: In response to a first operation by the user, displaying a camera preview interface; Performing recognition processing on each object displayed in the camera preview interface to determine the object type of each object; determining a focus frame type for each of the objects according to the object type of each of the objects; determining a first focus tracking priority for each of the objects according to an object type and a preset priority strategy, wherein the preset priority strategy is set according to the user's level of attention to each of the object types; generating a focus frame corresponding to each of the objects according to the focus frame type of each of the objects, and displaying the focus frame corresponding to each of the objects in the camera preview interface according to the coordinates of each of the objects; According to the first focus tracking priority of each of the objects, any one of the objects with the highest first focus tracking priority is determined as a target focus tracking object, and focus tracking processing is performed on the target focus tracking object.

2. The method according to claim 1, wherein The object types include the following types: portraits ranked in the top N in the gallery, portraits ranked in the top N in non-gallery, and non-face objects of preset types, where N is a positive integer greater than or equal to 1; The focus frame type of the portraits ranked in the top N in the image gallery is the first focus frame type or the second focus frame type, and the focus frame type of the portraits ranked in the top N in the non-image gallery and the focus frame type of the preset non-face object is the third focus frame type, wherein the first focus frame type, the second focus frame type, and the third focus frame type are all different; The preset priority strategy is: the first focus tracking priority of the portraits ranked in the top N in the gallery is higher than the first focus tracking priority of the portraits ranked in the top N in the non-gallery, and the first focus tracking priority of the portraits ranked in the top N in the non-gallery is higher than the first focus priority of the preset type of non-face objects.

3. The method according to claim 2, wherein Generating a focus frame corresponding to each object according to the focus frame type of each object, and displaying the focus frame corresponding to each object in the camera preview interface according to the coordinates of each object, includes: When the objects include a first object whose object type is a portrait ranked in the top N in the gallery, generating a first focus frame corresponding to the first object according to the first focus frame type, and displaying the first focus frame corresponding to the first object in the camera preview interface according to the coordinates of the first object; When the objects include a first object whose object type is a portrait ranked in the top N in the gallery and at least one second object, a first tracking frame corresponding to the first object is generated according to the first tracking frame type, and a second tracking frame corresponding to each second object is generated according to the second tracking frame type, and the first tracking frame corresponding to the first object and the second tracking frame corresponding to each second object are displayed in the camera preview interface according to the coordinates of the first object and each second object, wherein the size of the first object is larger than the size of each second object.

4. The method according to claim 3, wherein After generating a first tracking focus frame corresponding to the first object according to the first tracking focus frame type, generating a second tracking focus frame corresponding to each second object according to the second tracking focus frame type, and displaying the first tracking focus frame corresponding to the first object and the second tracking focus frames corresponding to each second object in the camera preview interface according to the coordinates of the first object and each second object, the method further includes: If it is determined that the size of any second object is larger than the size of the first object, then any second object is determined as a new first object, and the first object is determined as a new second object, and the focus frame corresponding to the new first object is changed from the second focus frame to the first focus frame, and the focus frame corresponding to the new second object is changed from the first focus frame to the second focus frame.

5. The method according to claim 2, wherein After generating a focus frame corresponding to each object according to the focus frame type of each object, and displaying the focus frame corresponding to each object in the camera preview interface according to the coordinates of each object, the method further includes: When the object type of the object is a portrait ranked in the top N in the non-image library or a non-face object of the preset type, determining a display duration of a focus tracking frame corresponding to the object; When the display time of the focus frame corresponding to the object reaches a time threshold, the focus frame corresponding to the object stops being displayed.

6. The method according to claim 2, wherein The step of determining, based on the first focus tracking priorities of the objects, any one of the objects with the highest first focus tracking priority as the target focus tracking object includes: Determining whether the objects include a first object whose object type is a portrait ranked in the top N in the image library; If the objects include at least one of the first objects, determining the first objects as the objects with the highest first priority, and determining any of the first objects as the target focus tracking object; If the objects do not include the first object, determining whether the objects include a second object whose object type is not a portrait and is ranked in the top N in the image library; If the objects include at least one of the second objects, determining the second object as the object with the highest first priority, and determining any of the second objects as the target tracking focus object; If the objects do not include the second object, determining whether the objects include a third object whose object type is a non-face object of the preset type; If the objects include at least one third object, determining the third object as the object with the highest first priority, and determining any third object as the target tracking focus object; If the objects do not include the third object, center focus processing is performed in the preview interface.

7. The method according to any one of claims 2 to 6, wherein: After determining, based on the first focus tracking priorities of the objects, any one of the objects with the highest first focus tracking priority as a target focus tracking object and performing focus tracking processing on the target focus tracking object, the method further includes: In response to a click operation of the user on any object, determining the any object as a designated focus tracking object; If the designated focus object is the target focus object, the focus process is continued on the designated focus object until a click operation of the user on any of the objects is obtained; If the designated focus object is not the target focus object, the focus processing on the target focus object is suspended, and the focus processing on the designated focus object is continued until a click operation of the user on any of the objects is obtained.

8. The method according to claim 7, wherein After determining any object as a designated focus object in response to the user's click operation on the object, the method further includes: If the designated focus object is the target focus object, and the object type of the designated focus object is a portrait ranked in the top N in the gallery, then the focus frame type of the designated focus object remains unchanged, the focus frame of the designated focus object continues to be displayed, and the focus frames of the objects other than the designated focus object are temporarily displayed; If the designated focus object is not the target focus object, and the object type of the designated focus object is a portrait ranked in the top N in the gallery, then changing the focus frame type of the designated focus object to the first focus frame type, generating and continuously displaying a first focus frame corresponding to the designated focus object based on the first focus frame type, and pausing display of focus frames for each of the objects except the designated focus object; If the object type of the specified focus object is not a portrait ranked in the top N in the gallery, the focus frame type of the specified focus object is changed to the fourth focus frame type, and the fourth focus frame corresponding to the specified focus object is generated and continuously displayed according to the fourth focus frame type, and the focus frames of each of the objects except the specified focus object are temporarily displayed.

9. The method according to claim 7, wherein After determining any object as a designated focus object in response to the user's click operation on the object, the method further includes: In response to the user's click operation on the designated focus object, the focus processing of the designated focus object is paused, and according to the first focus priority of each object, any one of the objects with the highest first focus priority is determined as the target focus object, and the focus processing is performed on the target focus object.

10. The method according to any one of claims 1 to 6, characterized in that: The objects include a plurality of objects with the highest first focus tracking priority, and determining, based on the first focus tracking priorities of the objects, any one of the objects with the highest first focus tracking priority as a target focus tracking object includes: determining a second focus tracking priority for each of the objects having the highest first focus tracking priority; The object with the highest second focus tracking priority is determined as the target focus tracking object.

11. The method according to claim 10, wherein Determining the second focus priority of each of the objects having the highest first focus priority includes: Obtaining the preset priority of each of the objects having the highest priority in the first focus tracking; determining the preset priority of each of the objects as the second focus tracking priority of each of the objects; or, determining the size of each of the objects having the highest priority in the first focus tracking; A second focus tracking priority of each of the objects is determined according to a size of each of the objects.

12. A focus tracking device, characterized in that: include: A first display module, configured to display a camera preview interface in response to a first operation by a user; A first determining module is used to perform recognition processing on each object displayed in the camera preview interface to determine the object type of each object; A second determining module is configured to determine a focus frame type of each of the objects according to the object type of each of the objects; a third determining module, configured to determine a first focus tracking priority for each of the objects based on an object type and a preset priority strategy, wherein the preset priority strategy is set based on a degree of attention the user pays to each of the object types; a second display module, configured to generate a focus frame corresponding to each of the objects according to the focus frame type of each of the objects, and display the focus frame corresponding to each of the objects in the camera preview interface according to the coordinates of each of the objects; The first focusing module is configured to determine, according to the first focusing priority of each of the objects, any one of the objects having the highest first focusing priority as a target focusing object, and perform focusing processing on the target focusing object.

13. A terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 11 is implemented.

14. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 11 is implemented.

Images