Character labeling method and related equipment

Abstract

The embodiment of the invention provides a figure labeling method and related equipment, and the method comprises the steps: responding to a first operation of a user on a photo, and detecting whether the photo comprises a human face or not; if the photo comprises a human face, extracting human face features of the photo; and if the image database of the electronic equipment comprises the portrait photo set corresponding to the face features of the photos, displaying a figure labeling control. And in response to a second operation executed by the user on the character labeling control, displaying a character labeling interface, and receiving character labeling information for the portrait photo collection input by the user through the character labeling interface. According to the embodiment of the invention, when the portrait photo is displayed, the portrait photo set corresponding to the face features in the portrait photo is subjected to character labeling, so that character labeling can be performed on a plurality of photos at the same time, and the efficiency of performing character labeling on the photos is improved.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, which belongs to image processing technology, and in particular to a person labeling method and related equipment. Background Art

[0002] With the development of terminal technology, smart terminal devices such as smartphones and tablet computers with camera functions allow users to record various scenes by taking photos or videos. With the use of smart terminal devices, more and more photos are taken and stored, among which there are also many portrait photos. When users search for photos of specified people in the gallery, it takes a lot of time, which affects the user experience. Summary of the invention

[0003] In view of the above, it is necessary to provide a person tagging method and related equipment to solve the problem that it takes a lot of time to find photos of a specified person in a gallery.

[0004] In a first aspect, the present application provides a method for character annotation, which is applied to an electronic device, and the method includes: responding to a first operation of a user on a first photo, detecting whether the first photo includes a face; if the first photo includes a face, extracting facial features of the first photo; if a gallery database of the electronic device includes a collection of portrait photos corresponding to the facial features of the first photo, displaying a character annotation control; responding to a second operation performed by the user on the character annotation control, displaying a character annotation interface, and receiving character annotation information for the collection of portrait photos input by the user through the character annotation interface.

[0005] Through the above embodiment, when a photo includes a face and a photo collection corresponding to facial features is stored in the gallery database, a character annotation interface can be displayed to receive facial annotation information for the portrait photo collection input by the user through the face annotation interface, thereby annotating multiple photos with character information at the same time, effectively improving the efficiency of character information annotation.

[0006] In one possible implementation, the detecting whether the first photo includes a face includes: detecting whether the first photo has face information; if the first photo does not have face information, detecting whether the first photo has undergone face detection; if the first photo has not undergone face detection, detecting whether the first photo includes a face by using a face detection algorithm; if the first photo has the face information, or the face information is detected by the face detection algorithm, determining that the first photo includes a face; if the first photo does not have the face information and has undergone face detection, or the face information is not detected by the face detection algorithm, determining that the first photo does not include a face.

[0007] According to the above technical solution, when a photo has facial information, it means that the photo has undergone face detection and it is detected that it includes a face. When a photo has undergone face detection and does not have facial information, it means that the photo has undergone face detection and it is detected that it does not include a face. When a photo has not undergone face detection and does not have facial information, a face detection algorithm can be used to detect whether the photo includes a face. This makes it possible to quickly and accurately determine whether the photo includes a face, thereby improving the efficiency of creating a photo collection related to the photo based on the result that the photo includes a face.

[0008] In one possible implementation, detecting whether the first photo includes a face by using a face detection algorithm includes: detecting a face rectangular frame of the first photo by using the face detection algorithm, and if the face rectangular frame is not detected from the first photo, determining that the first photo does not include a face; if the face rectangular frame is detected from the first photo, outputting the face information.

[0009] According to the above technical solution, whether a photo includes a face is determined by whether a face detection algorithm detects a face rectangular frame in the photo. The detection speed is fast, which effectively improves the detection efficiency.

[0010] In a possible implementation, the face information includes the number of faces and the coordinates of at least one vertex on the face rectangular frame.

[0011] According to the above technical solution, the face detection algorithm can be used to quickly detect the number of faces in the photo and the coordinates of at least one vertex on the face rectangular frame, thereby effectively improving the efficiency of acquiring face information.

[0012] In one possible implementation, detecting whether a photo has the facial information includes: obtaining a photo identification of the photo, and searching a gallery database for photo information corresponding to the photo identification according to the photo identification; if the facial information corresponding to the photo identification is found in the gallery database, determining that the photo has the facial information; if the facial information corresponding to the photo identification is not found in the gallery database, determining that the photo does not have the facial information.

[0013] Through the above technical solution, photo information is searched in the gallery database according to the photo identification, which improves the efficiency of searching the photo information, so that it can be quickly determined whether the photo includes facial information.

[0014] In one possible implementation, detecting whether a photo has undergone face detection includes: searching for a detection identifier corresponding to the photo identifier in the gallery database according to the photo identifier; if the detection identifier corresponding to the photo identifier is a first identifier, determining that the photo has undergone face detection; if the detection identifier corresponding to the photo identifier is a second identifier, determining that the photo has not undergone face detection.

[0015] According to the above technical solution, by adding a detection mark to a photo, it is possible to quickly determine whether the photo has undergone face detection.

[0016] In one possible implementation, extracting facial features from the photo includes: inputting the photo into a facial feature extraction model; determining a facial rectangular frame in the photo through the facial feature extraction model; detecting multiple feature points in the facial rectangular frame; obtaining coordinates of the multiple feature points; and outputting the coordinates of the multiple feature points through the facial feature extraction model.

[0017] According to the above technical solution, the coordinates of multiple feature points of the face in the photo are obtained as facial features through a facial feature extraction algorithm, so that the facial features can be accurately extracted and the extraction efficiency of the facial features is improved.

[0018] In a possible implementation, the method also includes: if the gallery database does not include a collection of portrait photos corresponding to the facial features of the first photo, clustering multiple photos in the gallery database according to the facial features in the first photo to obtain multiple second photos associated with the first photo; and creating a photo collection corresponding to the first photo based on the first photo and the multiple second photos.

[0019] According to the above technical solution, multiple photos in the gallery database are clustered by using facial features in the first photo, so that multiple photos similar to the first photo can be determined, thereby improving the efficiency of creating a photo collection.

[0020] In a possible implementation, clustering multiple photos in the gallery database according to facial features in the first photo to obtain multiple second photos associated with the first photo includes: acquiring facial features of other photos in the gallery database except the first photo; calculating the similarity between the facial features of the first photo and the facial features of each other photo; and determining multiple other photos corresponding to similarities greater than or equal to a preset value as the multiple second photos associated with the first photo.

[0021] According to the above technical solution, by calculating the similarity between photos and clustering the photos according to the similarity between the photos, the efficiency and accuracy of photo clustering are effectively improved.

[0022] In a possible implementation, the method also includes: obtaining multiple portrait photos in the gallery database that are not included in the portrait photo collection; clustering the multiple portrait photos obtained to determine the category of each portrait photo; and adding the multiple portrait photos to the photo collection corresponding to the category respectively.

[0023] According to the above technical solution, the efficiency of creating a photo collection is improved by clustering the facial features of multiple photos in the gallery database.

[0024] In a possible implementation, the method further includes: if the number of photos in the portrait photo collection corresponding to the facial features of the first photo is greater than or equal to a first preset number, displaying the character annotation control at the face position of the first photo.

[0025] According to the above technical solution, when there are a large number of photos in the portrait photo collection corresponding to the facial features of the first photo, the person annotation control can be displayed to annotate the person in the first photo, so that the display of the person annotation control better meets user needs.

[0026] In a possible implementation, the method further includes: if the number of newly added photos in the portrait photo collection corresponding to the facial features of the first photo is greater than or equal to a second preset number, displaying the character annotation control at the face position of the first photo.

[0027] According to the above technical solution, when the number of new photos in the portrait photo collection corresponding to the facial features of the first photo is large, the person annotation control can be displayed to annotate the person in the first photo, so that the display of the person annotation control better meets user needs.

[0028] In one possible implementation, the method further includes: if the character annotation control is not triggered within a preset time, or if the gallery application receives a user operation other than triggering the character annotation control, or if the gallery application interface returns to the photo display interface from the character annotation interface, canceling the display of the character annotation control.

[0029] According to the above technical solution, by canceling the display of the character annotation control if the character annotation control is not triggered within a preset time, or if the gallery application receives a user operation other than triggering the character annotation control, or if the gallery application interface returns to the photo display interface from the character annotation interface, the character annotation control can be not displayed when the user does not need to annotate the portrait in the first photo, thereby facilitating the user to view the first photo.

[0030] In a possible implementation, the method also includes: if the first photo includes multiple faces, extracting facial features of the multiple faces; if the gallery database includes a collection of portrait photos corresponding to the facial features of at least one face in the first photo, displaying the character annotation control at the facial position of the at least one face in the first photo.

[0031] According to the above technical scheme, when there are multiple faces in the first photo, facial features of multiple faces can be extracted, and when the gallery database includes a collection of portrait photos corresponding to the facial features of at least one face, a character annotation control can be displayed at the face position of at least one face in the first photo, so as to facilitate users to annotate character information of multiple faces in the first photo.

[0032] In a possible implementation, the method also includes: if the gallery database includes a collection of portrait photos corresponding to the facial features of at least one face in the first photo, determining the number of photos in the collection of portrait photos corresponding to the facial features of the at least one face; and displaying the character annotation control at the facial position of the at least one face in the first photo according to the number of photos in the collection of portrait photos corresponding to the facial features of the at least one face.

[0033] According to the above technical solution, when there are multiple faces in the first photo, the number of photos in the portrait photo collection corresponding to the facial features of at least one face can be determined respectively, and the number of photos in the portrait photo collection corresponding to the facial features of at least one face can be displayed at the face position of at least one face in the first photo, so as to facilitate the user to annotate the character information of the multiple faces in the first photo one by one.

[0034] In a second aspect, the present application provides an electronic device, comprising a memory and a processor: wherein the memory is used to store program instructions; the processor is used to read and execute the program instructions stored in the memory, and when the program instructions are executed by the processor, the electronic device executes the above-mentioned character labeling method.

[0035] In a third aspect, the present application provides a chip coupled to a memory in an electronic device, wherein the chip is used to control a processor of the electronic device to execute the above-mentioned character labeling method.

[0036] In a fourth aspect, the present application provides a computer storage medium storing program instructions, which, when executed on an electronic device, enables the electronic device to execute the above-mentioned character labeling method.

[0037] In addition, the technical effects brought about by the second to fourth aspects can be found in the descriptions related to the methods of each design in the above method part, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a schematic diagram of a gallery application program interface provided in an embodiment of the present application.

[0039] Figure 2It is a schematic diagram of another gallery application program interface provided by an embodiment of the present application.

[0040] Figure 3 It is a schematic diagram of another gallery application program interface provided by an embodiment of the present application.

[0041] Figure 4 It is a software architecture diagram of an electronic device provided in one embodiment of the present application.

[0042] Figure 5 This is a flowchart of a character labeling method provided in one embodiment of the present application.

[0043] Figure 6 It is a schematic diagram of another gallery application program interface provided by an embodiment of the present application.

[0044] Figure 7 It is a schematic diagram of another gallery application program interface provided by an embodiment of the present application.

[0045] Figure 8 This is a flowchart of face detection provided by an embodiment of the present application.

[0046] Fig. 9 This is a flowchart of facial feature extraction provided by an embodiment of the present application.

[0047] Fig.10 This is a flowchart of a character labeling method provided by another embodiment of the present application.

[0048] Fig.11 This is a flowchart of photo clustering provided by an embodiment of the present application.

[0049] Fig.12 This is a flowchart of a character labeling method provided by another embodiment of the present application.

[0050] Fig.13 It is a schematic diagram of another gallery application program interface provided by an embodiment of the present application.

[0051] Fig.14 This is a flowchart of a character labeling method provided by another embodiment of the present application.

[0052] Fig.15 This is a flowchart of a character labeling method provided by another embodiment of the present application.

[0053] Fig.16 This is a hardware architecture diagram of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0054] The terms "first" and "second" involved in an embodiment of the present application are only used for descriptive purposes and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more features. In the description of the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary" or "for example" in an embodiment of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a concrete way.

[0055] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those generally understood by technicians in the technical field in this application. The terms used in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application. It should be understood that, unless otherwise specified in this application, " / " means or. For example, A / B can represent A or B. "And / or" in this application is only a description of the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. "At least one" refers to one or more. "Multiple" refers to two or more than two. For example, at least one of a, b or c can represent: a, b, c, a and b, a and c, b and c, a, b and c. In the absence of conflict, the following embodiments and the features in the embodiments can be combined with each other.

[0056] For the detailed implementation process of the character labeling method, please refer to the description in each embodiment below.

[0057] In order to better understand the character labeling method provided in the embodiment of the present application, Figure 1 , Figure 2 , Figure 3 The application scenario of the character labeling method provided in the embodiment of the present application is described.

[0058] With the development of terminal technology, smart terminal devices such as smartphones and tablet computers with camera functions allow users to record various scenes by taking photos or videos. With the use of smart terminal devices, more and more photos are taken and stored, among which there are also many portrait photos. When users search for photos of specified people in the gallery, it takes a lot of time, which causes inconvenience to users and affects user experience.

[0059] The gallery application of the electronic device can provide a person tagging function, such as tagging the person's name, nickname, relationship with the owner of the electronic device, etc. Figure 1 , which is a schematic diagram of a gallery application interface provided by an embodiment of the present application. When the gallery application interface displays photos, a photo setting control is provided in the upper right corner of the gallery application interface. Figure 2 As shown in the figure, after the user clicks the settings control, the gallery application interface displays the photo settings items in the upper right corner, including the person information editing. Figure 3 As shown in the figure, by clicking the control corresponding to the person information editing, the gallery application displays the person information editing interface, and the user can choose to fill in the person's name and the relationship between the person and the owner in the person information editing interface. However, the current person annotation function can only annotate one photo at a time. When the number of portrait photos is large, person annotation takes a lot of time.

[0060] In order to facilitate users to annotate batches of photos with people, an embodiment of the present application provides a method for annotating people, which can add multiple portrait photos to a corresponding portrait photo collection according to facial features. By annotating the portrait photo collection with people, people can be annotated on multiple photos at the same time.

[0061] See also Figure 4 As shown, it is a software architecture diagram of the electronic device provided in the embodiment of the present application. The layered architecture divides the software into several layers, and each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. For example, the Android system is divided into four layers, from top to bottom, respectively, the application layer 101, the framework layer 102, the Android runtime (Androidruntime) and the system library 103, the hardware abstraction layer 104, the kernel layer 105, and the hardware layer 106.

[0062] The application layer 101 may include a series of application packages, such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message, device control service and other applications.

[0063] The framework layer 102 provides an application programming interface (API) and a programming framework for the application programs of the application layer. The application framework layer includes some predefined functions. For example, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, etc.

[0064] Among them, the window manager is used to manage window programs. The window manager can obtain the size of the display screen, determine whether there is a status bar, lock the screen, capture the screen, etc. The content provider is used to store and obtain data and make the data accessible to the application. The data may include video, images, audio, calls made and received, browsing history and bookmarks, phone books, etc. The view system includes visual controls, such as controls for displaying text, controls for displaying pictures, etc. The view system can be used to build applications. The display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying pictures. The phone manager is used to provide communication functions for electronic devices. For example, the management of call status (including connected, hung up, etc.). The resource manager provides various resources for the application, such as localized strings, icons, pictures, layout files, video files, etc. The notification manager enables the application to display notification information in the status bar, which can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify the completion of downloads, message reminders, etc. The notification manager can also be a notification that appears in the top status bar of the system in the form of an icon or scroll bar text, such as a notification of an application running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is prompted in the status bar, a prompt sound is emitted, an electronic device vibrates, an indicator light flashes, etc.

[0065] Android Runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system. The core library contains two parts: one is the function that the Java language needs to call, and the other is the Android core library.

[0066] The application layer 101 and the framework layer 102 run in a virtual machine. The virtual machine executes the Java files of the application layer and the framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0067] The system library 103 may include multiple functional modules, such as a surface manager, a media library, a 3D graphics processing library (eg, OpenGL ES), a 2D graphics engine (eg, SGL), and the like.

[0068] Among them, the surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications. The media library supports playback and recording of multiple commonly used audio and video formats, as well as static image files. The media library can support multiple audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc. The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis, and layer processing. The 2D graphics engine is a drawing engine for 2D drawing.

[0069] The hardware abstraction layer 104 runs in the user space, encapsulates the kernel layer driver, and provides a calling interface to the upper layer.

[0070] The kernel layer 105 is a layer between hardware and software and includes at least a display driver, a camera driver, an audio driver, and a sensor driver.

[0071] The kernel layer 105 is the core of the operating system of the electronic device. It is the first layer of software expansion based on hardware, provides the most basic functions of the operating system, is the basis of the operating system, is responsible for managing the system's processes, memory, device drivers, files and network systems, and determines the performance and stability of the system. For example, the kernel can determine the operating time of an application on a certain part of the hardware.

[0072] The kernel layer 105 includes programs closely related to the hardware, such as interrupt handlers, device drivers, etc., and also includes basic, common, and high-frequency modules, such as clock management modules, process scheduling modules, etc., and also includes key data structures. The kernel layer can be set in the processor or solidified in the internal memory.

[0073] The hardware layer 106 includes the hardware of the electronic device, such as a display screen, buttons, a camera, etc.

[0074] See also Figure 5 FIG. 1 is a flowchart of a method for labeling characters provided by an embodiment of the present application. The method is applied in an electronic device, and the method for labeling characters includes:

[0075] S101, in response to a first operation of a user on a first photo, detecting whether the first photo includes a face. If the first photo includes a face, executing S102; if the photo does not include a face, the process repeatedly executes S101 until the first photo corresponding to the user operation includes a face.

[0076] In one embodiment of the present application, the first photo may be any photo in a gallery application of an electronic device, or any photo displayed in other applications of the electronic device. If the first photo is any photo in a gallery application of an electronic device, the first operation may be to perform a swipe up operation on the photo in the gallery application. For example, after the user opens the gallery application, the gallery application interface displays thumbnails of some photos in the gallery application, and the user can perform swipe up and swipe down operations on the gallery application interface to switch the display of thumbnails of different photos, and use the thumbnail clicked by the user as the first photo. When the user clicks on the thumbnail of the first photo in the gallery application interface, the gallery application interface enlarges and displays the first photo, and receives the swipe up operation performed by the user on the enlarged first photo.

[0077] In actual applications, the first operation is not limited to the swipe up operation, but may also be any operation such as a swipe down operation, a click operation, a drag operation, or a combination of multiple operations, which is not limited in the embodiments of the present application.

[0078] In one embodiment of the present application, it is detected whether the first photo has face information. If the first photo has face information, it is determined that the first photo has a face. If the first photo does not have face information, it is determined whether the first photo has undergone face detection. If the first photo has undergone face detection, it is determined that the first photo does not include a face. If the first photo has not undergone face detection, a face detection algorithm is called to detect whether the first photo includes a face. If the face information of the first photo is obtained through the face detection algorithm, it is determined that the first photo includes a face. If the face information of the first photo is not detected through the face detection algorithm, it is determined that the first photo does not include a face.

[0079] S102, extracting facial features from the first photo.

[0080] In one embodiment of the present application, facial features in a photo are extracted by a facial feature extraction model. The facial features include, but are not limited to, the position of the face (e.g., the coordinates of the facial parts) and the size of the face, as well as the feature vector of the face. The feature vector is usually composed of multiple feature points. The embodiment of the present application does not limit the facial feature extraction model, which may be a face recognition method based on a convolutional neural network model (CNN), a deep nonlinear face shape extraction model, etc.

[0081] S103, determining whether the gallery database includes a collection of portrait photos corresponding to the facial features of the first photo. If the gallery database includes a collection of portrait photos corresponding to the facial features of the first photo, executing S104; if the gallery database does not include a collection of portrait photos corresponding to the facial features of the first photo, returning to executing S101.

[0082] In one embodiment of the present application, by determining whether the first photo is included in the portrait photo collection in the gallery database, it is determined whether the gallery database includes a portrait photo collection corresponding to the facial features of the first photo. If the photo is included in the photo collection, the photo information corresponding to the photo in the gallery database includes photo collection information, and the photo collection information may be a sequence number or a name of the photo collection. According to the photo identification of the first photo, a search is made in the gallery database to determine whether the portrait photo collection information of the first photo is stored. If the portrait photo collection information of the first photo is stored in the gallery database, it is determined that the gallery database includes a portrait photo collection corresponding to the facial features of the first photo. If the portrait photo collection information of the first photo is not stored in the gallery database, it is determined that the gallery database does not include a portrait photo collection corresponding to the facial features of the first photo.

[0083] S104, displaying a character annotation control.

[0084] See also Figure 6 As shown, in one embodiment of the present application, the gallery application displays a character annotation control at the face position of the first photo in the form of a pop-up window, and the character annotation control includes a prompt, and the content of the prompt includes "Add name".

[0085] In one embodiment of the present application, if the first photo is a normal photo, after determining that the gallery database includes a collection of portrait photos corresponding to the facial features of the first photo, a character annotation control is displayed at the face position of the first photo. If the first photo is a dynamic photo, after determining that the gallery database includes a collection of portrait photos corresponding to the facial features of the first photo, if the dynamic photo is played, the character annotation control is displayed at the face position of the first photo, and if the dynamic photo is not played, the character annotation control is displayed at the face position of the first photo after the dynamic photo is played.

[0086] S105, in response to a second operation performed by the user on the character annotation control, displaying a character annotation interface, and receiving character annotation information for the portrait photo collection input by the user through the character annotation interface.

[0087] In one embodiment of the present application, the character annotation information includes a character name and a character relationship. The character name may be a name or nickname of the character, and the character relationship is a relationship between the character and the owner of the electronic device. The second operation may be a click operation performed by the user on the character annotation control. When it is detected that the user performs a click operation on the character annotation control, the following is displayed: Figure 7 The character tagging interface shown includes a cover selection option for the photo collection where the first photo is located, an input box for the character's name, and multiple options for character relationships, for users to enter the name of the character in the photo and select the character relationship.

[0088] In one embodiment of the present application, the character annotation method further includes: if the character annotation control is not triggered within a preset time, or if the gallery application receives a user operation other than triggering the character annotation control, or if the gallery application interface returns to the photo display interface from the character annotation interface, canceling the display of the character annotation control. For example, the preset time is 3, 4, or 5 seconds.

[0089] In one embodiment of the present application, the character annotation method also includes: if the first photo includes facial features of multiple faces, determining whether the gallery database includes a collection of portrait photos corresponding to the facial features of each face in the first photo; if the gallery database includes a collection of portrait photos corresponding to the facial features of at least one face in the first photo, displaying a character annotation control at the facial position of at least one face in the first photo.

[0090] In one embodiment of the present application, the person annotation method further includes: if the first photo includes facial features of multiple faces, determining whether the gallery database includes a collection of portrait photos corresponding to the facial features of each face in the first photo, if the gallery database includes a collection of portrait photos corresponding to the facial features of at least one face in the first photo, determining the number of photos in the collection of portrait photos corresponding to the facial features of at least one face, and displaying a person annotation control at the position of at least one face in the first photo according to the number of photos in the collection of portrait photos corresponding to the facial features of at least one face. The person annotation control is displayed at the position of the face with the largest number of photos in the collection of portrait photos of the first photo. After the face with the largest number of photos in the collection of portrait photos completes the person information annotation, the person annotation control is displayed at the position of the face with the second largest number of photos in the collection of portrait photos of the first photo, and so on, until all faces have completed the person information annotation.

[0091] In one embodiment of the present application, S101-S103 are executed by AiPluginEngine (artificial intelligence plug-in engine) of the framework layer calling relevant algorithms, and S104-S105 are executed by the gallery application of the application layer.

[0092] Through the above embodiment, when a photo includes a face and a photo collection corresponding to facial features is stored in the gallery database, a character annotation interface can be displayed to receive facial annotation information for the portrait photo collection input by the user through the face annotation interface, thereby annotating multiple photos with character information at the same time, effectively improving the efficiency of character information annotation.

[0093] See also Figure 8 , which is a flow chart of face detection provided by an embodiment of the present application.

[0094] S201, detecting whether the first photo has face information. If the first photo has face information, executing S204; if the first photo does not have face information, executing S202.

[0095] In one embodiment of the present application, the memory of the electronic device includes a gallery database for storing photos and corresponding photo information. The gallery application is a display window of the gallery database. The photo information includes, but is not limited to: photo identification and facial information of the photo. When each photo in the gallery database is taken by a camera and stored in the gallery database, a unique identifier ID (Identifier) ​​of the corresponding photo is automatically generated as the photo identification of the corresponding photo. In one embodiment of the present application, the photo identification can be generated according to the shooting time and shooting order of the photo. For example, the photo identification is "202306181", wherein "20230618" means that the shooting time of the photo is June 18, 2023, and "1" means that the photo is the first photo taken on that date. In other embodiments of the present application, the photo identification can also be a shooting sequence number determined according to the shooting order or a randomly generated identification consisting of numbers or letters + numbers.

[0096] In one embodiment of the present application, when a first operation performed by a user on a first photo is received, in response to the first operation, a photo identifier of the first photo is obtained, and photo information corresponding to the photo identifier is searched in a gallery database according to the photo identifier. If face information corresponding to the photo identifier is found in the gallery database, it is determined that the first photo has face information; if face information corresponding to the photo identifier is not found in the gallery database, it is determined that the first photo does not have face information.

[0097] S202, detecting whether the first photo has been subjected to face detection. If the first photo has been subjected to face detection, executing S205; if the first photo has not been subjected to face detection, executing S203.

[0098] In one embodiment of the present application, the gallery database also stores detection identification of photos, and the detection identification may include two identifications: a first identification and a second identification. For example, the first identification indicates that the photo has been detected, which is represented by 1, and the second identification indicates that the photo has not been detected, which is represented by 0. If the detection identification of the first photo is detected to be 0, it means that the photo has not been detected. If the detection identification of the first photo is detected to be 1, it means that the photo has been detected. Continuing from the above embodiment, if the first photo does not have facial information, the detection identification corresponding to the photo identification can be searched in the gallery database according to the photo identification. If the detection identification corresponding to the photo identification found in the gallery database is the first identification, that is, 1, it is determined that the first photo has been detected by the face. If the detection identification corresponding to the photo identification found in the gallery database is the second identification, that is, 0, it is determined that the first photo has not been detected by the face.

[0099] In another embodiment of the present application, the detection mark of the photo may also be included in the photo mark, for example, located in the first position of the photo mark. If the first photo does not have facial information, the first digit value of the photo mark can be obtained. If the first digit value of the photo mark is 1, the detection mark corresponding to the photo mark is 1, and it is determined that the first photo has undergone face detection. If the first digit value of the photo mark is 0, the detection mark corresponding to the photo mark is 0, and it is determined that the first photo has not undergone face detection. For example, if the photo mark is "0202306181" and the first digit is 0, it means that the photo has not undergone face detection.

[0100] S203, detecting whether the first photo includes a face by using a face detection algorithm. If face information is obtained by detecting the face detection algorithm, executing S204; if face information is not detected by the face detection algorithm, executing S205.

[0101] In one embodiment of the present application, the face detection algorithm is a convolutional neural network model, and the convolutional neural network model corresponding to the face detection algorithm is trained and generated with a face rectangular frame as a feature, and the face rectangular frame is the smallest rectangular frame that includes multiple parts of the face, and the multiple parts may include eyebrows, eyes, nose, and mouth. In other embodiments of the present application, the multiple parts may also include facial features.

[0102] In one embodiment of the present application, a thumbnail of the first photo is input into a model corresponding to a face detection algorithm, and a face rectangular frame of the first photo is extracted by the face detection algorithm. In another embodiment of the present application, the first photo is input into a model corresponding to a face detection algorithm, the size of the first photo is converted into a preset size (e.g., 1200*800) by the face detection algorithm, the first photo is grayed, and the face rectangular frame of the first photo is detected. If the face rectangular frame of the first photo is not detected, it is determined that the first photo does not include a face. If the face rectangular frame of the first photo is detected, the face information is output. Among them, the face information includes but is not limited to the number of faces and the coordinates of at least one vertex on the face rectangular frame. The number of faces is the same as the number of rectangular frames. The coordinates of at least one vertex on the rectangular frame may include the coordinates of the upper left vertex and the lower right vertex of the rectangular frame, and the coordinates of the vertex of the rectangular frame are pixel coordinates. For example, the face information output by the face detection algorithm is "2, target1rect(500*40, 600*80), target2rect(700*200, 800*240)".

[0103] In other embodiments of the present application, the face detection algorithm may also be a cascade classifier based on Haar features, a Dlib face detection algorithm, or the like.

[0104] S204: Determine whether the first photo includes a face.

[0105] S205: Determine that the first photo does not include a face.

[0106] Through the above embodiments, when a photo has facial information, it means that the photo has undergone face detection and it is detected that it includes a face. When a photo has undergone face detection and does not have facial information, it means that the photo has undergone face detection and it is detected that it does not include a face. When a photo has not undergone face detection and does not have facial information, a face detection algorithm can be used to detect whether the photo includes a face, thereby quickly and accurately determining whether the photo includes a face, thereby improving the efficiency of creating a photo collection related to the photo based on the result that the photo includes a face.

[0107] See also Fig. 9 , which is a flowchart of facial feature extraction provided by an embodiment of the present application.

[0108] S301, determining a face rectangular frame in a first photo by using a face feature extraction algorithm.

[0109] In one embodiment of the present application, the face feature extraction algorithm may also be a convolutional neural network model, and the convolutional neural network model corresponding to the face feature extraction algorithm is generated by training with face rectangular frames, face part rectangular frames, and face feature point positions as features. In other embodiments of the present application, the face feature extraction algorithm may also be a cascade classifier based on Haar features, a Dlib face detection algorithm, and the like.

[0110] In one embodiment of the present application, the size of the first photo is converted to a preset size (e.g., 800*600) by a facial feature extraction algorithm, and the first photo is grayscaled, and then the face rectangular frame in the first photo is determined. In another embodiment of the present application, the thumbnail of the first photo can also be input into a model corresponding to the facial feature extraction algorithm. During the grayscale processing, the RGB value of each pixel is combined into a single grayscale value. Common grayscale methods include component method, maximum value method, and average value method, etc., which are not limited by the present application.

[0111] S302, detecting multiple feature points in a face rectangular frame.

[0112] In one embodiment of the present application, the facial feature extraction algorithm detects multiple feature points in the face rectangular frame through a key point detector. For example, the multiple feature points include the beginning and end of the two eyes and the beginning of the nose.

[0113] S303: extracting coordinates of multiple feature points.

[0114] In one embodiment of the present application, the pixel coordinates of the plurality of feature points in the first photo are determined by a facial feature extraction algorithm.

[0115] S304, outputting coordinates of multiple feature points through a facial feature extraction algorithm.

[0116] In one embodiment of the present application, the facial feature extraction algorithm outputs feature vectors corresponding to the coordinates of multiple feature points in float values, for example, the feature vector is points[(94, 238), (98, 271), (105, 303), (113, 334), (125, 362)]. If the first photo includes multiple faces, the feature vector includes target1, points[(100, 200), (50, 281), (60, 305), (156, 402), (95, 320)]; target2, points[(94, 238), (98, 271), (105, 303), (113, 334), (125, 362)].

[0117] Through the above embodiment, the coordinates of multiple feature points of the face in the photo are obtained as facial features through the facial feature extraction algorithm, so that the facial features can be accurately extracted and the extraction efficiency of the facial features is improved.

[0118] See also Fig.10 FIG. 1 is a flowchart of a method for labeling characters provided by another embodiment of the present application. The method is applied in an electronic device, and the method for labeling characters includes:

[0119] S401, in response to a first operation of a user on a first photo, detecting whether the first photo includes a face. If the first photo includes a face, executing S404; if the photo does not include a face, the process repeatedly executes S401 until the first photo includes a face.

[0120] S402, extracting facial features from the first photo.

[0121] In one embodiment of the present application, facial features in a photo are extracted by a facial feature extraction model. The facial features include, but are not limited to, the position of the face (e.g., the coordinates of the facial parts) and the size of the face, as well as the feature vector of the face. The feature vector is usually composed of multiple feature points. The embodiment of the present application does not limit the facial feature extraction model, which may be a face recognition method based on a convolutional neural network model (CNN), a deep nonlinear face shape extraction model, etc.

[0122] S403, determining whether the gallery database includes a collection of portrait photos corresponding to the facial features of the first photo. If the gallery database includes a collection of portrait photos corresponding to the facial features of the first photo, executing S404; if the gallery database does not include a collection of portrait photos corresponding to the facial features of the first photo, returning to executing S406.

[0123] S404, displaying a character annotation control.

[0124] S405 , in response to a second operation performed by the user on the character annotation control, displaying a character annotation interface, and receiving character annotation information for the portrait photo collection input by the user through the character annotation interface.

[0125] In one embodiment of the present application, the character annotation information includes a character name and a character relationship. The character name may be a name or nickname of the character, and the character relationship is a relationship between the character and the owner of the electronic device. The second operation may be a click operation performed by the user on the character annotation control. When it is detected that the user performs a click operation on the character annotation control, the following is displayed: Figure 7 The character tagging interface shown includes a cover selection option for the photo collection where the first photo is located, an input box for the character's name, and multiple options for character relationships, for users to enter the name of the character in the photo and select the character relationship.

[0126] S406: Cluster multiple photos in the gallery database according to the facial features in the first photo to obtain multiple second photos associated with the first photo.

[0127] In one embodiment of the present application, multiple photos in the gallery database are clustered by a clustering algorithm based on the facial features in the first photo. The clustering algorithm can group similar photos together, making it easier for users to find common points between photos and facilitate users to view similar photos. For example, clustering algorithms may include: K-Means, Spectral, DBSCAN, Rank Order, etc. and variants of each clustering algorithm. The embodiments of the present application do not limit the specific clustering algorithm used. By clustering multiple photos, multiple second photos associated with the first photo can be obtained.

[0128] S407: Create a photo collection corresponding to the first photo according to the first photo and the plurality of second photos.

[0129] In one embodiment of the present application, the first photo and the plurality of second photos are packaged into a folder and stored in a gallery database to create a photo collection corresponding to the first photo.

[0130] See also Fig.11 As shown, it is a flowchart of photo clustering provided by an embodiment of the present application.

[0131] S501, obtaining facial features of other photos except the first photo in the gallery database.

[0132] In one embodiment of the present application, the facial information of the photos stored in the gallery database also includes facial features. The facial information of other photos in the gallery database except the first photo is queried. If the other photos have facial information, the corresponding facial features can be directly obtained. If any of the other photos does not have facial information, the detection mark of the photo is queried. If it is determined that any of the photos has undergone face detection based on the detection mark, there is no need to obtain the facial features of the photo. If it is determined that any of the photos has not undergone face detection based on the detection mark, the facial features of the photo are used. Figure 8 The process shown in the figure is to perform face detection on any photo through the face detection algorithm. After the detection, if it is determined that any photo contains a face, use Fig. 9 In the process shown, the facial features of any photo are extracted by a facial feature extraction algorithm. When it is determined that any photo does not include a face, there is no need to obtain a detection identifier of the photo.

[0133] S502, obtaining facial features of the first photo and facial features of other photos through a clustering algorithm.

[0134] In one embodiment of the present application, the clustering algorithm is a similarity detection algorithm, and the similarity of facial features between different photos can be calculated by the similarity detection algorithm.

[0135] S503: Calculate the similarity between the facial features of the first photo and the facial features of each other photo by using a clustering algorithm.

[0136] In one embodiment of the present application, the similarity between the facial features of the first photo and the facial features of each other photo is the Euclidean distance. The calculation formula of the Euclidean distance is:

[0137]

[0138] In the calculation formula (1), A i is the i-th feature of the face feature of the first photo, the i-th feature of the first photo is the i-th feature point coordinate of the first photo, B i is the i-th feature of the face features of other photos, and the i-th feature of other photos is the i-th feature point coordinates of other photos.

[0139] In another embodiment of the present application, the similarity between the facial features of the first photo and the facial features of each other photo is cosine similarity. The calculation formula of cosine similarity is:

[0140]

[0141] In the calculation formula (2), A iis the i-th feature of the face feature of the first photo, the i-th feature of the first photo is the i-th feature point coordinate of the first photo, B i is the i-th feature of the face features of other photos, and the i-th feature of other photos is the i-th feature point coordinates of other photos.

[0142] S504: Determine a plurality of other photos corresponding to similarities greater than or equal to a preset value as a plurality of second photos associated with the first photo.

[0143] In one embodiment of the present application, the similarity between the facial features of the first photo and the facial features of each other photo is compared with a preset value to determine whether the similarity is greater than or equal to the preset value. If the similarity is greater than or equal to the preset value, the other photos corresponding to the similarity are determined as second photos associated with the first photo, thereby determining multiple other photos corresponding to similarities greater than or equal to the preset value as multiple second photos associated with the first photo.

[0144] According to the above embodiment, by calculating the similarity between photos and clustering the photos according to the similarity between the photos, the efficiency and accuracy of photo clustering are effectively improved.

[0145] See also Fig.12 FIG. 1 is a flowchart of a method for labeling characters provided by another embodiment of the present application. The method is applied in an electronic device, and the method for labeling characters includes:

[0146] S601, obtaining a plurality of portrait photos that are not included in the portrait photo collection from a gallery database.

[0147] In one embodiment of the present application, multiple portrait photos that are not included in the portrait photo collection can be obtained in the gallery database when the electronic device is in a preset state. In one embodiment of the present application, the preset state includes charging with the screen off, the power is greater than or equal to the preset power, and the temperature is less than or equal to the preset temperature. Among them, the preset power is a percentage of power, for example, 50%, 60%, 70% or other percentage values, and the preset temperature is 30°C, 25°C, 20°C or other temperature values. The present application does not limit the specific content of the preset state. In other embodiments of the present application, the preset state may include charging with the screen off and the power is greater than or equal to the preset power, or the preset state may only include charging with the screen off.

[0148] In one embodiment of the present application, the status listener is a function for monitoring the status of an electronic device. For example, the status of an electronic device may include battery power, temperature, network connection status, memory usage, etc. The status listener may communicate with the power management service, display management service, and temperature sensor of the operating system to monitor the status parameters of the electronic device such as the screen on and off status, charging status, power level, and temperature. For example, if the status listener determines that the electronic device is in a state of charging with the screen off, the power level is greater than or equal to the preset power level, and the temperature is less than or equal to the preset temperature, a generation instruction for a photo collection is generated, and the moment service responds to the generation instruction for the photo collection to obtain multiple first photos that are not included in the photo collection from the gallery database. In other embodiments of the present application, the status listener may also be a tool for monitoring the status of a device.

[0149] In one embodiment of the present application, when the electronic device is in a preset state, Figure 8 The process shown obtains portrait photos (i.e., photos including faces) from the gallery database of the electronic device, and determines whether the gallery of the electronic device includes a portrait photo collection. If the gallery application does not include a portrait photo collection, all portrait photos in the gallery are not included in the portrait photo collection. Therefore, all portrait photos in the gallery application can be obtained. If the gallery application includes a portrait photo collection, the portrait photos that are not stored in the folder corresponding to the portrait photo collection in the gallery application are photos that are not included in the portrait photo collection. Therefore, the portrait photos that are not stored in the folder corresponding to the portrait photo collection can be obtained.

[0150] In another embodiment of the present application, the gallery database stores photo identifications and photo information of photos. If a portrait photo is included in a portrait photo collection, the photo information of the portrait photo includes portrait photo collection information. For example, the portrait photo collection information may be the serial number of the portrait photo collection or the subject name of the portrait photo collection. It can be determined whether the photo information of each portrait photo in the gallery database includes the portrait photo collection information. If the photo information of the portrait photo includes the portrait photo collection information, it is determined that the photo is included in the portrait photo collection. If the photo information of the portrait photo does not include the portrait photo collection information, it is determined that the portrait photo is not included in the portrait photo collection. Therefore, multiple portrait photos that are not included in the portrait photo collection can be obtained by obtaining portrait photos whose photo information does not include the portrait photo collection information.

[0151] S602: Cluster the acquired multiple portrait photos to determine the category of each portrait photo.

[0152] In one embodiment of the present application, a plurality of portrait photos acquired are clustered by a preset clustering algorithm. The preset clustering algorithm may be a K-means clustering algorithm. If a collection of portrait photos is not included in the gallery application, the number of categories K of the plurality of portrait photos is set, and K initial clustering centers are randomly selected. The clustering center may be the facial features of any portrait photo, and the distance (e.g., Euclidean distance) between the facial features of each portrait photo and each clustering center is calculated, and the portrait photo is assigned to the category with the smallest distance. Then, the K clustering centers are updated by calculating the average value of the facial features in each category, and the distance between the facial features of each portrait photo and each updated clustering center is calculated, and the portrait photo is assigned to the category with the smallest distance. Then, the above steps are repeated until the K-means clustering algorithm converges.

[0153] In another embodiment of the present application, if the gallery includes a collection of portrait photos, the cluster center of each portrait photo collection is calculated by a preset clustering algorithm (the cluster center is the average value of the facial features of all portrait photos in the portrait photo collection), the distance between each photo and the cluster center of each portrait photo collection is calculated, and the category of the photo is determined as the category corresponding to the cluster center with the shortest distance.

[0154] S603: Add the multiple portrait photos to photo collections corresponding to the categories of the portrait photos respectively.

[0155] S604, in response to the user's first operation on the first photo, detecting whether the first photo includes a face. If the first photo includes a face, executing S605; if the first photo does not include a face, the process repeats executing S604.

[0156] S605: Extract facial features from the first photo.

[0157] S606, determining whether the gallery database includes a collection of portrait photos corresponding to the facial features of the first photo. If the gallery database includes a collection of portrait photos corresponding to the facial features of the first photo, executing S607; if the gallery database does not include a collection of portrait photos corresponding to the facial features of the first photo, returning to executing S604.

[0158] S607, displaying a person annotation control at the face position of the first photo.

[0159] S608, in response to a second operation performed by the user on the character annotation control, displaying a character annotation interface, and receiving character annotation information for the portrait photo collection input by the user through the character annotation interface.

[0160] In one embodiment of the present application, after receiving the person annotation information of the portrait photo collection input by the user through the person annotation interface, the person name in the person annotation information can be displayed on the cover of the portrait photo collection. The person name on the cover of the portrait photo collection is a control. By triggering the control, the gallery application displays the following Figure 7 The character information editing interface shown in the figure, users can update the character information and the cover of the character photo collection in the character information editing interface. Fig.13 As shown in FIG. 1 , in the creation interface of the gallery application, a portrait photo collection is displayed, and the cover of the portrait photo collection displays the name of the person corresponding to the portrait photo collection. If the portrait photo collection is not labeled with person information, a control for adding a name can be displayed on the cover, and a second operation performed by the user on the control for adding a name is received, and the gallery application displays the following Figure 7 The character information editing interface shown is where the user can input character information and update the cover of the character photo collection.

[0161] See also Fig.14 FIG. 1 is a flowchart of a method for labeling characters provided by another embodiment of the present application. The method is applied in an electronic device, and the method for labeling characters includes:

[0162] S701, obtaining a plurality of portrait photos that are not included in the portrait photo collection from a gallery database.

[0163] S702: Cluster the acquired multiple portrait photos to determine the category of each portrait photo.

[0164] S703, adding the multiple portrait photos to the photo collection corresponding to the category respectively.

[0165] S704, in response to the user's first operation on the first photo, detecting whether the first photo includes a face. If the first photo includes a face, executing S705; if the first photo does not include a face, the process repeats executing S704.

[0166] S705, extracting facial features from the first photo.

[0167] S706, determining whether the gallery database includes a collection of portrait photos corresponding to the facial features of the first photo. If the gallery database includes a collection of portrait photos corresponding to the facial features of the first photo, executing S707; if the gallery database does not include a collection of portrait photos corresponding to the facial features of the first photo, returning to executing S704.

[0168] S707, determining whether the number of photos in the portrait photo collection corresponding to the facial features of the first photo is greater than or equal to a first preset number. If the number of photos in the portrait photo collection corresponding to the facial features of the first photo is greater than or equal to the first preset number, executing S708; if the number of photos in the portrait photo collection corresponding to the facial features of the first photo is greater than or equal to the first preset number, returning to execute S704.

[0169] S708, displaying a person annotation control at the face position of the first photo.

[0170] S709, in response to a second operation performed by the user on the character annotation control, displaying a character annotation interface, and receiving character annotation information for the portrait photo collection input by the user through the character annotation interface.

[0171] See also Fig.15 FIG. 1 is a flowchart of a method for labeling characters provided by another embodiment of the present application. The method is applied in an electronic device, and the method for labeling characters includes:

[0172] S801, obtaining a plurality of portrait photos that are not included in the portrait photo collection from a gallery database.

[0173] S802: Cluster the acquired multiple portrait photos to determine the category of each portrait photo.

[0174] S803, adding the multiple portrait photos to the photo collection corresponding to the category respectively.

[0175] S804, in response to the user's first operation on the first photo, detecting whether the first photo includes a face. If the first photo includes a face, executing S805; if the first photo does not include a face, the process repeats executing S804.

[0176] S805, extracting facial features from the first photo.

[0177] S806, determining whether the gallery database includes a collection of portrait photos corresponding to the facial features of the first photo. If the gallery database includes a collection of portrait photos corresponding to the facial features of the first photo, executing S807; if the gallery database does not include a collection of portrait photos corresponding to the facial features of the first photo, returning to executing S804.

[0178] S807, determining whether the number of newly added photos in the portrait photo collection corresponding to the facial features of the first photo is greater than or equal to the second preset number. If the number of newly added photos in the portrait photo collection corresponding to the facial features of the first photo is greater than or equal to the second preset number, executing S808; if the number of newly added photos in the portrait photo collection corresponding to the facial features of the first photo is greater than or equal to the second preset number, returning to execute S804.

[0179] In one embodiment of the present application, the number of newly added photos in the portrait photo collection is the number of photos added to the portrait photo collection most recently.

[0180] S808, displaying a person annotation control at the face position of the first photo.

[0181] S809, in response to a second operation performed by the user on the character annotation control, displaying a character annotation interface, and receiving character annotation information for the portrait photo collection input by the user through the character annotation interface.

[0182] The present application also provides an electronic device 100. Fig.16 As shown, the electronic device 100 can be a mobile phone, a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a cellular phone, a personal digital assistant (PDA), an augmented reality (AR) device, a virtual reality (VR) device, an artificial intelligence (AI) device, a wearable device, a vehicle-mounted device, a smart home device and / or a smart city device. The embodiments of the present application do not impose any special restrictions on the specific type of the electronic device 100.

[0183] The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a Universal Serial Bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a Subscriber Identification Module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0184] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0185] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0186] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0187] The processor 110 may also be provided with a memory for storing instructions and data. In one embodiment of the present application, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use instructions or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0188] In one embodiment of the present application, the processor 110 may include one or more interfaces. The interface may include an Inter-integrated Circuit (I2C) interface, an Inter-integrated Circuit Sound (I2S) interface, a Pulse Code Modulation (PCM) interface, a Universal Asynchronous Receiver / Transmitter (UART) interface, a Mobile Industry Processor Interface (MIPI), a General-Purpose Input / Output (GPIO) interface, a Subscriber Identity Module (SIM) interface, and / or a Universal Serial Bus (USB) interface, etc.

[0189] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In one embodiment of the present application, the processor 110 may include multiple groups of I2C buses. The processor 110 can be coupled to the touch sensor 180K, the charger, the flash, the camera 193, etc. through different I2C bus interfaces. For example: the processor 110 can be coupled to the touch sensor 180K through the I2C interface, so that the processor 110 communicates with the touch sensor 180K through the I2C bus interface to realize the touch function of the electronic device 100.

[0190] The I2S interface can be used for audio communication. In one embodiment of the present application, the processor 110 may include multiple I2S buses. The processor 110 may be coupled to the audio module 170 via the I2S bus to achieve communication between the processor 110 and the audio module 170. In one embodiment of the present application, the audio module 170 may transmit an audio signal to the wireless communication module 160 via the I2S interface to achieve the function of answering a call via a Bluetooth headset.

[0191] The PCM interface can also be used for audio communication, sampling, quantizing and encoding analog signals. In one embodiment of the present application, the audio module 170 and the wireless communication module 160 can be coupled via a PCM bus interface. In one embodiment of the present application, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface to realize the function of answering a call via a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0192] The UART interface is a universal serial data bus for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In one embodiment of the present application, the UART interface is generally used to connect the processor 110 and the wireless communication module 160. For example: the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to implement the Bluetooth function. In one embodiment of the present application, the audio module 170 can transmit an audio signal to the wireless communication module 160 through the UART interface to implement the function of playing music through a Bluetooth headset.

[0193] The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), etc. In one embodiment of the present application, the processor 110 and the camera 193 communicate via the CSI interface to realize the shooting function of the electronic device 100. The processor 110 and the display screen 194 communicate via the DSI interface to realize the display function of the electronic device 100.

[0194] The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In one embodiment of the present application, the GPIO interface can be used to connect the processor 110 with the camera 193, the display screen 194, the wireless communication module 160, the audio module 170, the sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0195] The USB interface 130 is an interface that complies with the USB standard specification, and can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used to transfer data between the electronic device 100 and a peripheral device. It can also be used to connect headphones to play audio through the headphones. The interface can also be used to connect other electronic devices 100, such as AR devices, etc.

[0196] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present invention is only a schematic illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0197] The charging management module 140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the electronic device 100. While the charging management module 140 is charging the battery 142, it may also power the electronic device 100 through the power management module 141.

[0198] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0199] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0200] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0201] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (Low Noise Amplifier, LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In one embodiment of the present application, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In one embodiment of the present application, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0202] The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a receiver 170B, etc.), or displays an image or video through a display screen 194. In one embodiment of the present application, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be arranged in the same device as the mobile communication module 150 or other functional modules.

[0203] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) network), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0204] In one embodiment of the present application, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include Global System For Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time-Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a Global Positioning System (GPS), a Global Navigation Satellite System (GLONASS), a Beidou Navigation Satellite System (BDS), a Quasi-Zenith Satellite System (QZSS) and / or a Satellite Based Augmentation System (SBAS).

[0205] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for character annotation, connecting the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0206] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active matrix organic light-emitting diode or an active matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, Microled, Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In one embodiment of the present application, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0207] The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0208] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In one embodiment of the present application, ISP can be set in camera 193.

[0209] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In one embodiment of the present application, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0210] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the electronic device 100 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0211] The video codec is used to compress or decompress digital video. The electronic device 100 may support one or more video codecs. In this way, the electronic device 100 can play or record videos in multiple coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0212] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of the electronic device 100 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0213] The internal memory 121 may include one or more random access memories (RAM) and one or more non-volatile memories (NVM).

[0214] Random access memory may include static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM, for example, the fifth generation DDR SDRAM is generally referred to as DDR5 SDRAM), etc.;

[0215] Non-volatile memory may include disk storage devices and flash memory.

[0216] Flash memory can be divided into NOR FLASH, NAND FLASH, 3D NAND FLASH, etc. according to the operating principle. It can be divided into single-level cell (SLC), multi-level cell (MLC), triple-level cell (TLC), quad-level cell (QLC), etc. according to the storage unit potential level. It can be divided into universal flash storage (UFS), embedded Multi Media Card (eMMC), etc. according to the storage specification.

[0217] The random access memory can be directly read and written by the processor 110, and can be used to store executable programs (such as machine instructions) of the operating system or other running programs, and can also be used to store user and application data.

[0218] The non-volatile memory may also store executable programs and user and application data, etc., and may be loaded into the random access memory in advance for direct reading and writing by the processor 110 .

[0219] The external memory interface 120 can be used to connect to an external non-volatile memory to expand the storage capacity of the electronic device 100. The external non-volatile memory communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, files such as music and videos are stored in the external non-volatile memory.

[0220] The internal memory 121 or the external memory interface 120 is used to store one or more computer programs. The one or more computer programs are configured to be executed by the processor 110. The one or more computer programs include multiple instructions. When the multiple instructions are executed by the processor 110, the screen display detection method executed on the electronic device 100 in the above embodiment can be implemented to implement the screen display detection function of the electronic device 100.

[0221] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0222] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In one embodiment of the present application, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110.

[0223] The speaker 170A, also called a "speaker", is used to convert an audio electrical signal into a sound signal. The electronic device 100 can listen to music or listen to a hands-free call through the speaker 170A.

[0224] The receiver 170B, also called a "earpiece", is used to convert audio electrical signals into sound signals. When the electronic device 100 receives a call or voice message, the voice can be received by placing the receiver 170B close to the human ear.

[0225] Microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to microphone 170C to input the sound signal into microphone 170C. The electronic device 100 can be provided with at least one microphone 170C. In other embodiments, the electronic device 100 can be provided with two microphones 170C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the electronic device 100 can also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify the sound source, realize directional recording function, etc.

[0226] The earphone interface 170D is used to connect a wired earphone and can be a USB interface 130 or a 3.5 mm Open Mobile Terminal Platform (OMTP) standard interface or a Cellular Telecommunications Industry Association of the USA (CTIA) standard interface.

[0227] The key 190 includes a power key, a volume key, etc. The key 190 may be a mechanical key or a touch key. The electronic device 100 may receive key input and generate key signal input related to user settings and function control of the electronic device 100.

[0228] Motor 191 can generate vibration prompts. Motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. For touch operations acting on different areas of the display screen 194, motor 191 can also correspond to different vibration feedback effects. Different application scenarios (for example: time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0229] Indicator 192 may be an indicator light, which may be used to indicate charging status, power changes, messages, missed calls, notifications, etc.

[0230] The SIM card interface 195 is used to connect a SIM card. The SIM card can be connected to and separated from the electronic device 100 by inserting it into the SIM card interface 195 or pulling it out from the SIM card interface 195. The electronic device 100 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, and the like. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the multiple cards can be the same or different. The SIM card interface 195 can also be compatible with different types of SIM cards. The SIM card interface 195 can also be compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to implement functions such as calls and data communications. In one embodiment of the present application, the electronic device 100 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100. The embodiment of the present application also provides a computer storage medium, in which computer instructions are stored. When the computer instructions are executed on the electronic device 100, the electronic device 100 executes the above-mentioned related method steps to implement the character labeling method in the above-mentioned embodiment.

[0231] The embodiment of the present application also provides a computer program product. When the computer program product is run on a computer, the computer is caused to execute the above-mentioned related steps to implement the character labeling method in the above-mentioned embodiment.

[0232] In addition, an embodiment of the present application also provides a device, which can specifically be a chip, component or module, and the device may include a connected processor and memory; wherein the memory is used to store computer execution instructions, and when the device is running, the processor can execute the computer execution instructions stored in the memory so that the chip executes the character labeling method in the above-mentioned method embodiments.

[0233] Among them, the electronic device, computer storage medium, computer program product or chip provided in this embodiment is used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above and will not be repeated here.

[0234] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0235] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0236] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0237] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0238] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the software product is stored in a storage medium, including a number of instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (RandomAccess Memory, RAM), disk or optical disk and other media that can store program code.

[0239] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present application and are not intended to limit it. Although the present application has been described in detail with reference to the preferred embodiments, a person of ordinary skill in the art should understand that the technical solution of the present application may be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present application.

Claims

1. A person tagging method, applied to electronic equipment, characterized in that: The method comprises: In response to a first operation of a user on a first photo, detecting whether the first photo includes a face; If the first photo includes a face, extracting facial features of the first photo; If the gallery database of the electronic device includes a collection of portrait photos corresponding to the facial features of the first photo, displaying a person annotation control; In response to a second operation performed by the user on the character annotation control, a character annotation interface is displayed, and character annotation information for the portrait photo collection input by the user through the character annotation interface is received.

2. The character labeling method according to claim 1, characterized in that: The detecting whether the first photo includes a face includes: Detecting whether the first photo has facial information; If the first photo does not have face information, detecting whether the first photo has been subjected to face detection; If the first photo has not been subjected to face detection, detecting whether the first photo includes a face by using a face detection algorithm; If the first photo has the face information, or the face information is detected by the face detection algorithm, it is determined that the first photo includes a face; If the first photo does not have the facial information and has been subjected to face detection, or the facial information is not detected by the face detection algorithm, it is determined that the first photo does not include a face.

3. The character labeling method according to claim 2, characterized in that: The detecting whether the first photo includes a face by using a face detection algorithm includes: Detecting a face rectangular frame of the first photo by using the face detection algorithm, If the face rectangular frame is not detected from the first photo, determining that the first photo does not include a face; If the face rectangular frame is detected from the first photo, the face information is output.

4. The character labeling method according to claim 3, characterized in that: The face information includes the number of faces and the coordinates of at least one vertex on the face rectangular frame.

5. The character labeling method according to claim 2, characterized in that: The detecting whether the photo has the face information includes: Obtaining a photo identification of the photo, and searching a photo database for photo information corresponding to the photo identification according to the photo identification; If the face information corresponding to the photo identification is found in the photo library database, it is determined that the photo has the face information; If the facial information corresponding to the photo identification is not found in the gallery database, it is determined that the photo does not have the facial information.

6. The character labeling method according to claim 5, characterized in that: The detection of whether the photo has been subjected to face detection includes: Searching the image library database for a detection identifier corresponding to the photo identifier according to the photo identifier; If the detection identifier corresponding to the photo identifier is the first identifier, determining that the photo has undergone face detection; If the detection identifier corresponding to the photo identifier is the second identifier, it is determined that the photo has not undergone face detection.

7. The character labeling method according to claim 1, characterized in that: The extracting facial features from the photo includes: Inputting the photo into a facial feature extraction model; Determine a face rectangular frame in the photo using a face feature extraction model; Detecting multiple feature points in the face rectangular frame; Obtaining coordinates of the plurality of feature points; The coordinates of the multiple feature points are outputted through the facial feature extraction model.

8. The character labeling method according to claim 1, characterized in that: The method further comprises: If the image library database does not include a collection of portrait photos corresponding to the facial features of the first photo, clustering multiple photos in the image library database according to the facial features in the first photo to obtain multiple second photos associated with the first photo; A photo collection corresponding to the first photo is created according to the first photo and the multiple second photos.

9. The character labeling method according to claim 8, characterized in that: The clustering of the plurality of photos in the gallery database according to the facial features in the first photo to obtain a plurality of second photos associated with the first photo includes: Acquire facial features of other photos in the gallery database except the first photo; Calculating the similarity between the facial features of the first photo and the facial features of each other photo; A plurality of other photos corresponding to similarities greater than or equal to a preset value are determined as the plurality of second photos associated with the first photo.

10. The character labeling method according to claim 1, characterized in that: The method further comprises: Acquire a plurality of portrait photos in the image library database that are not included in the portrait photo collection; Clustering the acquired multiple portrait photos to determine the category of each portrait photo; The multiple portrait photos are respectively added to the photo collection corresponding to the category.

11. The character labeling method according to claim 1, characterized in that: The method further comprises: If the number of photos in the portrait photo collection corresponding to the facial features of the first photo is greater than or equal to a first preset number, the person annotation control is displayed at the face position of the first photo.

12. The character labeling method according to claim 1, characterized in that: The method further comprises: If the number of newly added photos in the portrait photo collection corresponding to the facial features of the first photo is greater than or equal to a second preset number, the character annotation control is displayed at the face position of the first photo.

13. The character labeling method according to claim 1, characterized in that: The method further comprises: If the character annotation control is not triggered within a preset time, or if the gallery application receives a user operation other than triggering the character annotation control, or if the gallery application interface returns to the photo display interface from the character annotation interface, the character annotation control is canceled.

14. The character labeling method according to claim 1, characterized in that: The method further comprises: If the first photo includes multiple faces, extracting facial features of the multiple faces; If the gallery database includes a collection of portrait photos corresponding to the facial features of at least one face in the first photo, the person annotation control is displayed at the facial position of the at least one face in the first photo.

15. The character labeling method according to claim 14, characterized in that: The method further comprises: If the gallery database includes a collection of portrait photos corresponding to the facial features of at least one face in the first photo, determining the number of photos in the collection of portrait photos corresponding to the facial features of the at least one face; The character annotation control is displayed at the face position of the at least one face in the first photo according to the number of photos in the portrait photo collection corresponding to the facial features of the at least one face.

16. An electronic device, characterized in that: The electronic device comprises a memory and a processor: Wherein, the memory is used to store program instructions; The processor is used to read and execute the program instructions stored in the memory. When the program instructions are executed by the processor, the electronic device executes the character labeling method as described in any one of claims 1 to 15.

17. A chip coupled to a memory in an electronic device, characterized in that: The chip is used to control the electronic device to execute the person labeling method according to any one of claims 1 to 15.

18. A computer storage medium, characterized in that: The computer storage medium stores program instructions, and when the program instructions are executed on an electronic device, a processor of the electronic device executes the person labeling method according to any one of claims 1 to 15.

Images