Image annotation method, device and storage medium

Abstract

The embodiment of the present application discloses an image annotation method, which compares the face quality of two images to be annotated as a comparison pair, so that the basis for user judgment has a measurable objective standard, reducing subjective influence. And based on the comparison result, the annotated image can be used as a node to construct a balanced binary tree, and in any target node in the balanced binary tree, the subnodes on different sides have different face quality comparison results from the target node. Therefore, when the face quality of the target image to be annotated is compared later, the image related to it can be selected according to the balanced binary tree to form a comparison pair, wherein the face quality comparison result of the i-th image to be compared and the i-1-th image to be compared is the same as that of the target image to be annotated and the i-1-th image to be compared in the i-1-th comparison pair. Thereby reducing the number of comparison pairs in the process of completing the identification of the image set to be annotated, and improving the annotation efficiency of face images.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to an image annotation method, device and storage medium. Background Art

[0002] Currently, images with human faces can be automatically identified through a trained network model, and the quality of the face in the image can be determined. One way to train a network model is to train it through labeled training samples.

[0003] In the field of face quality recognition, training samples used to train face quality recognition models can be obtained through manual annotation.

[0004] However, due to the inherent ambiguity and subjectivity of the definition of human facial image quality, it is very difficult to construct appropriate standards and have people annotate the quality of facial images. For example, traditional methods often require people to rate individual facial images on a scale of 1 to 5, which is very unreasonable because there is no consensus standard to define different levels of facial quality, or different users have different standards for dividing facial quality levels, that is, each user has different preferences. For example, the same facial photo may be rated 1 point by user A and 5 points by user B. Therefore, the training samples determined by the above method have a lot of annotation noise, which is not conducive to model training. Summary of the invention

[0005] In order to solve the above technical problems, the present application provides an image annotation method, device and storage medium, which reduce the subjective influence in the annotation process and improve the annotation efficiency of face images.

[0006] The embodiments of the present application disclose the following technical solutions:

[0007] In a first aspect, an embodiment of the present application provides an image annotation method, the method comprising:

[0008] Take two images to be labeled from the image set to be labeled as initial comparison pairs;

[0009] Annotating the to-be-annotated images in the initial comparison pair according to the face quality comparison result for the initial comparison pair;

[0010] The annotated images in the initial comparison pair are added as nodes to a balanced binary tree; for any target node in the balanced binary tree, the node where the first type of image is located and the node where the second type of image is located are located on different sides of the target node, the first type of image and the second type of image are images annotated according to the face quality comparison results in the image set to be annotated, the face quality of the first type of image is lower than that of the image corresponding to the target node, and the face quality of the second type of image is higher than that of the image corresponding to the target node;

[0011] Taking out a target image to be labeled from the set of images to be labeled and forming a first comparison pair with the image corresponding to the root node of the balanced binary tree;

[0012] Annotating the target image to be annotated according to the face quality comparison result of the first comparison pair;

[0013] If the second image to be compared is determined from the balanced binary tree, a second comparison pair is formed based on the target image to be labeled and the second image to be compared, and the image to be compared is further determined based on the face quality comparison result of the second comparison pair; wherein the face quality comparison result between the i-th image to be compared and the i-1-th image to be compared is the same as the face quality comparison result between the target image to be labeled and the i-1-th image to be compared in the i-1-th comparison pair, wherein i is used to identify the order in which the images to be compared are determined for the target image to be labeled from the balanced binary tree, and the number of i is determined according to the labeling process of the target image to be labeled.

[0014] In a second aspect, an embodiment of the present application provides an image annotation device, the device comprising a fetching unit, an annotation unit, and an adding unit:

[0015] The extracting unit is used to extract two images to be labeled from the set of images to be labeled as an initial comparison pair;

[0016] The labeling unit is used to label the to-be-labeled image in the initial comparison pair according to the face quality comparison result for the initial comparison pair;

[0017] The adding unit is used to add the annotated images in the initial comparison pair as nodes to the balanced binary tree; for any target node in the balanced binary tree, the node where the first type of image is located and the node where the second type of image is located are located on different sides of the target node, the first type of image and the second type of image are images annotated according to the face quality comparison results in the image set to be annotated, the face quality of the first type of image is lower than that of the image corresponding to the target node, and the face quality of the second type of image is higher than that of the image corresponding to the target node;

[0018] The extracting unit is further used to extract a target image to be labeled from the set of images to be labeled and form a first comparison pair with the image corresponding to the root node of the balanced binary tree;

[0019] The labeling unit is also used to label the target image to be labeled according to the face quality comparison result of the first comparison pair; if the second image to be compared is determined from the balanced binary tree, a second comparison pair is formed according to the target image to be labeled and the second image to be compared, and the image to be compared is further determined according to the face quality comparison result of the second comparison pair; wherein the face quality comparison result of the i-th image to be compared and the i-1-th image to be compared is the same as the face quality comparison result of the target image to be labeled and the i-1-th image to be compared in the i-1-th comparison pair, wherein i is used to identify the order in which the images to be compared are determined for the target image to be labeled from the balanced binary tree, and the number of i is determined according to the labeling process of the target image to be labeled.

[0020] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store program code, and the program code is used to execute the image annotation method described in the first aspect.

[0021] It can be seen from the above technical solution that, unlike the traditional method, by using two images to be annotated as a comparison pair for face quality comparison, the basis for user judgment has a measurable objective standard, reducing subjective influence. Based on the comparison results, the annotated images can be used as nodes to construct a balanced binary tree. In any target node in the balanced binary tree, the face quality comparison results of the subnodes on different sides are different from those of the target node. Therefore, when the face quality of the target image to be annotated is compared later, the images related to it can be selected from the balanced binary tree to form a comparison pair, wherein the face quality comparison result of the i-th image to be compared and the i-1-th image to be compared is the same as the face quality comparison result of the target image to be annotated in the i-1-th comparison pair and the i-1-th image to be compared. This greatly reduces the number of comparison pairs in the process of completing the identification of the image set to be annotated, and improves the annotation efficiency of face images. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0023] Figure 1A schematic diagram of an image annotation system architecture provided in an embodiment of the present application;

[0024] Figure 2 A schematic diagram of an image annotation interactive interface provided in an embodiment of the present application;

[0025] Figure 3 A method flow chart of an image annotation method provided in an embodiment of the present application;

[0026] Figure 4 A schematic diagram of the principle of a comparison pair generator provided in an embodiment of the present application;

[0027] Figure 5 A schematic diagram of a red-black binary tree provided in an embodiment of the present application;

[0028] Figure 6a A schematic diagram of a left-handed node in a red-black binary tree provided in an embodiment of the present application;

[0029] Figure 6b A schematic diagram of a right-handed node in a red-black binary tree provided in an embodiment of the present application;

[0030] Figure 7 A device structure diagram of an image annotation device provided in an embodiment of the present application;

[0031] Figure 8 A structural diagram of a device for image annotation provided in an embodiment of the present application;

[0032] Fig. 9 A structural diagram of a server provided in an embodiment of the present application. DETAILED DESCRIPTION

[0033] The embodiments of the present application are described below in conjunction with the accompanying drawings.

[0034] At present, most facial image annotations are done manually by scoring a single facial image, which is subjective. Therefore, the training samples determined by the above method have a lot of annotation noise, which is not conducive to model training.

[0035] Therefore, the embodiment of the present application provides an image annotation method, which is different from the traditional method. By using two images to be annotated as a comparison pair for face quality comparison, the basis for user judgment has a measurable objective standard, reducing subjective influence. Based on the comparison results, the annotated images can be used as nodes to construct a balanced binary tree. In any target node in the balanced binary tree, the subnodes on different sides have different face quality comparison results with the target node. Therefore, when the face quality of the target image to be annotated is compared later, the image related to it can be selected according to the balanced binary tree to form a comparison pair, wherein the face quality comparison result of the i-th image to be compared and the i-1-th image to be compared is the same as the face quality comparison result of the target image to be annotated in the i-1-th comparison pair and the i-1-th image to be compared. Thereby, the number of comparison pairs in the process of completing the identification of the image set to be annotated is greatly reduced, and the annotation efficiency of face images is improved.

[0036] The embodiments of the present application can be applied to an image processing device, which has the functions of displaying and modifying images, building and maintaining a balanced binary tree, and generating and displaying comparison pairs based on the balanced binary tree.

[0037] The image processing device may be a terminal device, and the terminal device may be, for example, an intelligent terminal, a computer, a personal digital assistant (PDA for short), a tablet computer, etc.

[0038] The image processing device may also be a server, which provides image processing services to the terminal device. The terminal device may upload the image to be annotated to the server, and the server annotates the image to be annotated using the image annotation method provided in the embodiment of the present application. During the annotation process, the server may display the comparison pair on the terminal device, and construct a balanced binary tree based on the comparison result returned by the terminal device and generate a new comparison pair. The server may be an independent server or a server in a cluster.

[0039] Figure 1 A possible system architecture diagram of an embodiment of the present application is shown in the figure. In the diagram, the balanced binary tree is specifically a red-black binary tree (RB tree). For the image set to be annotated, two images to be annotated can be selected from the image set, and an initial comparison pair can be generated by a comparison pair (pairwise) image generator, which is displayed to the user through an interactive interface.

[0040] The interactive interface can be Figure 2As shown, the algorithm will automatically generate the pairwise comparison results based on the previous comparison results, and then generate the face quality comparison results by interactively clicking 'Left picture is better', 'Right picture is better', and 'Both pictures are equally good'.

[0041] Or mark 'the left image is not a face image', withdraw, etc., until all pairwise comparisons are completed.

[0042] Based on the corresponding first face quality comparison result (i.e., one of the interaction results), one of the two is selected as the root node of R-BTree (the root node of the initialized RB tree) to establish the RB Tree, and the next pairwise is sent to the interaction interface for display and wait for the interaction result.

[0043] Use the dialog function in qyqt5 to interact with the user about the path of the image to be compared, write the absolute paths of all images under the path into the queue, and shuffle the queue.

[0044] The Undo button is used to go back to the previous state. There is a state stack in the history state module. Every time the state changes, the operation and type are stored in the form of a dictionary and pushed into the state stack. Figure 1 As shown, whenever a pullback is required, the state stack pops the dictionary and reads the state, which is sent to the interactive interface by the pairwise generator for display.

[0045] The Submit button can save the current result at any time. Saving the result will generate a complete ranking of the currently labeled images from bad to good quality. It will also save the queue of images to be labeled, the state stack, and the pre- and in-order traversal results of the nodes in the RB Tree.

[0046] The reload button provides a reload function, which can rebuild the binary tree through the in-order and pre-order traversal results, and restore the unfinished results to the last saved state. This function can also mark the results on a pre-calibrated ordered image sequence.

[0047] Next, the image annotation method provided by the embodiment of the present application is further described by means of the accompanying drawings. Figure 3 As shown, the method includes:

[0048] S301: taking two images to be annotated from a set of images to be annotated as an initial comparison pair.

[0049] In the embodiment of the present application, both the initial comparison pair and the subsequent comparison pairs can be generated by a comparison pair generator or a comparison pair image generator.

[0050] The schematic diagram of the generator can be compared as follows: Figure 4As shown, the out-of-queue images of the queue composed of the image set to be compared and the images in the node returned by the RB Tree generate comparison pairs. When the interactive interface generates an interactive result, such as a face quality comparison result, the result will be returned to the RB Tree controller. The controller performs a series of operations on the balanced binary tree according to the user's interactive result, and returns the left child, right child, or root node of the current node according to these operations, and sets the returned node as the current node.

[0051] S302: Annotate the to-be-annotated images in the initial comparison pair according to the face quality comparison result for the initial comparison pair.

[0052] The face quality comparison result may include the face quality judgment result of the user on the two face images to be labeled in the comparison, for example, it may include that the face quality of one of them is better, or the two face qualities are the same.

[0053] S303: Add the annotated image in the initial comparison as a node to the balanced binary tree.

[0054] For any target node in the balanced binary tree, the node where the first type of image is located and the node where the second type of image is located are located on different sides of the target node. The first type of image and the second type of image are images annotated according to the face quality comparison results in the set of images to be annotated. The face quality of the first type of image is lower than that of the image corresponding to the target node, and the face quality of the second type of image is higher than that of the image corresponding to the target node.

[0055] That is to say, for a target node, its left child can be an image with relatively low face quality, and its right child can be an image with relatively high face quality. Of course, it can also be the other way around, the right child can be an image with relatively low face quality, and the left child can be an image with relatively high face quality.

[0056] By arranging the nodes in the balanced binary tree in this way, for any target node, its child nodes all have the above characteristics, thereby facilitating reducing the number of comparison pairs generated subsequently.

[0057] S304: Taking out a target image to be labeled from the set of images to be labeled and forming a first comparison pair with the image corresponding to the root node of the balanced binary tree.

[0058] After the initial comparison pair is determined, the images to be annotated are extracted from the set of images to be annotated for annotation. The target image to be annotated can be one of them. When all comparisons of the target image to be annotated are completed, the target image to be annotated is continuously extracted from the set of images to be annotated until there are no more images to be annotated in the set of images to be annotated.

[0059] Next, the entire face quality comparison process for a target image to be compared is described through S305-S306. Through the following process, the face quality ranking position of the target image to be compared in the entire set of images to be annotated can be clearly determined.

[0060] It should be noted that, in a possible implementation, the image corresponding to the target node includes at least one image, that is, may include multiple images (picture set).

[0061] S305: Annotate the target image to be annotated according to the face quality comparison result of the first comparison pair.

[0062] S306: If the second image to be compared is determined from the balanced binary tree, a second comparison pair is formed based on the target image to be labeled and the second image to be compared, and the image to be compared is further determined based on the face quality comparison result of the second comparison pair; wherein the face quality comparison result between the i-th image to be compared and the i-1-th image to be compared is the same as the face quality comparison result between the target image to be labeled and the i-1-th image to be compared in the i-1-th comparison pair. wherein i is used to identify the order in which the images to be compared are determined for the target image to be labeled from the balanced binary tree, and the number of i is determined based on the labeling process of the target image to be labeled.

[0063] That is to say, when the comparison result between the target image to be compared and the image corresponding to the root node is good quality, if the better quality is on the left and the root node has a left child, then the target image to be compared will be compared with the image corresponding to the left child for comparison. If the target image to be compared is better quality than the image corresponding to the left child, and the left child also has a left child, then the target image to be compared will continue to be compared with the left child for comparison until no image to be compared is determined (i.e., the quality of the comparison image is similar to that of the node image in the tree) or the node in the tree reaches a leaf node. After completing the face quality comparison of the target image to be compared, the position of the target image to be compared in the balanced binary tree can be determined.

[0064] For example Figure 5 In the scenario shown, it is assumed that the selected i-1th image to be compared is image node 40, the face quality of image node 20 is better than that of image node 40, and the face quality of image node 60 is worse than that of image node 40. If the face quality comparison result of the target image to be labeled and image node 40 is that the face quality of the target image to be labeled is better, then the next image to be compared, i.e., the i-th image to be compared, is image node 20 instead of image node 60, because the face quality comparison result of image node 20 and image node 40 is the same as the face quality comparison result of the target image to be labeled and image node 40 (i.e., the i-1th comparison pair).

[0065] based on Figure 2 It can be seen that the face quality comparison result can include not only one side being better, but also the situation where the quality is the same. If the face quality comparison result for a comparison pair is the same quality, the two images in the comparison pair are added to the same node of the balanced binary tree.

[0066] In an optional implementation, if a recall operation is obtained for an image of a comparison pair, the update of the balanced binary tree will be recalled according to the face quality comparison result of the comparison pair.

[0067] In an optional implementation, if an unavailable operation is obtained for an image to be annotated, the image to be annotated is deleted.

[0068] In the face quality comparison process for the target image to be compared, images related to the target image to be compared can be selected from the balanced binary tree to form a comparison pair. The face quality comparison result between the i-th image to be compared and the i-1-th image to be compared is the same as the face quality comparison result between the target image to be labeled and the i-1-th image to be compared in the i-1-th comparison pair. This greatly reduces the number of comparison pairs in the process of completing the labeling of the image set to be labeled, and the number of generated comparison pairs or the number of comparisons is reduced by Reduced to nlogn times. Improved the efficiency of face image annotation.

[0069] Next, we will introduce the balanced binary tree.

[0070] The characteristic of a balanced binary tree is that it dynamically adjusts the positions of nodes in the tree in order to keep the number of child nodes on both sides of the root node balanced as much as possible.

[0071] The red-black binary tree is a balanced binary tree. Due to the characteristics of the red-black binary tree, it is more efficient in adjusting the node balance and will not affect the node relationship previously formed based on the face quality comparison results.

[0072] The red-black binary tree is a special self-balancing binary search tree. Each node of the RB Tree has a storage bit representing the color of the node, which can be red or black.

[0073] Features of RB Tree:

[0074] (1) Each node is either black or red.

[0075] (2) The root node is black.

[0076] (3) Each leaf node (NIL) is black.

[0077] (4) If a node is red, then its child nodes must be black.

[0078] (5) All paths from a node to its descendant nodes contain the same number of black nodes.

[0079] The structure of a red-black binary tree can be Figure 5 As shown. For easy distinction, the white nodes in the figure are red. The numbers in the nodes in the figure are used to distinguish different nodes.

[0080] In order to reduce the number of comparisons to nlogn times, this technical solution checks the above five characteristics each time a new RB Tree node is created. If the five properties cannot be satisfied at the same time, the tree structure needs to be adjusted (left-hand or right-hand) to achieve a balanced binary tree.

[0081] (1) Left-hand rotation: Figure 6a As shown, here we assume that the right child of x is y. Set "y's left child" to "x's right child", that is, set β to the right child of x. Set "x" to "the father of y's left child", that is, set β's father to x. Set "x's father" to "y's father". There are three cases at this time: Case 1: If "x's father" is an empty node, set y to the root node; Case 2: If x is the left child of its parent node, set y to "the left child of x's parent node"; Case 3: (x is the right child of its parent node) set y to "the right child of x's parent node". Finally, set "x" to "y's left child" and set "x's parent node" to "y".

[0082] (2) Right-handed: Figure 6b As shown, it is exactly the opposite of left rotation.

[0083] Next, the addition of nodes in RB Tree is described.

[0084] Step 1: Treat RB Tree as a binary search tree and insert the node.

[0085] Step 2: Color the inserted node red.

[0086] Step 3: Through a series of operations such as rotation or coloring, it becomes a RB Tree again.

[0087] The coloring and rotation strategies when adding nodes are shown in Table 1:

[0088]

[0089]

[0090] Table 1: Coloring and rotation strategies when adding nodes

[0091] Next, the deletion of nodes in the RB Tree is described.

[0092] Step 1: Treat RB Tree as a binary search tree and delete node y.

[0093] Step 2: Modify the tree through a series of operations such as "rotation and recoloring" to make it an R-BTree again.

[0094] We assume that "x contains an extra black" (the original color of x still exists), then the coloring and rotation strategies when deleting nodes are shown in Table 2:

[0095]

[0096]

[0097] Table 2: Coloring and rotation strategies when deleting nodes

[0098] Figure 7 The device structure diagram of an image annotation device provided in an embodiment of the present application, the device 700 includes a fetching unit 701, a labeling unit 702 and an adding unit 703:

[0099] The extracting unit 701 is used to extract two images to be labeled from the set of images to be labeled as an initial comparison pair;

[0100] The labeling unit 702 is used to label the to-be-labeled image in the initial comparison pair according to the face quality comparison result of the initial comparison pair;

[0101] The adding unit 703 is used to add the annotated images in the initial comparison pair as nodes to the balanced binary tree; for any target node in the balanced binary tree, the node where the first type of image is located and the node where the second type of image is located are located on different sides of the target node, the first type of image and the second type of image are images annotated according to the face quality comparison results in the image set to be annotated, the face quality of the first type of image is lower than that of the image corresponding to the target node, and the face quality of the second type of image is higher than that of the image corresponding to the target node;

[0102] The extraction unit 701 is further used to extract a target image to be labeled from the set of images to be labeled and form a first comparison pair with the image corresponding to the root node of the balanced binary tree;

[0103] The labeling unit 702 is also used to label the target image to be labeled according to the face quality comparison result of the first comparison pair; if the second image to be compared is determined from the balanced binary tree, a second comparison pair is formed according to the target image to be labeled and the second image to be compared, and the image to be compared is further determined according to the face quality comparison result of the second comparison pair; wherein the face quality comparison result of the i-th image to be compared and the i-1-th image to be compared is the same as the face quality comparison result of the target image to be labeled and the i-1-th image to be compared in the i-1-th comparison pair, wherein i is used to identify the order in which the images to be compared are determined for the target image to be labeled from the balanced binary tree, and the number of i is determined according to the labeling process of the target image to be labeled.

[0104] Optionally, the target node corresponds to at least one image.

[0105] Optionally, the adding unit is further configured to add two images in a comparison pair to the same node of the balanced binary tree if a face quality comparison result for a comparison pair is that the quality is the same.

[0106] Optionally, if a recall operation for an image of a comparison pair is obtained, the adding unit is further configured to recall the update of the balanced binary tree according to the face quality comparison result of the comparison pair.

[0107] Optionally, if an unavailable operation for an image to be annotated is obtained, the adding unit is further configured to delete the image to be annotated.

[0108] Optionally, the balanced binary tree is a red-black binary tree.

[0109] It can be seen that, unlike the traditional method, by using two images to be annotated as comparison pairs for face quality comparison, the basis for user judgment has a measurable objective standard, reducing subjective influence. Based on the comparison results, the annotated images can be used as nodes to construct a balanced binary tree. In any target node in the balanced binary tree, the face quality comparison results of the child nodes on different sides are different from those of the target node. Therefore, when the face quality of the target image to be annotated is compared later, the images related to it can be selected from the balanced binary tree to form a comparison pair, where the face quality comparison result of the i-th image to be compared and the i-1-th image to be compared is the same as the face quality comparison result of the target image to be annotated in the i-1-th comparison pair and the i-1-th image to be compared. This greatly reduces the number of comparison pairs in the process of completing the identification of the image set to be annotated, and improves the annotation efficiency of face images.

[0110] The present application also provides a device for image identification. The device for image identification is described below in conjunction with the accompanying drawings. Figure 8 As shown, the embodiment of the present application provides a device 1300 for image identification, which may be a terminal device, and the terminal device may be any intelligent terminal including a mobile phone, a tablet computer, a personal digital assistant (PDA), a point of sales (POS), a vehicle-mounted computer, etc., and the terminal device is a mobile phone as an example:

[0111] Figure 8 FIG. 1 is a block diagram showing a partial structure of a mobile phone related to a terminal device provided in an embodiment of the present application. Figure 8 The mobile phone includes: a radio frequency (RF) circuit 1310, a memory 1320, an input unit 1330, a display unit 1340, a sensor 1350, an audio circuit 1360, a wireless fidelity (WiFi) module 1370, a processor 1380, and a power supply 1390. Those skilled in the art can understand that Figure 8 The mobile phone structure shown in the figure does not constitute a limitation on the mobile phone, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0112] Combine the following Figure 8 A detailed introduction to the various components of the mobile phone:

[0113] The RF circuit 1310 can be used for receiving and sending signals during the process of sending and receiving information or making calls. In particular, after receiving the downlink information of the base station, it is sent to the processor 1380 for processing; in addition, the designed uplink data is sent to the base station. Usually, the RF circuit 1310 includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (Low Noise Amplifier, referred to as LNA), a duplexer, etc. In addition, the RF circuit 1310 can also communicate with the network and other devices through wireless communication. The above-mentioned wireless communication can use any communication standard or protocol, including but not limited to the Global System of Mobile communication (Global System of Mobile communication, referred to as GSM), General Packet Radio Service (General Packet Radio Service, referred to as GPRS), Code Division Multiple Access (Code Division Multiple Access, referred to as CDMA), Wideband Code Division Multiple Access (Wideband Code Division Multiple Access, referred to as WCDMA), Long Term Evolution (Long Term Evolution, referred to as LTE), email, Short Messaging Service (SMS), etc.

[0114] The memory 1320 can be used to store software programs and modules. The processor 1380 executes various functional applications and data processing of the mobile phone by running the software programs and modules stored in the memory 1320. The memory 1320 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory 1320 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0115] The input unit 1330 can be used to receive input digital or character information, and to generate key signal input related to the user settings and function control of the mobile phone. Specifically, the input unit 1330 may include a touch panel 1331 and other input devices 1332. The touch panel 1331, also known as a touch screen, can collect the user's touch operation on or near it (such as the user's operation on the touch panel 1331 or near the touch panel 1331 using any suitable object or accessory such as a finger, stylus, etc.), and drive the corresponding connection device according to a pre-set program. Optionally, the touch panel 1331 may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into contact coordinates, and then sends it to the processor 1380, and can receive and execute commands sent by the processor 1380. In addition, the touch panel 1331 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic waves. In addition to the touch panel 1331, the input unit 1330 may also include other input devices 1332. Specifically, other input devices 1332 may include but are not limited to one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, etc.

[0116] The display unit 1340 can be used to display information input by the user or information provided to the user and various menus of the mobile phone. The display unit 1340 may include a display panel 1341. Optionally, the display panel 1341 may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc. Further, the touch panel 1331 may cover the display panel 1341. When the touch panel 1331 detects a touch operation on or near it, it is transmitted to the processor 1380 to determine the type of touch event. Subsequently, the processor 1380 provides a corresponding visual output on the display panel 1341 according to the type of touch event. Although in Figure 8 In the embodiment, the touch panel 1331 and the display panel 1341 are used as two independent components to realize the input and output functions of the mobile phone, but in some embodiments, the touch panel 1331 and the display panel 1341 can be integrated to realize the input and output functions of the mobile phone.

[0117] The mobile phone may also include at least one sensor 1350, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel 1341 according to the brightness of the ambient light, and the proximity sensor may turn off the display panel 1341 and / or the backlight when the mobile phone is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in all directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that identify the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors that can be configured in the mobile phone, such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., they will not be repeated here.

[0118] The audio circuit 1360, the speaker 1361, and the microphone 1362 can provide an audio interface between the user and the mobile phone. The audio circuit 1360 can transmit the received audio data to the speaker 1361 after converting the received audio data into an electrical signal, which is converted into a sound signal for output; on the other hand, the microphone 1362 converts the collected sound signal into an electrical signal, which is received by the audio circuit 1360 and converted into audio data, and then the audio data is output to the processor 1380 for processing, and then sent to another mobile phone through the RF circuit 1310, or the audio data is output to the memory 1320 for further processing.

[0119] WiFi is a short-range wireless transmission technology. The mobile phone can help users send and receive emails, browse web pages and access streaming media through the WiFi module 1370. It provides users with wireless broadband Internet access. Figure 8 A WiFi module 1370 is shown, but it is understandable that it is not an essential component of the mobile phone and can be omitted as needed without changing the essence of the invention.

[0120] The processor 1380 is the control center of the mobile phone. It uses various interfaces and lines to connect various parts of the entire mobile phone. It executes various functions of the mobile phone and processes data by running or executing software programs and / or modules stored in the memory 1320, and calling data stored in the memory 1320. Optionally, the processor 1380 may include one or more processing units; preferably, the processor 1380 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 1380.

[0121] The mobile phone also includes a power supply 1390 (such as a battery) for supplying power to various components. Preferably, the power supply can be logically connected to the processor 1380 through a power management system, so that the power management system can manage functions such as charging, discharging, and power consumption management.

[0122] Although not shown, the mobile phone may also include a camera, a Bluetooth module, etc., which will not be described in detail here.

[0123] In this embodiment, the processor 1380 included in the terminal device also has the following functions:

[0124] Take two images to be labeled from the image set to be labeled as initial comparison pairs;

[0125] Annotating the to-be-annotated images in the initial comparison pair according to the face quality comparison result for the initial comparison pair;

[0126] The annotated images in the initial comparison pair are added as nodes to a balanced binary tree; for any target node in the balanced binary tree, the node where the first type of image is located and the node where the second type of image is located are located on different sides of the target node, the first type of image and the second type of image are images annotated according to the face quality comparison results in the image set to be annotated, the face quality of the first type of image is lower than that of the image corresponding to the target node, and the face quality of the second type of image is higher than that of the image corresponding to the target node;

[0127] Taking out a target image to be labeled from the set of images to be labeled and forming a first comparison pair with the image corresponding to the root node of the balanced binary tree;

[0128] Annotating the target image to be annotated according to the face quality comparison result of the first comparison pair;

[0129] If the second image to be compared is determined from the balanced binary tree, a second comparison pair is formed based on the target image to be labeled and the second image to be compared, and the image to be compared is further determined based on the face quality comparison result of the second comparison pair; wherein the face quality comparison result between the i-th image to be compared and the i-1-th image to be compared is the same as the face quality comparison result between the target image to be labeled and the i-1-th image to be compared in the i-1-th comparison pair, wherein i is used to identify the order in which the images to be compared are determined for the target image to be labeled from the balanced binary tree, and the number of i is determined according to the labeling process of the target image to be labeled.

[0130] The device for image annotation provided in the embodiment of the present application may be a server. Fig. 9 As shown, Fig. 9The structural diagram of the server 1400 provided in the embodiment of the present application, the server 1400 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPUs) 1422 (for example, one or more processors) and a memory 1432, and one or more storage media 1430 (for example, one or more mass storage devices) storing application programs 1442 or data 1444. Among them, the memory 1432 and the storage medium 1430 can be short-term storage or permanent storage. The program stored in the storage medium 1430 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the server. Furthermore, the central processing unit 1422 can be configured to communicate with the storage medium 1430 and execute a series of instruction operations in the storage medium 1430 on the server 1400.

[0131] The server 1400 may also include one or more power supplies 1426, one or more wired or wireless network interfaces 1450, one or more input and output interfaces 1458, and / or one or more operating systems 1441, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

[0132] The steps performed by the server in the above embodiment can be based on the Fig. 9 The server structure shown.

[0133] The CPU 1422 is used to perform the following steps:

[0134] Take two images to be labeled from the image set to be labeled as initial comparison pairs;

[0135] Annotating the to-be-annotated images in the initial comparison pair according to the face quality comparison result for the initial comparison pair;

[0136] The annotated images in the initial comparison pair are added as nodes to a balanced binary tree; for any target node in the balanced binary tree, the node where the first type of image is located and the node where the second type of image is located are located on different sides of the target node, the first type of image and the second type of image are images annotated according to the face quality comparison results in the image set to be annotated, the face quality of the first type of image is lower than that of the image corresponding to the target node, and the face quality of the second type of image is higher than that of the image corresponding to the target node;

[0137] Taking out a target image to be labeled from the set of images to be labeled and forming a first comparison pair with the image corresponding to the root node of the balanced binary tree;

[0138] Annotating the target image to be annotated according to the face quality comparison result of the first comparison pair;

[0139] If the second image to be compared is determined from the balanced binary tree, a second comparison pair is formed based on the target image to be labeled and the second image to be compared, and the image to be compared is further determined based on the face quality comparison result of the second comparison pair; wherein the face quality comparison result between the i-th image to be compared and the i-1-th image to be compared is the same as the face quality comparison result between the target image to be labeled and the i-1-th image to be compared in the i-1-th comparison pair, wherein i is used to identify the order in which the images to be compared are determined for the target image to be labeled from the balanced binary tree, and the number of i is determined according to the labeling process of the target image to be labeled.

[0140] An embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store program codes, and the program codes are used to execute the image annotation method described in the above embodiments.

[0141] The terms "first", "second", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein, for example. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0142] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0143] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0144] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0145] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0146] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store program codes.

[0147] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An image annotation method, characterized in that: The method comprises: Take two images to be labeled from the image set to be labeled as initial comparison pairs; Annotating the to-be-annotated images in the initial comparison pair according to the face quality comparison result for the initial comparison pair; The annotated images in the initial comparison pair are added as nodes to a balanced binary tree; for any target node in the balanced binary tree, the node where the first type of image is located and the node where the second type of image is located are located on different sides of the target node, the first type of image and the second type of image are images annotated according to the face quality comparison results in the image set to be annotated, the face quality of the first type of image is lower than that of the image corresponding to the target node, and the face quality of the second type of image is higher than that of the image corresponding to the target node; Taking out a target image to be labeled from the set of images to be labeled and forming a first comparison pair with the image corresponding to the root node of the balanced binary tree; Annotating the target image to be annotated according to the face quality comparison result of the first comparison pair; If the second image to be compared is determined from the balanced binary tree, a second comparison pair is formed based on the target image to be labeled and the second image to be compared, and the image to be compared is further determined based on the face quality comparison result of the second comparison pair; wherein the face quality comparison result between the i-th image to be compared and the i-1-th image to be compared is the same as the face quality comparison result between the target image to be labeled and the i-1-th image to be compared in the i-1-th comparison pair, wherein i is used to identify the order in which the images to be compared are determined for the target image to be labeled from the balanced binary tree, and the number of i is determined according to the labeling process of the target image to be labeled.

2. The method according to claim 1, characterized in that The image corresponding to the target node includes at least one image.

3. The method according to claim 1, characterized in that If the face quality comparison result for a comparison pair is that the quality is the same, the two images in the comparison pair are added to the same node of the balanced binary tree.

4. The method according to claim 1, characterized in that: If a recall operation is obtained for a compared image, the update of the balanced binary tree will be recalled according to the face quality comparison result of the compared image.

5. The method according to claim 1, characterized in that If an unavailable operation is obtained for an image to be annotated, the image to be annotated is deleted.

6. The method according to any one of claims 1 to 5, characterized in that: The balanced binary tree is a red-black binary tree.

7. An image annotation device, characterized in that: The device comprises a taking-out unit, a marking unit and an adding unit: The extracting unit is used to extract two images to be labeled from the set of images to be labeled as an initial comparison pair; The labeling unit is used to label the to-be-labeled image in the initial comparison pair according to the face quality comparison result for the initial comparison pair; The adding unit is used to add the annotated images in the initial comparison pair as nodes to the balanced binary tree; for any target node in the balanced binary tree, the node where the first type of image is located and the node where the second type of image is located are located on different sides of the target node, the first type of image and the second type of image are images annotated according to the face quality comparison results in the image set to be annotated, the face quality of the first type of image is lower than that of the image corresponding to the target node, and the face quality of the second type of image is higher than that of the image corresponding to the target node; The extracting unit is further used to extract a target image to be labeled from the set of images to be labeled and form a first comparison pair with the image corresponding to the root node of the balanced binary tree; The labeling unit is also used to label the target image to be labeled according to the face quality comparison result of the first comparison pair; if the second image to be compared is determined from the balanced binary tree, a second comparison pair is formed according to the target image to be labeled and the second image to be compared, and the image to be compared is further determined according to the face quality comparison result of the second comparison pair; wherein the face quality comparison result of the i-th image to be compared and the i-1-th image to be compared is the same as the face quality comparison result of the target image to be labeled and the i-1-th image to be compared in the i-1-th comparison pair, wherein i is used to identify the order in which the images to be compared are determined for the target image to be labeled from the balanced binary tree, and the number of i is determined according to the labeling process of the target image to be labeled.

8. The device according to claim 7, characterized in that The image corresponding to the target node includes at least one image.

9. The device according to claim 7, characterized in that The adding unit is further configured to add two images in a comparison pair to the same node of the balanced binary tree if a comparison result of the face quality of the comparison pair is that the quality is the same.

10. The device according to claim 7, characterized in that If an image recall operation for a comparison pair is obtained, the adding unit is further configured to recall the update of the balanced binary tree according to the face quality comparison result of the comparison pair.

11. The device according to claim 7, characterized in that If an unavailable operation for an image to be annotated is obtained, the adding unit is further configured to delete the image to be annotated.

12. The device according to any one of claims 7 to 11, characterized in that: The balanced binary tree is a red-black binary tree.

13. A device for image marking, characterized in that: including memory and processor; The memory is used to store software programs and modules; The processor executes the image annotation method according to any one of claims 1 to 6 by running the software program and modules stored in the memory.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store program codes, and the program codes are used to execute the image annotation method according to any one of claims 1 to 6.

Images