A method, system, computer, and storage medium for multi-classification face image annotation.

Abstract

This application relates to the field of image processing and discloses a method, system, computer, and storage medium for multi-class face image annotation. The method includes: acquiring at least one face image and related information of the corresponding face image; grouping all the face images according to the related information and preset grouping rules; integrating all the face images in each group into a single group master image; and assigning corresponding labels to each face image in each group master image based on a preset label library and preset labeling rules. This application can perform multi-class face image annotation and can also establish visual relationships between each image, making image annotation faster and saving more time and costs.

Description

Technical Field

[0001] This application relates to the field of image processing, and in particular to a method, system, computer, and storage medium for labeling multi-class face images. Background Technology

[0002] With the continuous development of technology, more and more devices and services need to utilize image processing technology to solve various problems. Existing sample annotation systems primarily annotate binary sample sets, and cannot annotate sample sets with multiple classifications. Furthermore, the method of annotating each image individually results in significantly increased annotation time. Summary of the Invention

[0003] In view of the above problems, this application proposes a multi-classification face image annotation method, system, computer, and storage medium.

[0004] This application proposes a multi-class face image annotation method, including:

[0005] Obtain at least one face image and related information for the corresponding face image;

[0006] All the face images are grouped according to the relevant information and preset grouping rules;

[0007] All the face images in each group are combined into a single group master image;

[0008] Based on a preset tag library and preset tag rules, each face image in each group of main images is labeled with a corresponding tag.

[0009] Furthermore, in the above-described multi-class face image annotation method, after obtaining at least one face image and its corresponding information, the method further includes:

[0010] Establish a master image library corresponding to each of the aforementioned face images;

[0011] Establish a relationship table between the main image of each group and the overall image library;

[0012] After labeling each face image in each of the grouped main images with a corresponding tag, the method further includes:

[0013] Based on the association table, the tag is updated to the corresponding face image in the image library.

[0014] Furthermore, in the above-described multi-class face image annotation method, updating the label to the corresponding face image in the overall image database includes:

[0015] When the face image in the image library does not have a corresponding tag, the currently labeled tag is added to the corresponding face image;

[0016] When the face image in the image library already has a corresponding tag, the currently labeled tag will overwrite the existing tag.

[0017] Furthermore, in the above-described multi-class face image annotation method, the relevant information includes the image address corresponding to the face image and the image-related person ID, wherein the image address is the geographical location of the object appearing in the image.

[0018] Furthermore, in the above-described multi-class face image annotation method, the face images are grouped according to the relevant information in one of the following ways:

[0019] Method 1: Group the images according to the IDs of the people associated with them;

[0020] Method 2: Group the images according to their URLs;

[0021] Method 3: Group the images according to the IDs of the people associated with them and the addresses of the images.

[0022] Furthermore, in the above-described multi-class face image annotation method, the step of annotating each face image in each group of main images with a corresponding label based on a preset label library and preset label rules includes:

[0023] Based on the preset tag library and the preset tag rules, each face image in each group of main images is labeled with corresponding tags in sequence.

[0024] Furthermore, in the above-described multi-class face image annotation method, each face image corresponds to at least one of the aforementioned labels.

[0025] Another embodiment of this application proposes a multi-class image annotation system, including:

[0026] The acquisition unit is used to acquire at least one face image and related information of the corresponding face image;

[0027] A grouping unit is used to group all the face images according to the relevant information and preset grouping rules;

[0028] An integration unit is used to integrate all the face images in each group into a single group master image;

[0029] The annotation unit is used to annotate each face image in each group of main images with corresponding labels based on a preset label library and preset label rules.

[0030] Another embodiment of this application also proposes a computer, including a storage unit and a processing unit. The storage unit stores a computer program, and the processing unit executes the steps of the multi-classification face image annotation method described above by calling the computer program stored in the storage unit.

[0031] Another embodiment of this application also proposes a computer-readable storage medium storing a computer program adapted for loading by a processor to perform the steps of the multi-classification face image annotation method described above.

[0032] The embodiments of this application have the following beneficial effects:

[0033] This application proposes a multi-class image annotation method. By establishing a multi-class sample set and integrating images into groups, a visual correlation is established between each image, enabling rapid differentiation of differences between images and reducing annotation time. Furthermore, by establishing a link between the main images of each group and a comprehensive database of face images, after annotation is completed in groups, the link is queried to batch update the annotation data in the comprehensive face image database. This not only makes image annotation faster but also simplifies label maintenance. Additionally, it saves significant time and maintenance costs. Attached Figure Description

[0034] To more clearly illustrate the technical solutions of this application, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of this application and should not be considered as a limitation on the scope of protection of this application. In the various drawings, similar components are numbered similarly.

[0035] Figure 1 A flowchart illustrating some existing implementations of binary classification face image annotation methods is shown.

[0036] Figure 2 This paper illustrates a first flowchart of a multi-classification face image annotation method according to some embodiments of the present application;

[0037] Figure 3 This paper illustrates a second flowchart of a multi-classification face image annotation method according to some embodiments of the present application;

[0038] Figure 4 A third flowchart of a multi-class face image annotation method according to some embodiments of this application is shown;

[0039] Figure 5 A schematic diagram of the structure of a multi-class face image annotation system according to some embodiments of this application is shown. Detailed Implementation

[0040] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0041] The components of the embodiments of this application described and illustrated in the accompanying drawings can be arranged and designed in a variety of different configurations. Therefore, the following detailed description of the embodiments of this application provided in the drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0042] In the following, the terms “comprising,” “having,” and their cognates, which may be used in various embodiments of this application, are intended only to indicate a particular feature, number, step, operation, element, component, or combination thereof, and should not be construed as excluding, firstly, the presence of one or more other features, numbers, steps, operations, elements, components, or combinations thereof, or adding the possibility of one or more features, numbers, steps, operations, elements, components, or combinations thereof.

[0043] Furthermore, the terms "first," "second," and "third" are used only to distinguish descriptions and should not be interpreted as indicating or implying relative importance.

[0044] Unless otherwise specified, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which the various embodiments of this application pertain. Terms (such as those defined in commonly used dictionaries) shall be interpreted as having the same meaning as in their contextual meaning in the relevant technical field and shall not be construed as having an idealized or overly formal meaning, unless clearly defined in the various embodiments of this application.

[0045] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0046] Typically, such as Figure 1 As shown, existing annotation methods, besides their limitations with binary classification sample sets, also cause numerous problems due to the way they annotate the sample sets. For example, the method involves first popping up individual images sequentially, then annotating each image one by one, and finally updating the annotation information for each image in the overall image database. This not only results in a lack of visual correlation between images, thus consuming more annotation time, but also leads to inconsistent labels between different groups of images, making label maintenance more difficult.

[0047] Therefore, this application proposes a multi-class image annotation method to solve the above problems.

[0048] Please refer to Figure 2 This is a flowchart of a multi-class face image annotation method proposed in an embodiment of this application. Exemplarily, this multi-class face image annotation method is applied to an annotation system.

[0049] like Figure 2 As shown, a multi-class face image annotation method may include the following steps:

[0050] S101, Obtain at least one face image and related information of the corresponding face image.

[0051] Specifically, the system first acquires at least one facial image. Each facial image has a corresponding source, such as being uploaded to the system by staff or customers, or automatically acquired through methods like taking a photo with a camera. These are just examples and not the only possible sources; other methods are also possible and are not limited here. Each facial image contains corresponding related information, including but not limited to the uploader's information and information about the subject in the photograph. If the facial image is taken by a camera, the recognition system identifies the subject in the photograph and acquires that subject's information, using this information as the basis for the facial image's information.

[0052] For example, if the photo is of customer 'a', the system identifies the photo and determines that the photo belongs to customer 'a', obtains relevant information about customer 'a', and uses the obtained relevant information as the relevant information of the facial image.

[0053] In some implementations, the relevant information includes the image address corresponding to the face image and the image-related person ID. The image address is the geographical location of the object appearing in the image. This relevant information may also include, but is not limited to, the name, gender, job title, address, and age of the object in the face image. Whenever a face image is acquired, the system identifies the object in the sample image, generates a unique ID corresponding to that object, and uses this ID as the image-related person ID. For example, when photo b is acquired, after identifying the object in photo b, a corresponding and unique ID of 1000056789 is generated for this object; therefore, the image-related person ID for photo b is 1000056789. Of course, the number of digits in the ID is not unique and depends on the specific circumstances.

[0054] S201, group all face images according to relevant information and preset grouping rules.

[0055] Specifically, after obtaining the relevant information corresponding to each face image, the images are grouped according to one or more pieces of information based on preset grouping rules. The purpose of grouping is to better manage, update, and maintain the tags, and to facilitate various subsequent operations.

[0056] In some real-time methods, in the above-mentioned multi-class face image annotation method, the grouping rule for grouping face images according to relevant information and a preset grouping rule is one of the following:

[0057] Method 1: Group according to the ID of the person associated with the image.

[0058] Specifically, the grouping is based solely on the relevant person ID, indicating that the same person may correspond to multiple facial images. The advantage of this grouping is that when a person's facial image is needed, all facial images of that person can be quickly retrieved. This facilitates the analysis of relevant data based on different facial images of the same person, providing comparative analysis and enabling the extraction of important related data.

[0059] For example, client c corresponds to image c1, image d2, and image h1.

[0060] Method 2: Group by image URL.

[0061] Specifically, the grouping is based solely on image URLs, where the image URL is the geographical location of the object in the face image. For example, customer g's geographical location is province h. Of course, the geographical location can be expanded to countries, such as country m, country z, or country y. The geographical location can also be narrowed down to a specific neighborhood, such as a specific neighborhood in a specific street of a specific city in a specific province. These can be arbitrarily set according to the required requirements and are not limited here. This grouping method includes, but is not limited to, facilitating the identification of differences between faces in each region.

[0062] Method 3: Group the images according to the ID of the person associated with them and the image URL.

[0063] Specifically, the grouping is based on two conditions: the relevant person's ID and the image address. This makes the grouping more detailed. Since the same person may have facial images in different regions, only facial images from the same person and from the same location can be grouped together if both conditions are met. For example, if customer g has images c1 and c3 in country m and image c2 in country c, then c1 and c3 can only be grouped together, and image c2 can be grouped separately.

[0064] As can be imagined, the above three methods are just examples in this embodiment and do not represent only these three. The above description also includes other information, which can be used as the basis for grouping and can be combined with each other as grouping conditions. There are no restrictions here.

[0065] S301 integrates all the face images in each group into a single main group image.

[0066] Specifically, in order to establish a visual connection between multiple photos, all the facial images in each group can be integrated into a single large image. This integrated large image can then be used as the main image for the group. This allows for a quick and intuitive identification of the differences between each facial image based on this single large image, which also facilitates subsequent labeling.

[0067] S401, based on a preset tag library and preset tag rules, assigns corresponding tags to each face image in the main image of each group.

[0068] Specifically, facial landmarks are annotations used to locate facial features and contours in an image. They are mainly used to locate key facial positions, such as the face outline, eyebrows, eyes, and lips. Facial landmark detection is a crucial step in the face recognition process. The preset label library contains various labels, such as nose shape, eye shape, and facial contour. Each label also contains corresponding data. For example, the nose shape label contains data on the nose shape, including the positions of multiple key points. Optionally, the number of facial landmarks can range from 25 to 109; a larger number results in more precise annotations. Of course, the key points selected here are just an example; more than 109 are also possible, depending on the specific situation. The annotation method can be automatic or manual by annotators. The annotator's basic skills, the annotation team's review capabilities, and the quality of the annotations all significantly impact the accuracy of the AI ​​face model algorithm.

[0069] In some implementations, such as Figure 3 As shown, the multi-class face image annotation method also includes:

[0070] S402, Establish a master image library corresponding to each face image.

[0071] Specifically, in addition to labeling each image, the labeled images also need to be updated to the corresponding image database. Therefore, it is necessary to establish an image database at the beginning to store all the acquired face images in the image database for easy acquisition and updating later.

[0072] S403, Establish a relationship table between the main image of each group and the overall image library.

[0073] Specifically, updating the overall image database requires a corresponding relationship. This necessitates establishing a relationship table between each group's main image and the overall image database. This table is used to update the labeled face images in each group's main image to the overall image database. The relationship table includes the connections between each group's main image and the overall image database. Because each image sample in the overall image database is shuffled, updating each individual face image would be extremely cumbersome and time-consuming. With the relationship table, the group's main image to which each face image belongs in the overall image database can be accurately retrieved, ensuring a one-to-one correspondence. This allows for rapid updates to each face image in the overall image database.

[0074] S404: After the labels are marked, update the labels to the corresponding face images in the image database based on the association table.

[0075] As an example, face images a1, a2, g3, and o3 form the first group. These four face images in the first group are then integrated into the main image of group A1. The association table will then store the correspondence between a1, a2, g3, and o3 and A1. This way, if it is necessary to update the tags corresponding to these four face images to the overall image library later, it can be quickly found and executed.

[0076] In some implementations, such as Figure 4 As shown, in this multi-class face image annotation method, updating the labels to the corresponding face images in the overall image database based on the association table includes:

[0077] If a face image in the overall image library does not have a corresponding tag, then the currently labeled tag is added to the corresponding face image.

[0078] Specifically, when the labels corresponding to a face image are updated to the overall image database for the first time, the labels for the face image are not yet available. At this time, the label field symbol is NULL, representing an empty field. The current label data is then directly stored in the corresponding location. For example, if the label corresponding to face image a1 is a1005, the corresponding field symbol changes from NULL to a1005.

[0079] If a face image in the overall image library already has a corresponding label, then the currently labeled label will overwrite the existing label.

[0080] Specifically, when a face image has been previously labeled but is now being re-labeled, the current label needs to overwrite the old label, meaning it needs to be directly replaced. For example, if face image a1 was previously labeled with the label a1005, then the corresponding field symbol would be a1005. If the current label is a1006, then the field symbol for the corresponding label of that face image would be directly changed from a1005 to a1006.

[0081] Of course, the second scenario usually occurs during technology updates. With improvements in annotation methods and advancements in face recognition, annotations become more detailed. The labels on previous face images are less precise compared to those obtained with later improved technologies, requiring updates to the corresponding labels. This allows for timely acquisition of the latest experimental data, leading to more accurate results in subsequent operations.

[0082] In some implementations, the multi-class face image annotation method, based on a preset label library and preset label rules, assigns corresponding labels to each face image in each group's main image, including:

[0083] Based on a preset tag library and preset tag rules, each face image in each group's main image is labeled with its corresponding tag in sequence.

[0084] As an example, if there are three groups (Group 1, Group 2, and Group 3), and the default labeling rule is to label in the order of priority (first, second, and third), then Group 1 will be labeled first. After Group 1 is labeled, Group 2 will be labeled, and finally Group 3. Optionally, the priority can also be third, second, and first, or second, first, and third, etc. There are no limitations here. It can be set arbitrarily according to requirements.

[0085] In some implementations, in the multi-class face image annotation method, each face image corresponds to at least one label.

[0086] Specifically, many existing annotation systems assign only one label to a single face image. This is clearly insufficient to cover the requirements in various situations. Therefore, the method in this embodiment can annotate the same face image with multiple labels. Multiple label values ​​refer to various types of labels, but only one label of the same type is used. Each label has a unique code or symbol. This allows for corresponding updates under various circumstances.

[0087] Alternatively, each tag symbol can correspond to specific content data, including the positions of various key points. In this case, it's not necessary to change the code or symbol corresponding to each tag type; only the content data corresponding to each tag needs to be changed. For example, the a1005 eye tag also corresponds to key points (x1, y1), (x2, y2), and (x3, y3). The latest tag is also a1005 eye tag, but the key points obtained are different. If the key points obtained at the current moment are (x1, y1), (x2, y3), and (x3, y3), then all the current key points will replace all the previous key points. In other words, all the content data corresponding to the same type of tag at the current moment will replace the previous content data.

[0088] The following will show several tag update scenarios:

[0089] Scenario 1: When a single face image has multiple tags, and only some tags need updating, simply replace the content corresponding to the tag with the latest tag. For example, if face image a1 has three tags: a1005 (eyes), b1006 (nose), and f1008 (face outline), and only the old eye tag a1005 needs updating, then simply replace the content data corresponding to the a1005 eye tag with the latest content data.

[0090] Scenario 2: When a single face image has multiple tags, and all tags need to be updated at some point, simply replace the corresponding tag symbols with the latest tags. For example, if face image a1 corresponds to three tags: a1005 (eyes), b1006 (nose), and f1008 (face outline), and all tags need to be updated, then the content data corresponding to the a1005 (eyes), b1006 (nose), and f1008 (face outline) tags needs to be replaced with the latest content data.

[0091] Scenario 3: When the same face image has multiple tags, and at some point it is necessary to add some tags, then you only need to add the tags and corresponding content data.

[0092] Another embodiment of this application also proposes a multi-class image annotation system 500, such as... Figure 5 As shown, system 500 includes:

[0093] The acquisition unit 501 is used to acquire at least one face image and related information of the corresponding face image.

[0094] Grouping unit 502 is used to group all face images according to relevant information and preset grouping rules.

[0095] Integration unit 503 is used to integrate all face images in each group into a single group master image.

[0096] Labeling unit 504 is used to label each face image in each group's main image with a corresponding label based on a preset label library and preset label rules.

[0097] Another embodiment of this application also proposes a computer, including a storage unit and a processing unit. The storage unit stores a computer program, and the processing unit executes the steps of the multi-classification face image annotation method described above by calling the computer program stored in the storage unit.

[0098] Another embodiment of this application also proposes a computer-readable storage medium storing a computer program adapted for loading by a processor to perform the steps of the multi-classification face image annotation method described above.

[0099] It is understood that the method steps in this embodiment correspond to the multi-class image annotation method in the above embodiments. The options of the above multi-class image annotation method are also applicable to this embodiment, and will not be described again here.

[0100] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that, in alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0101] In addition, the functional modules or units in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0102] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a smartphone, personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0103] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A method for labeling multi-classified face images, characterized in that, include: Obtain at least one face image and related information for the corresponding face image; The relevant information includes the image address corresponding to the face image and the ID of the person associated with the image, wherein the image address is the geographical location of the object appearing in the image; All the face images are grouped according to the relevant information and preset grouping rules; All the face images in each group are combined into a single group master image; Based on a preset tag library and preset tag rules, each face image in each group of main images is labeled with a corresponding tag. The preset tag library contains multiple tags, and each tag also contains corresponding data. After obtaining at least one face image and the corresponding information about the face image, the process further includes: Establish a master image library corresponding to each of the aforementioned face images; Establish a relationship table between the main image of each group and the overall image library; After labeling each face image in each of the grouped main images with a corresponding tag, the method further includes: Based on the association table, the tag is updated to the corresponding face image in the image library.

2. The multi-class face image annotation method according to claim 1, characterized in that, The step of updating the tag to the corresponding face image in the image library includes: When the face image in the image library does not have a corresponding tag, the currently labeled tag is added to the corresponding face image; When the face image in the image library already has a corresponding tag, the currently labeled tag will overwrite the existing tag.

3. The multi-class face image annotation method according to claim 1, characterized in that, The facial images can be grouped according to the relevant information in one of the following ways: Method 1: Group the images according to the IDs of the people associated with them; Method 2: Group the images according to their URLs; Method 3: Group the images according to the IDs of the people associated with them and the addresses of the images.

4. The multi-class face image annotation method according to claim 1, characterized in that, The step of labeling each face image in each group of main images with a corresponding label based on a preset label library and preset label rules includes: Based on the preset tag library and the preset tag rules, each face image in each group of main images is labeled with corresponding tags in sequence.

5. The multi-class face image annotation method according to claim 1, characterized in that, Each of the aforementioned face images corresponds to at least one of the aforementioned tags.

6. A multi-class image annotation system, characterized in that, include: The acquisition unit is used to acquire at least one face image and related information of the corresponding face image; The relevant information includes the image address corresponding to the face image and the ID of the person associated with the image, wherein the image address is the geographical location of the object appearing in the image; A grouping unit is used to group all the face images according to the relevant information and preset grouping rules; An integration unit is used to integrate all the face images in each group into a single group master image; And to establish a total image library corresponding to each of the aforementioned face images; Establish a relationship table between the main image of each group and the overall image library; The annotation unit is used to annotate each face image in each group of main images with a corresponding label based on a preset label library and preset label rules. The preset label library contains multiple labels, and each label also contains corresponding data. Based on the association table, the label is updated to the corresponding face image in the total image library.

7. A computer, characterized in that, It includes a storage unit and a processing unit. The storage unit stores a computer program, and the processing unit executes the steps of the multi-class face image annotation method as described in any one of claims 1 to 5 by calling the computer program stored in the storage unit.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program adapted for loading by a processor to perform the steps of the multi-class face image annotation method as described in any one of claims 1 to 5.

Images