A method, device and medium for identifying archive objects

Abstract

The present application proposes a method for identifying file objects, including: obtaining files to be identified, each of which includes multiple facial images; searching an object library based on the class center feature vector of the file to be identified to obtain a candidate object, wherein the class center feature vector is used to characterize the average vector of the feature vectors of multiple facial images in the file to be identified, and the candidate object includes a single facial image; obtaining multi-dimensional feature information of the facial images in the file to be identified and the candidate object, as well as cross-feature information between the two; inputting the multi-dimensional feature information and cross-feature information into a trained machine learning model, and judging whether the file to be identified and archived as the candidate object based on the output result of the machine learning model. The file object identification method of the present application integrates feature information of more dimensions, and the object information of the file is more accurate, thereby improving the real-name rate of the file.

Description

Technical Field

[0001] The present application relates to the technical field of image processing, and in particular to a method, device and medium for identifying archive objects. Background Art

[0002] Image archiving is a crucial technology in the security field. It first uses a deep neural network to extract image features from IoT sensor images. Then, images are clustered based on these features. The clustering results are then compared with the archive. If a match is not found, a new archive is created; if a match is found, the archive is merged into the existing archive. For archived files, feature comparison is often performed against an object library (typically consisting of clear, front-facing photos of the object). Files with a similarity that meets a preset threshold are annotated, effectively identifying the object to which the file belongs.

[0003] An archive is a collection of multiple images. Applications often use the average vector of multiple image feature vectors as the central feature of the archive class, or use the most representative image (e.g., the one with the highest quality score) as the representative image, to convert archive features into image features. Existing technologies generally compare archive features with object features. If the similarity reaches a preset threshold, it is considered a match, and the object to which the archive belongs is identified. This method generally has a low match rate, resulting in a large number of anonymous files in the archive, that is, a low real-name rate. Summary of the Invention

[0004] In order to solve the technical problem of low accuracy in file recognition in the prior art, the present application proposes a file object recognition method, device and medium.

[0005] According to a first aspect of the present application, a method for identifying an archive object is proposed, comprising:

[0006] Obtaining files to be identified, each of which includes multiple facial images;

[0007] Searching an object library according to a class center feature vector of the to-be-recognized file to obtain a candidate object, wherein the class center feature vector is used to represent an average vector of feature vectors of multiple facial images in the to-be-recognized file, and the candidate object includes a single facial image;

[0008] Obtaining multi-dimensional feature information of the facial images in the to-be-identified file and the candidate object, as well as cross-feature information between the two;

[0009] The multi-dimensional feature information and cross-feature information are input into a trained machine learning model, and it is determined whether the file to be identified is identified and archived as the candidate object based on the output result of the machine learning model.

[0010] Preferably, the step of searching an object library based on the class center feature vector of the to-be-identified file to obtain a candidate object comprises:

[0011] Searching an object library according to the class center feature vector of the to-be-identified file to obtain the candidate objects corresponding to different similarities;

[0012] It is determined whether the maximum similarity among the candidate objects reaches a first preset threshold; if not, the identification of the to-be-identified file is terminated.

[0013] Preferably, the multi-dimensional features of the facial images in the to-be-identified files and the candidate objects include:

[0014] The age, gender, whether wearing a mask, average quality score, cohesion, number of images, and number of days from the last appearance of the facial images in the to-be-identified file, where the cohesion is used to represent the average value of the similarity between the feature vectors of any two facial images in the to-be-identified file; and

[0015] The age, gender, and number of days from the present time of the facial image of the candidate object with the maximum similarity.

[0016] Preferably, the cross-feature information of the facial images in the to-be-identified file and the candidate object includes:

[0017] The gender consistency, age consistency and similarity change pattern of the facial image in the to-be-identified file and the facial image in the candidate object with the maximum similarity, based on the search object library, wherein the change pattern of the similarity of the search results specifically includes: the maximum similarity in the candidate objects, the reduction rate of the maximum similarity relative to the secondary similarity, and the average value and standard deviation of the difference between the maximum similarity and the secondary similarity.

[0018] Preferably, the feature extraction method of the change pattern of the similarity of the search results includes at least one of the following:

[0019] The similarity of the search results of the search object library is taken as a sequence, and a sequence model in deep learning is used to automatically extract features;

[0020] The similarity of the retrieval results of the retrieval object library is used as a graph structure network, and a graph neural network is used to extract graph-level features.

[0021] Preferably, the method for searching the object library includes at least one of the following:

[0022] Vector engine search, similar vector retrieval.

[0023] Preferably, judging whether the to-be-identified file is filed as the candidate object according to the output result of the machine learning model includes:

[0024] The machine learning model outputs a confidence level that the file to be identified and the candidate object with the greatest similarity belong to the same target. If the confidence level reaches a second preset threshold, the file to be identified is identified as the candidate object with the greatest similarity.

[0025] Preferably, the method further comprises:

[0026] The candidate object after identification and archiving is updated to the archive library, and at the same time, it is checked whether there are other files corresponding to the candidate object in the archive library. If so, a merge update is performed.

[0027] According to a second aspect of the present application, a device for identifying an archive object is provided, comprising:

[0028] An acquisition module configured to acquire files to be identified, each of which includes multiple facial images;

[0029] a retrieval module configured to retrieve an object library based on a class center feature vector of the to-be-identified file to obtain a candidate object, wherein the class center feature vector is used to represent an average vector of feature vectors of multiple facial images in the to-be-identified file, and the candidate object includes a single facial image;

[0030] A feature extraction module configured to obtain multi-dimensional feature information of the face image in the to-be-identified file and the candidate object, as well as cross-feature information between the two;

[0031] The prediction module is configured to input the multi-dimensional feature information and the cross-feature information into a trained machine learning model, and determine whether the file to be identified is identified and archived as the candidate object based on the output result of the machine learning model.

[0032] According to a third aspect of the present application, a computer-readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the file object identification method as described in the first aspect of the present application is implemented.

[0033] This application proposes a method, device and medium for archival object recognition. By characterizing the feature information of multiple dimensions of the archive to be identified and the facial images of the retrieved candidate objects, as well as the cross-feature information of the two, a machine learning model is used to give the confidence that the archive to be identified and the candidate object belong to the same target. Compared with traditional solutions, it integrates feature information of more dimensions, which can make the object information of the archive more accurate and improve the real-name rate of the archive. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The accompanying drawings are included to provide a further understanding of the embodiments and are incorporated into and constitute a part of this specification. The accompanying drawings illustrate the embodiments and, together with the description, serve to explain the principles of the present application. Other embodiments and many of the expected advantages of the embodiments will be readily apparent as they become better understood by reference to the following detailed description. The elements of the drawings are not necessarily to scale with respect to each other. Like reference numerals designate corresponding similar parts.

[0035] Figure 1 is a flow chart of a method for identifying an archive object according to an embodiment of the present application;

[0036] Figure 2 1 is a schematic diagram of a search result of a class center feature vector search object library of a file to be identified according to a specific embodiment of the present application;

[0037] Figure 3 4 is a structural block diagram of a file object identification device according to an embodiment of the present application. DETAILED DESCRIPTION

[0038] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and Examples. It should be understood that the specific embodiments described herein are only configured to explain the present application and are not configured to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is merely to provide a better understanding of the present application by illustrating the examples of the present application.

[0039] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, the elements defined by the phrase "comprising..." do not exclude the presence of additional identical elements in the process, method, article, or device comprising the elements.

[0040] According to a first aspect of the present application, a method for identifying archive objects is proposed. Figure 1 A flow chart of a method for identifying an archive object according to an embodiment of the present application is shown in FIG. Figure 1As shown, the method includes the following steps:

[0041] S101: Obtain files to be identified, each file to be identified includes multiple facial images.

[0042] In a specific embodiment, the acquisition of files to be identified is performed offline daily in actual projects. Due to limited computing resources, the issue of which files should be prioritized arises. In this embodiment, the criteria for acquiring files are as follows: 1. Newly generated files are prioritized; 2. Files that have not undergone object identification are prioritized; 3. Files with the earliest failed object identification are prioritized; and 4. Files with the highest average image quality scores are prioritized.

[0043] S102: Searching an object library based on the class center feature vector of the file to be identified to obtain a candidate object, wherein the class center feature vector is used to represent the average vector of feature vectors of multiple facial images in the file to be identified, and the candidate object includes a single facial image.

[0044] In a specific embodiment, the file to be identified is a collection of multiple facial images. Therefore, the average vector of the feature vectors of these multiple facial images is used as the class center feature vector of the file to be identified. The class center feature vector of the file to be identified is searched in the object library using mainstream vector search methods such as vector engine search (milvus) and similarity vector search (faiss).

[0045] Figure 2 FIG. 1 shows a schematic diagram of a search result of a class center feature vector search object library of a to-be-identified file according to a specific embodiment of the present application, as shown in FIG. Figure 2 As shown, in a specific embodiment, when searching the object library based on the class center feature vector of the file to be identified, the search results are output as multiple candidate objects with different similarity levels. When the maximum similarity among the candidate objects reaches a first preset threshold, the identification process of the file to be identified is continued, otherwise the identification is terminated and it is considered that the identification of the file to be identified has failed and the corresponding candidate object cannot be found in the object library.

[0046] S103: Acquire multi-dimensional feature information of the facial images in the to-be-recognized file and the candidate object, as well as cross-feature information between the two.

[0047] In a specific embodiment, the multi-dimensional features of the facial images in the identification file include, but are not limited to, age, gender, whether or not a mask is worn, average quality score, cohesion, number of images, and the number of days since the last appearance. Cohesion represents the average similarity of the feature vectors of any two facial images in the identification file.

[0048] In a specific embodiment, a candidate with the greatest similarity to the profile to be identified is selected to calculate multi-dimensional feature information of the facial image, including but not limited to: age, gender, and the number of days from the time the image was taken to the present.

[0049] In a specific embodiment, cross-feature information between facial images in the profile to be identified and the candidate with the greatest similarity includes, but is not limited to, gender consistency and age consistency of the facial images, and the variation pattern of similarity in search results based on the search object database. The variation pattern of similarity in search results specifically includes the maximum similarity among the candidate objects, the magnitude of the decrease in the maximum similarity relative to the secondary similarity, and the average and standard deviation of the difference between the maximum similarity and the secondary similarity.

[0050] in:

[0051] Cohesion calculation: Assume that the facial image feature vector in the file to be identified is x=(x1,x2,…,x n ), then cohesion can be transformed into the inner product of x and x, that is,

[0052] The variation pattern of similarity of retrieval results: Assume that the similarities of all candidate objects retrieved from the object library are sorted from large to small as s1, s2, ..., s n , then the reduction of s2 relative to s1 is The average value m of the difference between s2-s4 and s1 is The standard deviation is

[0053] It should be noted that in this embodiment, the top four similarities with the largest values ​​among the search result similarities are selected. The reduction in the maximum similarity relative to the secondary similarities is calculated based on the reduction in similarity s1 relative to s2. The average and standard deviation of the difference between the maximum similarity and the secondary similarities are calculated based on s1 and s2-s4. In other implementations, different similarities can be selected for calculation based on actual needs.

[0054] In a specific embodiment, the feature extraction method of the changing law of the similarity of the retrieval results can be to treat the similarity of the retrieval results of the retrieval object library as a sequence, and then automatically extract features using the sequence model in deep learning; or it can be to treat the similarity of the retrieval results of the retrieval object library as a graph structure network, and then use the graph neural network to extract graph-level features; or it can be features manually constructed based on experience or data analysis.

[0055] S104: Input the multi-dimensional feature information and the cross-feature information into the trained machine learning model, and determine whether the file to be identified is identified and archived as a candidate object based on the output result of the machine learning model.

[0056] In a specific embodiment, a pre-trained machine learning model is used as input, along with the multi-dimensional feature information of the facial images of the profile to be identified and the candidate object, as well as the cross-feature information between the two. The machine learning model outputs a confidence level that the profile to be identified and the candidate object with the greatest similarity belong to the same target. If the confidence level reaches a second preset threshold, the profile to be identified is identified as the candidate object with the greatest similarity. The pre-trained machine learning model can be constructed using an ensemble learning model, such as a random forest, xgboost, or lightgbm, to establish a classification model. The second preset threshold can be determined based on the performance of the machine learning model on a validation set during offline training. In practical scenarios, precision is prioritized, requiring the highest recall possible with a high precision. Therefore, the probability threshold at which F0.5 or F1 reaches its maximum value, while maintaining a precision constraint of no less than 99%, is recommended as the second preset threshold.

[0057] S105: The candidate object after identification and archiving is updated into the archive library, and at the same time, it is checked whether there are other archives corresponding to the candidate object in the archive library. If so, a merge update is performed.

[0058] In a specific embodiment, the files to be identified whose confidence reaches a second preset threshold are identified as candidate objects with the maximum similarity, and the identified and archived candidate objects are updated to the archive library. At the same time, it is necessary to verify whether there is a file corresponding to the candidate object in the archive library. If so, multiple files of the same object need to be merged and updated.

[0059] In summary, the file object recognition method of this application uses a machine learning model to give the confidence that the file to be identified and the candidate object belong to the same target by characterizing the multi-dimensional feature information of the file to be identified and the facial image of the retrieved candidate object, as well as the cross-feature information of the two, so as to judge whether the two are successfully identified and archived. Compared with traditional solutions, the integration of more dimensional feature information can make the object information of the file more accurate and improve the real-name rate of the file.

[0060] According to the second aspect of the present application, based on the same concept, the embodiment of the present application also proposes a file object identification device. Figure 3 FIG. 1 shows a structural block diagram of a file object identification device according to an embodiment of the present application. Figure 3 As shown, the device includes:

[0061] An acquisition module 201 is configured to acquire files to be identified, each file to be identified including multiple facial images;

[0062] A retrieval module 202 is configured to search an object library based on a class center feature vector of a file to be identified to obtain a candidate object, wherein the class center feature vector is used to represent an average vector of feature vectors of multiple facial images in the file to be identified, and the candidate object includes a single facial image;

[0063] The feature extraction module 203 is configured to obtain multi-dimensional feature information of the facial images in the to-be-identified profile and the candidate object, as well as cross-feature information between the two;

[0064] The prediction module 204 is configured to input the multi-dimensional feature information and the cross-feature information into the trained machine learning model, and determine whether the file to be identified is identified and archived as a candidate object based on the output result of the machine learning model.

[0065] In a preferred embodiment, the device further comprises:

[0066] The updating module 205 is configured to update the candidate object after identification and archiving into the archive library, and at the same time check whether there is other archive corresponding to the candidate object in the archive library. If so, merge and update.

[0067] According to the third aspect of the present application, based on the same concept, the embodiment of the present application further proposes a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, it implements the archive object recognition as the first aspect of the present application.

[0068] In the embodiments of the present application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device / system / method embodiments described above are merely schematic. For example, the division of the units can be a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0069] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0070] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0071] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0072] Obviously, those skilled in the art can make various modifications and changes to the embodiments of the present application without departing from the spirit and scope of the present application. In this way, if these modifications and changes are within the scope of the claims of the present application and their equivalents, the present application is also intended to cover these modifications and changes. The word "comprising" does not exclude the presence of other elements or steps not listed in the claims. The simple fact that certain measures are recited in mutually different dependent claims does not indicate that the combination of these measures cannot be used to advantage. Any reference signs in the claims should not be considered as limiting the scope.

Claims

1. A method for identifying an archive object, characterized in that: include: Obtaining files to be identified, each of which includes multiple facial images; Searching an object library according to a class center feature vector of the to-be-recognized file to obtain a candidate object, wherein the class center feature vector is used to represent an average vector of feature vectors of multiple facial images in the to-be-recognized file, and the candidate object includes a single facial image; Obtaining multi-dimensional feature information of the facial images in the to-be-identified file and the candidate object, as well as cross-feature information therebetween, wherein the multi-dimensional features of the facial images in the to-be-identified file and the candidate object include: The age, gender, whether wearing a mask, average quality score, cohesion, number of images, and number of days from the last appearance of the facial images in the to-be-identified file, where the cohesion is used to represent the average value of the similarity between the feature vectors of any two facial images in the to-be-identified file; and The age, gender, and number of days from the present time of the facial image of the candidate with the greatest similarity; Cross-feature information of the facial images in the to-be-identified file and the candidate objects, including: gender consistency and age consistency between the facial image in the to-be-identified file and the facial image in the candidate object with the greatest similarity, and a variation pattern of similarity of retrieval results based on the retrieval object library, wherein the variation pattern of similarity of retrieval results specifically includes: a decrease in the maximum similarity relative to the secondary similarity, and an average and standard deviation of the difference between the maximum similarity and the secondary similarity; The multi-dimensional feature information and cross-feature information are input into a trained machine learning model, and it is determined whether the file to be identified is identified and archived as the candidate object based on the output result of the machine learning model.

2. The method according to claim 1, characterized in that The step of searching an object library according to the class center feature vector of the to-be-identified file to obtain a candidate object comprises: Searching an object library according to the class center feature vector of the to-be-identified file to obtain the candidate objects corresponding to different similarities; It is determined whether the maximum similarity among the candidate objects reaches a first preset threshold; if not, the identification of the to-be-identified file is terminated.

3. The method according to claim 2, characterized in that The variation rule of the similarity of the search results specifically includes: the maximum similarity among the candidate objects.

4. The method according to claim 3, characterized in that The feature extraction method of the change law of the similarity of the search results includes at least one of the following: The similarity of the search results of the search object library is taken as a sequence, and a sequence model in deep learning is used to automatically extract features; The similarity of the retrieval results of the retrieval object library is used as a graph structure network, and a graph neural network is used to extract graph-level features.

5. The method according to claim 1, wherein The method for retrieving the object library includes at least one of the following: Vector engine search, similar vector retrieval.

6. The method according to claim 2, characterized in that Determining whether the to-be-identified file is filed as the candidate object according to the output result of the machine learning model includes: The machine learning model outputs a confidence level that the file to be identified and the candidate object with the greatest similarity belong to the same target. If the confidence level reaches a second preset threshold, the file to be identified is identified as the candidate object with the greatest similarity.

7. The method according to claim 1, characterized in that The method further comprises: The candidate object after identification and archiving is updated to the archive library, and at the same time, it is checked whether there are other files corresponding to the candidate object in the archive library. If so, a merge update is performed.

8. A file object recognition device, characterized in that: include: An acquisition module configured to acquire files to be identified, each of which includes multiple facial images; a retrieval module configured to retrieve an object library based on a class center feature vector of the to-be-identified file to obtain a candidate object, wherein the class center feature vector is used to represent an average vector of feature vectors of multiple facial images in the to-be-identified file, and the candidate object includes a single facial image; A feature extraction module is configured to obtain multi-dimensional feature information of the facial images in the to-be-identified file and the candidate object, as well as cross-feature information therebetween, wherein the multi-dimensional features of the facial images in the to-be-identified file and the candidate object include: The age, gender, whether wearing a mask, average quality score, cohesion, number of images, and number of days from the last appearance of the facial images in the to-be-identified file, where the cohesion is used to represent the average value of the similarity between the feature vectors of any two facial images in the to-be-identified file; and The age, gender, and number of days from the present time of the facial image of the candidate with the greatest similarity; Cross-feature information of the facial images in the to-be-identified file and the candidate objects, including: gender consistency and age consistency between the facial image in the to-be-identified file and the facial image in the candidate object with the greatest similarity, and a variation pattern of similarity of retrieval results based on the retrieval object library, wherein the variation pattern of similarity of retrieval results specifically includes: a decrease in the maximum similarity relative to the secondary similarity, and an average and standard deviation of the difference between the maximum similarity and the secondary similarity; The prediction module is configured to input the multi-dimensional feature information and the cross-feature information into a trained machine learning model, and determine whether the file to be identified is identified and archived as the candidate object based on the output result of the machine learning model.

9. A computer-readable storage medium storing a computer program, wherein the computer program implements the method according to any one of claims 1 to 7 when executed by a processor.

Images