A multi-dimensional feature fusion personnel identification method and system

Abstract

The application discloses a multi-dimensional feature fusion personnel identification method and system, the method comprises the following steps: S100: a target personnel image library is established through video tracking analysis, and face features, body features and face attributes of the target personnel image library are identified to obtain a target personnel comprehensive feature library; S200: a to-be-detected image is collected through a camera; S300: face detection features, body detection features and body detection attributes of each to-be-matched personnel in the to-be-detected image are extracted; S400: similarity analysis is performed on a face base feature set and a body base feature set to obtain a face base target set and a body base target set; S500: a to-be-selected target personnel and a comprehensive score of the to-be-selected target personnel are obtained; S600: a body attribute result of the to-be-selected target personnel is obtained; and S700: an identification result is output according to the comprehensive score of the to-be-selected target personnel and the body attribute result. The personnel is identified with high precision from multiple dimensions of face features, body features and body attributes.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of personnel identification, and in particular to a multi-dimensional feature fusion personnel identification method and system. BACKGROUND

[0002] Biometric technology mainly refers to a technology of identity authentication through human biological characteristics. Human biological characteristics usually have the characteristics of uniqueness, measurability or automatic identification and verification, heredity or lifelong invariability, and therefore, the biometric authentication technology has greater advantages than the traditional authentication technology. The biometric system samples biological characteristics, extracts unique features thereof and converts them into digital codes, and further forms feature templates from the codes. Due to the continuous reduction in the cost of microprocessors and various electronic components and the gradual improvement in precision, the biometric technology is gradually applied to access control, enterprise attendance management system and security authentication field. The biological characteristics used for biometric identification include hand shape, fingerprint, face shape, iris, retina, pulse, ear, etc., and the behavioral characteristics include signature, voice, key force, etc. Based on these characteristics, people have developed various biometric technologies such as hand shape recognition, fingerprint recognition, face recognition, voice recognition, iris recognition and signature recognition.

[0003] At present, in order to improve the accuracy of personnel identification, multiple dimensions of biological characteristics are usually used for high-precision identification of personnel.

[0004] For example, the Chinese invention patent with the patent publication number CN109426777A and the name "a multi-directional identity recognition system" is composed of an information collection unit 1, a feature extraction unit 2, an original database 3, a WeChat user database 4, a data comparison unit 5, a decision unit 6 and a security system 7. The information collection unit 1 can simultaneously collect the body information, face information and voice information of a person within a certain range; the feature extraction unit 2 is internally provided with a core processor, which can extract, classify and refine the feature information from the information collection unit 1; the original database 3 is a task feature information database provided by the public security system and will be updated regularly; the WeChat user database 4 is bound by the WeChat user himself and the body, face and voice information of the user is recorded. The data comparison unit 5 is composed of a large-capacity storage and a central processor, which connects the original database 3 and the WeChat user database 4, can store the refined information from the feature extraction unit 2, and compare the stored information with the information of the original database 3 and the WeChat user database 4, and then transmit the compared information to the decision unit 6. The present scheme improves the accuracy of identity recognition by using body recognition, face recognition and voice recognition technologies, but with the progress of deep fake technology, deep fake voice will become more realistic and more difficult to detect.

[0005] For example, the patent with the patent number "CN109508524 A" and the name "authentication method, system and storage medium" discloses obtaining the identification information of the target user, and the identification information can also be the user's ID card number or the card identification of the smart card, etc. According to the identification information, target character information is obtained, and the target character information includes at least one of gender, age, height, weight and character image. The character information collected by the information collection terminal is obtained. It is detected whether the collected character information matches the target character information. If the collected character information matches the target character information, the target user is authenticated. In the scheme, when two or more kinds of character information are included in the character information, the preset proportion of the character information is matched, and it is considered that the collected character information matches the obtained target character information. For example, if four kinds of character information are included in the character information, at least three kinds of character information are matched with each other, and it is considered that the collected character information matches the obtained target character information. The scheme obtains the identification information of the user, obtains the target character information according to the identification information, and then determines that the target user is authenticated only when the collected character information matches the obtained target character information, which greatly improves the recognition efficiency. However, only the identification information is used for multi-character target analysis, and since the identification information cannot be updated in real time, the identification information is not accurate, and the recognition accuracy is reduced. SUMMARY

[0006] The present application aims to overcome the shortcomings of the prior art and provides a multi-dimensional feature fusion personnel identification method and system.

[0007] In a first aspect, the present application provides a multi-dimensional feature fusion personnel identification method, which comprises:

[0008] S100: A target personnel image library is established by video tracking analysis, each target personnel in the target personnel image library corresponds to a unique number, and the target personnel image library is identified for face features, body features and face attributes to obtain a target personnel comprehensive feature library, the target personnel comprehensive feature library includes a face base feature set, a body base feature set and a body base attribute, each feature in the face base feature set and the body base feature set is associated with the corresponding target personnel number, and the body base attribute is associated with the corresponding target personnel number;

[0009] S200: An image to be detected is collected by a camera;

[0010] S300: Face detection features, body detection features and body detection attributes corresponding to each matched personnel in the image to be detected are extracted;

[0011] S400: Perform similarity analysis on the face detection feature and the body detection feature of each to-be-matched person respectively with the face base feature set and the body base feature set in the target person comprehensive feature library, obtain a face base target set from the face base feature set, and obtain a body base target set from the body base feature set;

[0012] The face base target set includes face similarity scores of target persons corresponding to face features with top R similarities and numbers of each target person;

[0013] The body base target set includes body similarity scores of target persons corresponding to body features with top R similarities and numbers of each target person;

[0014] S500: Obtain a to-be-selected target person and a comprehensive score of the to-be-selected target person according to the face base target set and the body base target set;

[0015] S600: Obtain a body attribute of the to-be-selected target person from the body base attribute, analyze a matching result of the body attribute of the to-be-selected target person and a body detection attribute of a corresponding to-be-matched person, and obtain a body attribute result of the to-be-selected target person;

[0016] When the body attribute of the to-be-selected target person is consistent with the body detection attribute of the corresponding to-be-matched person, the body attribute result of the to-be-selected target person is consistent;

[0017] When the body attribute of the to-be-selected target person is inconsistent with the body detection attribute of the corresponding to-be-matched person, the body attribute result of the to-be-selected target person is inconsistent;

[0018] S700: Output a recognition result according to the comprehensive score of the to-be-selected target person and the body attribute result.

[0019] Further, the face base feature set includes face features of different target persons at different angles from -90° to 90°, and the face features include 512 face feature points;

[0020] The body base feature set includes body features of different target persons at different angles from -90° to 90°, and the body features include 512 body feature points;

[0021] The body base attribute is a body attribute that appears most frequently at different angles from -90° to 90° of different target persons.

[0022] Further, the S100 specifically includes:

[0023] S110: Real-time access to the video stream through the video acquisition device, continuously track the target personnel in the video stream through the human body tracking algorithm, create a database for the newly appearing target personnel and generate a unique number for it, continuously output the image corresponding to the target personnel, and obtain the target personnel image library;

[0024] S120: Recognize the face features and face angles in each image in the target personnel image library, extract face features at different angles, and obtain the face base library feature set;

[0025] S130: Recognize the body features and body angles in each image in the target personnel image library, extract body features at different angles, and obtain the body base library feature set;

[0026] S140: Recognize the body attributes in each image in the target personnel image library, and recognize the face angles or body angles in each image in the target personnel image library, extract the most frequently appearing body attributes from the body attributes corresponding to different face angles or body angles, and obtain the body base library attributes.

[0027] Further, in the S140, the face angles in each image in the target personnel image library are recognized, and the body attributes are gender.

[0028] Further, the S100 further comprises periodically obtaining the continuously output images of the target personnel in the S110, to realize the updating of the images in the target personnel image library.

[0029] Further, the step of obtaining the face base library target set comprises:

[0030] First step: Calculate the face feature difference value of all to-be-matched personnel corresponding to the face detection features in the to-be-detected image and all features in the face base library feature set, to obtain a face feature difference value matrix;

[0031] The number of all to-be-matched personnel in the detection image is M, and the number of all features in the face base library feature set is N. By using the matrix broadcast operation characteristic, a M×N order face feature difference value matrix is obtained;

[0032] Second step: Perform distance minimum value operation on each row of the face feature difference value matrix, extract R minimum values from each row of the face feature difference value matrix, and obtain a face minimum distance matrix;

[0033] Third step: According to the number of the target personnel corresponding to the features in the face base library feature set corresponding to each minimum distance in the face minimum distance matrix, obtain the number of the target personnel corresponding to the face features with the top R similarities;

[0034] Step 4: Normalizing the minimum distance matrix of the face to convert each distance into a score between 0 and 1 to obtain the face similarity score of the target person corresponding to the top R face features.

[0035] Further, the face detection features of all the to-be-matched persons in the to-be-detected image are B, wherein the face feature of the kth face is β k , wherein k = [1, 2, 3…M], representing the hth feature point of the kth face, h = [1, 2, 3…512];

[0036] The face feature set of the face database is A, A = [α1, α2, α3......α N ], wherein the ith face feature is a i , wherein i = [1, 2, 3…N], representing the jth feature point of the ith face feature, j = [1, 2, 3…512];

[0037] The calculation formula of the face feature difference d ki is as follows:

[0038]

[0039] The M×N order face feature difference matrix D1 is obtained by using the matrix broadcast operation characteristics:

[0040]

[0041] Further, the step of obtaining the target set of the human body database comprises:

[0042] Step 1: Calculating the human body feature difference between the human body detection features of all the to-be-matched persons in the to-be-detected image and all the features in the human body feature set to obtain a human body feature difference matrix;

[0043] The number of all the to-be-matched persons in the detection image is M, and the number of all the features in the human body feature set is N. The M×N order human body feature difference matrix is obtained by using the matrix broadcast operation characteristics.

[0044] Step 2: Obtaining the minimum distance of each row of the human body feature difference matrix to extract R minimum values from each row of the human body feature difference matrix to obtain a human body minimum distance matrix;

[0045] Step 3: Obtaining the number of the target person corresponding to the feature in the human body database feature set corresponding to each minimum distance in the human body minimum distance matrix to obtain the number of the target person corresponding to the top R human body features.

[0046] The fourth step is to normalize the minimum distance matrix of the human body, convert each distance into a score between 0 and 1, and obtain the human body similarity score of the target person corresponding to the first R human feature similarities.

[0047] Further, the value of R is 20.

[0048] Further, the S500 comprises:

[0049] S510: obtaining the number of face databases T1 corresponding to different numbers according to the target set of the face database, and obtaining the number of human body databases T2 corresponding to different numbers according to the target set of the human body database;

[0050] The number of face databases T1 corresponding to different numbers is the number of different numbers corresponding to the target set of the face database;

[0051] The number of human body databases T2 corresponding to different numbers is the number of different numbers corresponding to the target set of the human body database;

[0052] S520: obtaining the intersection number and the face similarity score and the human body similarity score corresponding to the intersection number from the intersection of the target set of the face database and the target set of the human body database;

[0053] S530: calculating the comprehensive score of the target person corresponding to the intersection number, returning at least two target persons with the highest score, and obtaining the candidate target person and the comprehensive score of the candidate target person.

[0054] Further, in the S530, five target persons with the highest score are returned.

[0055] Further, the calculation formula of the comprehensive score of the target person corresponding to the intersection number is:

[0056] The comprehensive score of the target person corresponding to the intersection number = the highest score of the face similarity score corresponding to the intersection number × (1+0.1× the number of face databases T1 corresponding to the intersection number) + the highest score of the human body similarity score corresponding to the intersection number × (1+0.1× the number of human body databases T2 corresponding to the intersection number).

[0057] Further, the S700 comprises:

[0058] S710: judging the comprehensive score of the candidate target person and the absolute hit threshold;

[0059] If the comprehensive score of a single candidate target person exceeds the threshold, output the single person hit result;

[0060] If the comprehensive scores of multiple candidate target persons exceed the threshold, proceed to the next step.

[0061] S720: judging the human body attribute result of the multiple candidate target persons whose scores exceed the threshold value;

[0062] if the human body attribute results of the single candidate target person are consistent, outputting a single person hit result;

[0063] if the human body attribute results of the multiple candidate target persons are consistent, outputting the candidate target person with the highest comprehensive score in the multiple candidate target persons as a hit result.

[0064] In a second aspect, the present application further provides a personnel identification system based on multi-dimensional feature fusion, comprising:

[0065] a target person comprehensive feature library establishing module, a to-be-detected image collecting module, a to-be-matched personnel multi-dimensional feature extracting module, a face feature and body feature similarity analyzing module, a candidate target person comprehensive score calculating module, a candidate target person human body attribute result analyzing module, and a personnel identification result judging module;

[0066] The target person comprehensive feature library establishing module establishes a target person image library through video tracking analysis, each target person in the target person image library corresponds to a unique number, and the target person image library is identified in terms of face features, body features, and face attributes to obtain a target person comprehensive feature library, which includes a face base feature set, a body base feature set, and body base attributes. The face base feature set and the body base feature set are respectively associated with the corresponding target person numbers, and the body base attributes are associated with the corresponding target person numbers.

[0067] The to-be-detected image collecting module collects to-be-detected images through a camera.

[0068] The to-be-matched personnel multi-dimensional feature extracting module is used to extract face detection features, body detection features, and body detection attributes of each to-be-matched person in the to-be-detected images.

[0069] The face feature and body feature similarity analyzing module respectively analyzes the similarity of the face detection features and the body detection features of each to-be-matched person with the face base feature set and the body base feature set in the target person comprehensive feature library to obtain a face base target set from the face base feature set and a body base target set from the body base feature set.

[0070] The face base target set includes face similarity scores of target persons corresponding to the top R face features in terms of similarity and the numbers of each target person.

[0071] The human body base target set includes human body similarity scores of target personnel corresponding to human body features with a similarity of R and the number of each target personnel;

[0072] The candidate target personnel comprehensive score calculation module obtains the candidate target personnel and the comprehensive score of the candidate target personnel according to the face base target set and the human body base target set.

[0073] The candidate target personnel human body attribute result analysis module obtains the human body attribute of the candidate target personnel from the human body base attribute, analyzes the matching result of the human body attribute of the candidate target personnel and the human body detection attribute of the corresponding to-be-matched personnel, and obtains the human body attribute result of the candidate target personnel.

[0074] When the human body attribute of the candidate target personnel is consistent with the human body detection attribute of the corresponding to-be-matched personnel, the human body attribute result of the candidate target personnel is consistent.

[0075] When the human body attribute of the candidate target personnel is inconsistent with the human body detection attribute of the corresponding to-be-matched personnel, the human body attribute result of the candidate target personnel is inconsistent.

[0076] The personnel identification result judgment module outputs an identification result according to the comprehensive score of the candidate target personnel and the human body attribute result.

[0077] The present application has the following advantages:

[0078] (1) A multi-dimensional feature fusion personnel identification method, which performs high-precision matching and identification on personnel from multiple dimensions of face features, human body features and human body attributes, establishes a face base feature set, a human body base feature set and a human body attribute set of a human body at different angles from -90° to 90° for different target personnel, and performs feature extraction from multiple angles for multi-dimensional features, thereby realizing accurate matching and identification of complex personnel information, regularly obtaining images corresponding to target personnel, updating images in the target personnel image library, and continuously updating and iterating multi-dimensional features in the base library to improve accuracy.

[0079] (2) The face detection features and the human body detection features in the to-be-detected image are obtained, similarity matching and identification are performed between the face detection features, the human body detection features and the face base feature set and the human body base feature set, the target personnel with high similarity is extracted, comprehensive score analysis is performed on the target personnel with high similarity, the candidate target personnel is obtained, and the only highly matched target personnel is output according to the comprehensive score of the candidate target personnel and the human body attribute matching condition, thereby solving the problems of low identification efficiency and low accuracy in the prior art, achieving automatic identification, and improving personnel identification efficiency and accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0080] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description only constitute a part of the embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of the provided drawings.

[0081] Figure 1 It is a flowchart of a multi-dimensional feature fusion personnel identification method.

[0082] Figure 2 It is a flowchart of S100.

[0083] Figure 3 It is a step of obtaining a face base target set.

[0084] Figure 4 It is a flowchart of S500.

[0085] Figure 5 It is a flowchart of S700.

[0086] Figure 6 It is a block diagram of a multi-dimensional feature fusion personnel identification system. DETAILED DESCRIPTION

[0087] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments only constitute a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0088] The embodiments of the present application provide a multi-dimensional feature fusion personnel identification method and system. Specifically, referring to Figure 1 , the method comprises:

[0089] S100: establishing a target personnel image library through video tracking analysis, each target personnel in the target personnel image library corresponding to a unique number, identifying the target personnel image library for face features, body features and face attributes, obtaining a target personnel comprehensive feature library, the target personnel comprehensive feature library including a face base feature set, a body base feature set and a body base attribute, respectively associating each feature in the face base feature set and the body base feature set with the corresponding target personnel number, and associating the body base attribute with the corresponding target personnel number;

[0090] S200: collecting a to-be-detected image through a camera;

[0091] S300: Extract the face detection features, body detection features and body detection attributes corresponding to each to-be-matched person in the to-be-detected image;

[0092] S400: Perform similarity analysis on the face detection features and the body detection features corresponding to each to-be-matched person respectively with the face base feature set and the body base feature set in the target person comprehensive feature library, obtain a face base target set from the face base feature set, and obtain a body base target set from the body base feature set;

[0093] The face base target set includes face similarity scores of target persons corresponding to the R face features with the highest similarity and the numbers of the target persons;

[0094] The body base target set includes body similarity scores of target persons corresponding to the R body features with the highest similarity and the numbers of the target persons;

[0095] S500: Obtain the to-be-selected target persons and the comprehensive scores of the to-be-selected target persons according to the face base target set and the body base target set;

[0096] S600: Obtain the body attributes of the to-be-selected target persons from the body base attributes, analyze the matching result of the body attributes of the to-be-selected target persons and the body detection attributes of the corresponding to-be-matched persons, and obtain the body attribute result of the to-be-selected target persons;

[0097] When the body attributes of the to-be-selected target persons are consistent with the body detection attributes of the corresponding to-be-matched persons, the body attribute result of the to-be-selected target persons is consistent;

[0098] When the body attributes of the to-be-selected target persons are inconsistent with the body detection attributes of the corresponding to-be-matched persons, the body attribute result of the to-be-selected target persons is inconsistent;

[0099] S700: Output the recognition result according to the comprehensive scores of the to-be-selected target persons and the body attribute result.

[0100] In this embodiment, the face features, the body features and the face attribute features are respectively generated through deep learning of the target person image library, high-precision matching and recognition of the persons are performed from multiple dimensions of the face features, the body features and the body attributes, and thus accurate matching and recognition of complex person information are realized.

[0101] In order to establish the target person comprehensive feature library with rich contrast features, in this embodiment, the face base feature set includes face features of different target persons at different angles from-90° to 90°, and the face features include 512 face feature points.

[0102] The human body base library feature set includes human body features of different target persons at different angles from -90° to 90°, and the human body features include 512 human body feature points.

[0103] The human body base library attribute is the most frequently appearing human body attribute of different target persons at different angles from -90° to 90°.

[0104] According to the requirements of face or human body feature recognition accuracy, different numbers of face or human body feature points can be selected for extraction. At present, 512 features are the most mature and have high algorithm accuracy. In the embodiment, 512 feature points are extracted for face features and human body features.

[0105] By establishing the face base library feature set, the human body base library feature set and the human body attribute set of different target persons at different angles from -90° to 90°, compared with the original target person matching information at a single angle, the features of the target persons are extracted from multiple angles in the scheme, so as to realize accurate matching and recognition of complex personnel information.

[0106] S100 realizes the establishment of multi-dimensional base library features. The flowchart of S100 is shown in Figure 2 S100 specifically includes:

[0107] S110: Real-time access to a video stream through a video acquisition device, continuously track the target persons in the video stream through a human body tracking algorithm, create a database for the newly appeared target persons and generate a unique number for them, continuously output the images corresponding to the target persons, and obtain a target person image library;

[0108] S120: Recognize the face features and face angles in each image in the target person image library, extract the face features at different angles, and obtain a face base library feature set;

[0109] S130: Recognize the human body features and human body angles in each image in the target person image library, extract the human body features at different angles, and obtain a human body base library feature set;

[0110] S140: Recognize the human body attributes in each image in the target person image library, and recognize the face angles or human body angles in each image in the target person image library. From the human body attributes corresponding to different face angles or human body angles, the most frequently appearing human body attributes are extracted, and a human body base library attribute is obtained.

[0111] The human body attribute extraction of the target person is implemented in S140, and the identity verification using the face attribute is the most natural and direct means. Compared with other human body biometric characteristics, it has the characteristics of directness, friendliness and convenience, is more easily accepted by users and is not easy to detect. The current mainstream face attribute recognition algorithm mainly includes gender recognition, race recognition, age estimation, expression recognition and the like. In the embodiment, the face attribute is gender. In order to identify the gender, different angle face features can be selected for identification, or different angle human body features can be selected for identification. In the embodiment, the face angle of each image in the target person image library is identified, and the human body attribute is gender.

[0112] In order to realize the updating of the images in the target person image library, the multi-dimensional features in the base library can be continuously updated and iterated to improve the accuracy. Further, the target person image library is updated by periodically acquiring the images corresponding to the target person in S100.

[0113] Specifically, S100 further includes periodically acquiring the images corresponding to the target person output in S110, to realize the updating of the images in the target person image library.

[0114] After the establishment of the target person comprehensive feature library, the images to be detected can be collected to further match and identify the personnel in the images to be detected. S200 collects the images to be detected through a camera; S300 completes the extraction of the face detection features, human body detection features and human body detection attributes corresponding to each to-be-matched personnel in the images to be detected. In S300, the multi-dimensional features of the personnel in the images to be detected are identified through deep learning of the images to be detected. For example, there are M personnel in the images to be detected, and the face features, human body features and face attribute features corresponding to the M personnel are identified.

[0115] S400 respectively analyzes the similarity of the face detection features and the human body detection features corresponding to each to-be-matched personnel with the face base library feature set and the human body base library feature set in the target person comprehensive feature library, to obtain a face base library target set from the face base library feature set and a human body base library target set from the human body base library feature set.

[0116] The step of obtaining the face base library target set is as shown in Figure 3 and includes:

[0117] Step 1: Calculate the face feature difference value between the face detection features corresponding to all to-be-matched personnel in the images to be detected and all features in the face base library feature set, to obtain a face feature difference value matrix;

[0118] The number of all to-be-matched personnel in the images to be detected is M, and the number of all features in the face base library feature set is N. A face feature difference value matrix of MxN order is obtained by using the matrix broadcast operation characteristics;

[0119] Second step: the minimum distance acquisition operation is performed on each row of the face feature difference value matrix, R minimum values are extracted from each row of the face feature difference value matrix, and a face minimum distance matrix is obtained;

[0120] Third step: according to the number of the target personnel corresponding to the feature in the face feature set in the face minimum distance matrix corresponding to each minimum distance, the number of the target personnel corresponding to the face feature with the first R similarity is obtained;

[0121] Fourth step: the face minimum distance matrix is normalized to convert the distances into scores between 0 and 1, and the face similarity score of the target personnel corresponding to the face feature with the first R similarity is obtained.

[0122] In this embodiment, the face similarity is analyzed by the feature difference value. Specifically, there are M face detection features corresponding to M matching personnel in the to-be-detected image, and there are N face features in the face feature set.

[0123] In the first step, the face detection features corresponding to all the matching personnel in the to-be-detected image are B, wherein the face feature of the kth face is β k , wherein k = [1, 2, 3…M], representing the hth feature point of the kth face, h = [1, 2, 3…512];

[0124] The face feature set is A, A = [α1, α2, α3......α N ], wherein the i th face feature is a i , wherein i = [1, 2, 3…N], representing the jth feature point of the ith face feature, j = [1, 2, 3…512];

[0125] The calculation formula of the face feature difference value d ki is as follows:

[0126]

[0127] The M*N order face feature difference value matrix D1 is obtained by using the matrix broadcast operation characteristics:

[0128]

[0129] The feature difference of the face corresponding to the detected feature is obtained by analyzing and adding the feature difference of 512 feature points of the face detection feature and the feature difference of 512 feature points of all features in the face feature set, the face feature difference represents the comprehensive deviation degree of 512 feature points, and the recognition accuracy is improved.

[0130] Correspondingly, the similarity of the human body detection feature and the feature in the human body feature set is analyzed, the similarity analysis method is consistent with the face feature similarity analysis method, and the steps of obtaining the human body target set include:

[0131] Step 1: Calculate the human body feature difference of all features in the human body feature set corresponding to the human body detection feature of all to-be-matched personnel in the detected image, and obtain a human body feature difference matrix;

[0132] The number of all to-be-matched personnel in the detected image is M, the number of all features in the human body feature set is N, and the M*N order human body feature difference matrix is obtained by using the matrix broadcast operation characteristic;

[0133] Step 2: Obtain the minimum distance of each row of the human body feature difference matrix, extract R minimum values from each row of the human body feature difference matrix, and obtain a human body minimum distance matrix;

[0134] Step 3: According to the number of the target personnel corresponding to the feature in the human body feature set corresponding to each minimum distance in the human body minimum distance matrix, the number of the target personnel corresponding to the human body feature with the first R similarity is obtained;

[0135] Step 4: Normalize the human body minimum distance matrix to convert the distances to scores between 0 and 1, and obtain the human body similarity scores of the target personnel corresponding to the human body feature with the first R similarity.

[0136] The first 5, the first 10, the first 15, the first 20, and the first 25 personnel are obtained respectively, the first 20 is a number that can maximize the reduction of error rate according to the existing research data, and specifically, in the embodiment, the value of R is 20.

[0137] S500: According to the face library target set and the human body library target set, the selected target personnel and the comprehensive score of the selected target personnel are obtained, and the flowchart of S500 is as shown in Figure 4 S500 includes:

[0138] S510: According to the face library target set, the number of face libraries corresponding to different numbers T1 is obtained, and according to the human body library target set, the number of human body libraries corresponding to different numbers T2 is obtained;

[0139] The number T1 of the face base corresponding to different numbers is the number of different numbers corresponding to the face base target set;

[0140] The number T2 of the body base corresponding to different numbers is the number of different numbers corresponding to the body base target set;

[0141] S520: obtaining the intersection number and the face similarity score and the body similarity score corresponding to the intersection number from the intersection of the face base target set and the body base target set;

[0142] S530: calculating the comprehensive score of the target person corresponding to the intersection number, and returning at least two target persons with the highest score to obtain the selected target person and the comprehensive score of the selected target person.

[0143] In order to improve the recognition accuracy, the comprehensive score of the target person corresponding to the intersection number is calculated, and at least two target persons with the highest score are returned. According to actual recognition requirements, two target persons, three target persons, five target persons, six target persons, etc. can be selected. In this embodiment, the number that can maximize the hit rate is 5 through actual production environment data running. Specifically, in S530, five target persons with the highest score are returned.

[0144] The comprehensive score needs to fully consider the face features and the body features. In this embodiment, the calculation formula of the comprehensive score of the target person corresponding to the intersection number is:

[0145] The comprehensive score of the target person corresponding to the intersection number = the highest score of the face similarity score corresponding to the intersection number x (1+0.1x the number T1 of the face base corresponding to the intersection number) + the highest score of the body similarity score corresponding to the intersection number x (1+0.1x the number T2 of the body base corresponding to the intersection number).

[0146] In order to simplify the calculation process, another embodiment optimizes S500. In this embodiment, S500 includes:

[0147] S510: obtaining the intersection number and the face similarity score and the body similarity score corresponding to the intersection number from the intersection of the face base target set and the body base target set;

[0148] S520: obtaining the number T1 of the face base corresponding to different intersection numbers according to the face base target set, and obtaining the number T2 of the body base corresponding to different intersection numbers according to the body base target set;

[0149] The number T1 of the face base corresponding to different intersection numbers is the number of different intersection numbers corresponding to the face base target set;

[0150] The number T2 of the human body base library corresponding to the different intersection numbers is the number corresponding to the different intersection numbers in the human body base library target set;

[0151] S530: Calculate the comprehensive score of the target personnel corresponding to the intersection number, return at least two target personnel with the highest score, obtain the selected target personnel and the comprehensive score of the selected target personnel.

[0152] In this embodiment, first, the intersection number is obtained from the intersection of the face base library target set and the human body base library target set, and only the number T1 of the face base library corresponding to the intersection number and the number T2 of the human body base library are obtained, thereby reducing the calculation amount and improving the calculation speed.

[0153] The selected target personnel and the comprehensive score of the selected target personnel are obtained through S500, and the human body attribute result of the selected target personnel is obtained through S600.

[0154] In S600, the human body attribute corresponding to the target personnel number is obtained from the human body base library attribute according to the target personnel number corresponding to the selected target personnel, so as to obtain the human body attribute of the selected target personnel, analyze the matching result of the human body attribute of the selected target personnel and the human body detection attribute corresponding to the to-be-matched personnel, and obtain the human body attribute result of the selected target personnel.

[0155] Finally, the comprehensive score of the selected target personnel and the human body attribute result are integrated through S700, and the unique hit personnel recognition result is output.

[0156] The flowchart of S700 is shown in Figure 5 The S700 includes:

[0157] S710: The comprehensive score of the selected target personnel is judged with the absolute hit threshold value;

[0158] If the comprehensive score of a single selected target personnel exceeds the threshold value, a single person hit result is output;

[0159] If the comprehensive scores of multiple selected target personnel exceed the threshold value, the next step is entered;

[0160] S720: The human body attribute result of the selected target personnel is introduced to judge the multiple selected target personnel with the score exceeding the threshold value;

[0161] If the human body attribute result of a single selected target personnel is consistent, a single person hit result is output;

[0162] If the human body attribute results of multiple selected target personnel are consistent, the person with the highest comprehensive score in the multiple selected target personnel is output as the hit result.

[0163] In the embodiment, the face detection features and the human body detection features in the to-be-detected image are acquired, the face detection features and the human body detection features are matched and recognized with the face base feature set and the human body base feature set, the target personnel with high similarity is extracted, the target personnel with high similarity is comprehensively scored and analyzed, the to-be-selected target personnel is obtained, and the only target personnel with high matching is output according to the comprehensive score of the to-be-selected target personnel and the human body attribute matching condition, so as to solve the problems of low recognition efficiency and low accuracy rate in the prior art, and to achieve automatic recognition, improved personnel recognition efficiency and accuracy rate.

[0164] In a second aspect, the present application also provides a multi-dimensional feature fusion personnel identification system, as shown in the accompanying drawings, the system comprises: Figure 6

[0165] A target personnel comprehensive feature base establishing module, a to-be-detected image collecting module, a to-be-matched personnel multi-dimensional feature extracting module, a face feature and human body feature similarity analysis module, a to-be-selected target personnel comprehensive score calculating module, a to-be-selected target personnel human body attribute result analyzing module, and a personnel identification result judging module.

[0166] The target personnel comprehensive feature base establishing module establishes a target personnel image base through video tracking analysis, each target personnel in the target personnel image base corresponds to a unique number, the target personnel image base is identified for face features, human body features and face attributes, a target personnel comprehensive feature base is obtained, the target personnel comprehensive feature base includes a face base feature set, a human body base feature set and a human body base attribute, each feature in the face base feature set and the human body base feature set is associated with the corresponding target personnel number, and the human body base attribute is associated with the corresponding target personnel number.

[0167] The to-be-detected image collecting module collects the to-be-detected image through a camera.

[0168] The to-be-matched personnel multi-dimensional feature extracting module is used to extract the face detection features, the human body detection features and the human body detection attributes of each to-be-matched personnel in the to-be-detected image.

[0169] The face feature and human body feature similarity analysis module respectively analyzes the similarity of the face detection features and the human body detection features of each to-be-matched personnel with the face base feature set and the human body base feature set in the target personnel comprehensive feature base, obtains a face base target set from the face base feature set, and obtains a human body base target set from the human body base feature set.

[0170] The face base target set includes the face similarity scores of the target personnel corresponding to the face features with high similarity and the numbers of each target personnel.

[0171] ​The human body library target set includes human body similarity scores of target personnel corresponding to the R human feature similarities in the order of similarity, and numbers of each target personnel;

[0172] The candidate target personnel comprehensive score calculation module obtains the candidate target personnel and a comprehensive score of the candidate target personnel according to the face library target set and the human body library target set;

[0173] The candidate target personnel human body attribute result analysis module obtains human body attributes of the candidate target personnel from the human body library attributes, analyzes a matching result of the human body attributes of the candidate target personnel and human body detection attributes of the corresponding to-be-matched personnel, and obtains a human body attribute result of the candidate target personnel;

[0174] When the human body attributes of the candidate target personnel are consistent with the human body detection attributes of the corresponding to-be-matched personnel, the human body attribute result of the candidate target personnel is obtained as consistent;

[0175] When the human body attributes of the candidate target personnel are inconsistent with the human body detection attributes of the corresponding to-be-matched personnel, the human body attribute result of the candidate target personnel is obtained as inconsistent;

[0176] The personnel recognition result judgment module outputs a recognition result according to the comprehensive score of the candidate target personnel and the human body attribute result.

[0177] It should be noted that each module (or unit) in the embodiment is in logical sense, and multiple modules (or units) can be combined into one module (or unit), or one module (or unit) can be split into multiple modules (or units) in specific implementation.

[0178] The multi-dimensional feature fusion personnel recognition system in the embodiment realizes automatic recognition, is low in cost, simple and convenient, and obvious in effect, can accurately and quickly realize high-precision recognition of personnel, and has the advantages of reducing the workload of workers and improving work efficiency.

[0179] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by programs instructing related hardware, and the programs can be stored in a computer-readable storage medium. When the program is executed, the processes of the above-mentioned embodiments can be included. The storage medium can be a magnetic disc, an optical disc, a read-only memory (ROM) or a random access memory (RAM).

[0180] The above merely describes the preferred examples of the present application and is not intended to limit the present application. Although the present application has been described in detail with reference to the foregoing examples, those skilled in the art should understand that they can modify the technical solutions described in the foregoing examples, or make equivalent replacements to some of the technical features. Any modification, equivalent replacement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A multi-dimensional feature fusion personnel identification method, characterized in that, The method comprises: S100: establishing a target personnel image library through video tracking analysis, each target personnel in the target personnel image library corresponding to a unique number, identifying the target personnel image library for face features, body features, and face attributes to obtain a target personnel comprehensive feature library, the target personnel comprehensive feature library comprising a face base feature set, a body base feature set, and a body base attribute, and respectively associating each feature in the face base feature set and the body base feature set with the corresponding target personnel number and associating the body base attribute with the corresponding target personnel number; The face base feature set comprises face features of different target personnel at different angles from -90° to 90°, and the face features comprise 512 face feature points; The body base feature set comprises body features of different target personnel at different angles from -90° to 90°, and the body features comprise 512 body feature points; The body base attribute is the most frequently occurring body attribute of different target personnel at different angles from -90° to 90°, and the body attribute is gender; S200: collecting a to-be-detected image through a camera; S300: extracting face detection features, body detection features, and body detection attributes of each to-be-matched personnel in the to-be-detected image, identifying the corresponding face features, body features, and body attributes of the personnel in the to-be-detected image through deep learning of the to-be-detected image, and respectively obtaining the face detection features, body detection features, and body detection attributes; S400: performing similarity analysis on the face detection features and the body detection features of each to-be-matched personnel and the face base feature set and the body base feature set in the target personnel comprehensive feature library, obtaining a face base target set from the face base feature set, and obtaining a body base target set from the body base feature set; The face database target set comprises a similarity front R face similarity scores of target personnel corresponding to the face features and numbers of each target personnel; The human body database target set includes similarity rankings. R The similarity score of the target person corresponding to the human body features and the number of each target person; S500: obtaining a to-be-selected target personnel and a comprehensive score of the to-be-selected target personnel according to the face base target set and the body base target set; S600: obtaining a body attribute of the to-be-selected target personnel from the body base attribute, analyzing a matching result of the body attribute of the to-be-selected target personnel and the body detection attribute of the corresponding to-be-matched personnel, and obtaining a body attribute result of the to-be-selected target personnel; When the body attribute of the to-be-selected target personnel is consistent with the body detection attribute of the corresponding to-be-matched personnel, the body attribute result of the to-be-selected target personnel is consistent; When the body attribute of the to-be-selected target personnel is inconsistent with the body detection attribute of the corresponding to-be-matched personnel, the body attribute result of the to-be-selected target personnel is inconsistent; S700: outputting a recognition result according to the comprehensive score of the to-be-selected target personnel and the body attribute result.

2. The multi-dimensional feature fusion personnel identification method according to claim 1, characterized in that, The S100 specifically comprises: S110: real-time accessing a video stream through a video collection device, continuously tracking target personnel in the video stream through a body tracking algorithm, creating a database for newly appearing target personnel and generating a unique number for the target personnel, continuously outputting images corresponding to the target personnel, and obtaining the target personnel image library; S120: identify the face features and face angles in each image in the target personnel image library, extract face features at different angles, and obtain a face base library feature set; S130: identify the body features and body angles in each image in the target personnel image library, extract body features at different angles, and obtain a body base library feature set; S140: identify the body attributes in each image in the target personnel image library, and identify the face angles or body angles in each image in the target personnel image library, extract the most frequently occurring body attributes from the body attributes corresponding to different face angles or body angles, and obtain a body base library attribute.

3. The multi-dimensional feature fusion person recognition method according to claim 2, characterized in that, The S100 further includes periodically obtaining the images corresponding to the target personnel continuously output in the S110, to realize updating of the images in the target personnel image library.

4. The multi-dimensional feature fusion personnel identification method according to claim 1, characterized in that, The step of obtaining the face base library target set includes: First step: calculate the face feature difference values between the face detection features of all to-be-matched personnel in the detection image and all features in the face base library feature set, to obtain a face feature difference value matrix; The number of all to-be-matched personnel in the detection image is M, and the number of all features in the face base library feature set is N, and a face feature difference value matrix of MxN order is obtained by using the matrix broadcast operation characteristics; Second step: the minimum distance of each row of the face feature difference matrix is obtained, and the minimum value of each row of the face feature difference matrix is extracted R to obtain the face minimum distance matrix; Step 3: According to the number of the target personnel corresponding to the feature in the feature set in the face database corresponding to each minimum distance in the minimum distance matrix of the face, the number of the target personnel corresponding to the feature of the face feature is obtained. R corresponding to the feature of the face feature. Step 4: Normalization operation is performed on the face minimum distance matrix, and each distance is converted into a score between 0 and 1 to obtain the similarity score of the target personnel corresponding to the face feature R before the face feature.

5. The multi-dimensional feature fusion person recognition method according to claim 4, characterized in that, The face detection features corresponding to all the to-be-matched persons in the to-be-detected image are , wherein the first k face feature of the first β person is k , , wherein k =[1, 2, 3…M], , the first k feature point of the first h person's face is h =[1, 2, 3…512]; The human face library feature set is A , , wherein the first i personal face feature is a i , , wherein i =[1, 2, 3…N], represents the first i personal face feature j feature point, j =[1, 2, 3…512]; The face feature difference value The calculation formula is: The M*N order face feature difference matrix is obtained by using the matrix broadcast operation characteristics D 1: .

6. The multi-dimensional feature fusion person recognition method according to claim 1, characterized in that, The S500 includes: S510: Obtain the number of face databases corresponding to different numbers according to the face database target set T 1 Obtain the number of human body databases corresponding to different numbers according to the human body database target set T 2 ; The number of different numbers corresponding to the face base T 1 The number of different numbers corresponding to the face base target set The number of different numbers corresponding to the human body library T 2 The number of different numbers corresponding to the human body library target set S520: obtain intersection numbers and face similarity scores and body similarity scores corresponding to the intersection numbers from the intersection of the face base library target set and the body base library target set; S530: calculate the comprehensive scores of the target personnel corresponding to the intersection numbers, return at least two target personnel with the highest scores, and obtain to-be-selected target personnel and the comprehensive scores of the to-be-selected target personnel.

7. The multi-dimensional feature fusion person recognition method according to claim 6, characterized in that, The calculation formula of the comprehensive scores of the target personnel corresponding to the intersection numbers is: The comprehensive score of the target person corresponding to the intersection number = the highest score of the face similarity score corresponding to the intersection number × (1 + 0.1 × the number of face databases corresponding to the intersection number) + the highest score of the body similarity score corresponding to the intersection number × (1 + 0.1 × the number of body databases corresponding to the intersection number) T 1 ). T 2 ).

8. The multi-dimensional feature fusion person recognition method according to claim 1, characterized in that, The S700 includes: S710: judge the comprehensive scores of the to-be-selected target personnel against the absolute hit threshold value; If the comprehensive score of a single to-be-selected target personnel exceeds the threshold value, output a single-person hit result; If the comprehensive scores of multiple to-be-selected target personnel exceed the threshold value, proceed to the next step; S720: judge the body attribute results of the multiple to-be-selected target personnel whose scores exceed the threshold value; If the body attribute results of a single to-be-selected target personnel are consistent, output a single-person hit result; If the body attribute results of multiple to-be-selected target personnel are consistent, output the to-be-selected target personnel with the highest comprehensive score as the hit result.

9. A multi-dimensional feature fusion based person identification system characterized in that, The system includes: a target personnel comprehensive feature library establishing module, a to-be-detected image collecting module, a to-be-matched personnel multi-dimensional feature extracting module, a face feature and body feature similarity analyzing module, a to-be-selected target personnel comprehensive score calculating module, a to-be-selected target personnel body attribute result analyzing module, and a personnel recognition result judging module. The target personnel comprehensive feature library establishing module establishes a target personnel image library through video tracking analysis, each target personnel in the target personnel image library corresponds to a unique number, and the target personnel image library is subjected to face feature, body feature, and face attribute recognition to obtain a target personnel comprehensive feature library, which includes a face base feature set, a body base feature set, and a body base attribute, each feature in the face base feature set and the body base feature set is associated with the corresponding target personnel number, and the body base attribute is associated with the corresponding target personnel number; The face base feature set includes face features of different target personnel at different angles from -90° to 90°, and the face features include 512 face feature points; The body base feature set includes body features of different target personnel at different angles from -90° to 90°, and the body features include 512 body feature points; The body base attribute is the most frequently occurring body attribute of different target personnel at different angles from -90° to 90°, and the body attribute is gender; The to-be-detected image acquisition module acquires a to-be-detected image through a camera; The to-be-matched personnel multi-dimensional feature extraction module is configured to extract face detection features, body detection features, and body detection attributes of each to-be-matched personnel in the to-be-detected image, recognize corresponding face features, body features, and body attributes of the personnel in the to-be-detected image through deep learning of the to-be-detected image, and obtain the face detection features, the body detection features, and the body detection attributes, respectively; The face feature and body feature similarity analysis module performs similarity analysis on the face detection features and the body detection features of each to-be-matched personnel and the face base feature set and the body base feature set in the target personnel comprehensive feature library, respectively, obtains a face base target set from the face base feature set, and obtains a body base target set from the body base feature set; The face database target set comprises a similarity R face feature corresponding to the target personnel and the number of each target personnel; The human body library target set includes the similarity before R The human body feature corresponding to the target person's human body similarity score and the number of each target person; The to-be-selected target personnel comprehensive score calculation module obtains a to-be-selected target personnel and a comprehensive score of the to-be-selected target personnel according to the face base target set and the body base target set; The to-be-selected target personnel body attribute result analysis module obtains a body attribute of the to-be-selected target personnel from the body base attribute, analyzes a matching result of the body attribute of the to-be-selected target personnel and a body detection attribute of the corresponding to-be-matched personnel, and obtains a body attribute result of the to-be-selected target personnel; When the body attribute of the to-be-selected target personnel is consistent with the body detection attribute of the corresponding to-be-matched personnel, the body attribute result of the to-be-selected target personnel is consistent; When the body attribute of the to-be-selected target personnel is inconsistent with the body detection attribute of the corresponding to-be-matched personnel, the body attribute result of the to-be-selected target personnel is inconsistent; The personnel recognition result judgment module outputs a recognition result according to the comprehensive score and the body attribute result of the to-be-selected target personnel.

Images