A person detection system and method based on dynamic face recognition and a dynamic face recognition method

Abstract

The present invention discloses a system and method for simultaneous inspection of persons based on dynamic face recognition, and a dynamic face recognition method, wherein human body security inspection and luggage security inspection information are bound through dynamic face recognition technology: a residual network ResNet‑50 is implemented using Pytorch in the face recognition part, and the CNN model is used to train the model through a 50-layer residual neural network, so that the recognition accuracy is higher; through the recognition and comparison of facial features of a first camera and a second camera, the identity ID of the passenger (the inspected passenger) is used as the association intermediary, and the X-ray image and luggage suspect information collected by the X-ray machine, the human body security inspection image and the human body carrying suspect information collected by the terahertz security inspection instrument are bound to their identity ID, so as to achieve the effect of binding and displaying the luggage security inspection result and the human body security inspection result through the face information without stopping during the security inspection process.

Description

Technical Field

[0001] The present invention relates to the field of security inspection technology, and in particular to a person inspection system and method based on dynamic face recognition, and a dynamic face recognition method. Background Art

[0002] Due to the special requirements of subway and railway security checks on passenger flow, traditional body security checks are not sufficient to cope with large passenger flows and often cause congestion. Terahertz body security inspection machines are used in more and more projects because they solve a series of problems in body security checks. For example, some stations in Shanghai and Hefei subway stations are equipped with machines.

[0003] With the addition of new machines, it is gradually found that the information of human body security inspection and X-ray baggage security inspection cannot be integrated, because the two machines are independent systems and it is difficult to link the systems, which leads to problems such as complex information retrieval of the inspected persons and difficulty in tracing back afterwards. In addition, passengers do not need to stop when passing through the terahertz human body security inspection machine, so the difficulty of collecting and identifying facial information while walking is greatly increased, and factors such as the angle of the walking face, lighting expression, etc. also pose challenges to recognition.

[0004] In order to solve the above problems, a terahertz body security inspection and X-ray luggage security inspection binding system is proposed. Summary of the invention

[0005] In view of the problems in the above-mentioned background technology, an important purpose of the present invention is to provide a person detection method based on dynamic face recognition.

[0006] To achieve the above purpose, the specific technical solution of the present invention is as follows:

[0007] A method for detecting people based on dynamic face recognition, the specific steps are as follows:

[0008] (1) The X-ray machine scans the passenger's baggage, extracts X-ray image information A1 of the items in the passenger's baggage and information A2 of the suspected luggage, and uploads the extracted image information to the server;

[0009] (2) When the passenger places the package at the X-ray machine, the first camera collects facial image information, and the dynamic face recognition method is used to detect, extract features, and identify and compare the facial image information to obtain the passenger's facial feature image information B1, and establish a corresponding identity ID;

[0010] (3) Using the time attribute of the facial image information collected by the first camera as the association medium, the passenger's ID is associated with the X-ray image information A1 of the package and the suspected luggage information A2, and the association is then stored in the server;

[0011] (4) The passenger enters the terahertz security inspection machine for security inspection, and the body security inspection image information C1 and the body suspected object information C2 of the passenger being inspected are extracted; at the same time, the facial image information of the passenger being inspected is collected through the second camera, and the facial image information is detected, feature extracted, and identified and compared using a dynamic face recognition method to obtain the facial feature image information B2 of the passenger being inspected;

[0012] (5) Perform real-time face recognition comparison between the facial feature image information B2 of the passenger being inspected and the facial feature image information B1 of the passenger. When the facial feature comparison similarity exceeds 90%, it is determined that they are the same person. The passenger's ID is searched and the passenger's ID is used as the association point to associate the human body security inspection image information C1 and the human body suspected object information C2.

[0013] (6) The associated data - the X-ray image information A1 of the package, the suspicious luggage information A2, the human body security inspection image information C1, and the suspicious human body information C2 are stored in the server.

[0014] The dynamic face recognition method uses Pytorch to build a residual network ResNet-50, uses a convolutional neural network model, trains the model through a deep learning algorithm, uses scored face image information to train and test the model, and uses the trained network model to identify and analyze face information.

[0015] Furthermore, the training of the network model of the dynamic face recognition method includes the following factors: face angle, face missing, face lighting and face expression.

[0016] Furthermore, the data that are not successfully associated in step (6) - X-ray image information A1, suspected luggage information A2 and human body inspection image information C1, suspected human body information C2 are stored in the corresponding X-ray database and terahertz database respectively.

[0017] A second object of the present invention is to provide a person detection system based on dynamic face recognition, comprising:

[0018] An X-ray machine is used to scan passenger baggage information, extract X-ray image information A1 of items in the passenger baggage, and information A2 of suspected luggage, and upload the extracted image information to a server; including:

[0019] The first camera is used to dynamically collect, extract and identify the facial information of different passengers when placing the package, obtain the facial feature image information B1 of the passenger, and establish the corresponding identity ID, and send it to the server at the same time;

[0020] The terahertz security inspection machine is used to scan the passenger's body information and extract the passenger's body security inspection image information C1 and the information of suspicious objects carried by the human body C2; including:

[0021] The second camera is used to dynamically collect, extract and identify the facial information of the inspected passenger, obtain the passenger's facial feature picture information B2, and send it to the server;

[0022] The server binds the collected X-ray image information A1 of the package, the suspected luggage information A2, the passenger's body inspection image information C1, and the suspected body information C2 to the passenger's identity ID, and stores the data in the server's database and storage medium by category.

[0023] Furthermore, the server comprises:

[0024] The first image processing unit is used to receive the X-ray image information A1 of the package and the suspected luggage information A2 uploaded by the X-ray machine, and at the same time receive and identify the facial feature picture information B1 of the passenger uploaded by the first camera, and associate the passenger's identity ID with the X-ray image information A1 of the package and the suspected luggage information A2 using the time attribute of the facial feature picture information B1 as an association medium;

[0025] The second image processing unit is used to receive the passenger body security inspection image information C1 and the human body suspicious object information C2 uploaded by the terahertz security inspection machine, and at the same time receive and identify the facial feature picture information B2 of the inspected passenger uploaded by the second camera; and perform real-time face recognition comparison between the facial feature picture information B2 of the inspected passenger and the facial feature picture information B1 of the passenger. After the comparison and recognition is successful, the identity ID of the corresponding passenger is searched, and the identity ID of the passenger is associated with the passenger body security inspection image information C1 and the human body suspicious object information C2.

[0026] Furthermore, the time attribute of the facial feature image information B1 refers to: the initial time of the passenger's facial information collected by the first camera and the time period when the X-ray machine transmits the package.

[0027] Furthermore, the database includes a relational database, an X-ray database and a terahertz database.

[0028] Furthermore, the X-ray database is used to store the X-ray image information A1 and the luggage suspicious object information A2 uploaded by the X-ray machine; the terahertz database is used to store the human body security image information C1 and the human body carrying suspicious object information C2 uploaded by the terahertz security inspection machine.

[0029] The third object of the present invention is to provide a method for identifying people based on dynamic face recognition, the specific steps of which are as follows:

[0030] S1. Build the ResNet-50 residual network model:

[0031] S1.1 Obtain a face recognition dataset and score the images in the dataset based on face angle, face missing, face lighting, and face expression;

[0032] S1.2 randomly divides the data set into a training set and a test set, with 80% for the training set and 20% for the test set;

[0033] S1.3 Implement the ResNet-50 residual network algorithm structure on the pytorch platform; import the data set for training and parameter adjustment, and verify the model on the test set after convergence;

[0034] S2, using the ResNet-50 residual network model to screen, detect, and extract features from the real-time face image captured by the second camera located at the human body security inspection machine, and compare and analyze it with the face image feature library captured and extracted by the first camera located at the X-ray machine. After the comparison is successful, proceed to step S3;

[0035] S3. Using the passenger's identity as an intermediary, the body security information uploaded by the body security inspection machine is associated with the package security information uploaded by the X-ray machine.

[0036] Furthermore, the successful comparison in step S2 means that: when the facial image features are analyzed and compared and the feature similarity exceeds 90%, it is determined that they are the same passenger.

[0037] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0038] In the present invention, human body security inspection and luggage security inspection information are bound through dynamic face recognition technology: the face recognition part uses Pytorch to implement the residual network ResNet-50, and uses the CNN model to train the model through a 50-layer residual neural network, so that the recognition accuracy is higher; through the facial feature recognition and comparison of the first camera and the second camera, the identity ID of the passenger (the inspected passenger) is used as the association intermediary, the X-ray image collected by the X-ray machine, the luggage suspicious object information, the human body security inspection image collected by the terahertz security inspection instrument, and the information of the human body carrying suspicious objects are bound to the identity ID, so as to achieve the effect of binding and displaying the luggage security inspection results and the human body security inspection results through the face information without stopping during the security inspection process.

[0039] The dynamic face recognition technology of the present invention performs detection based on comprehensive factors such as face angle based on deep learning, face missing based on algorithm, face lighting, face expression, etc., gives corresponding scores, and retains images that meet the standards and pushes them to the next stage for recognition. The recognition model has been trained with a large number of samples, with a recognition accuracy of more than 97%, and a recognition extraction speed of less than 100ms. The fast execution speed meets the real-time requirements of security inspection, the information collection and binding speed is fast, pedestrians do not stop, and the flexibility is high. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A schematic diagram of the person co-examination system of the present invention;

[0041] Figure 2 The network structure diagram of the dynamic face recognition system of the present invention;

[0042] Figure 3 It is the overall flow chart of the person co-inspection system of the present invention;

[0043] Figure 4 It is a detailed flow chart of the person co-inspection system of the present invention.

[0044] Figure numerals: 1. second camera; 2. terahertz body security inspection machine; 3. first camera; 4. X-ray machine. DETAILED DESCRIPTION

[0045] The specific implementation of the present invention is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described here is only used to illustrate and explain the present invention, and is not used to limit the present invention.

[0046] like Figure 1 As shown, the system of the present invention includes an X-ray machine, a terahertz body security inspection machine, and a first camera and a second camera respectively installed at the X-ray machine and the terahertz body security inspection machine, and also includes a dynamic face recognition system.

[0047] An X-ray machine is used to scan the passenger's baggage information, extract the X-ray image information A1 of the items in the passenger's baggage, and the information A2 of the suspected luggage, and upload the extracted image information to the server;

[0048] The first camera is used to dynamically collect, extract and identify the facial information of different passengers when placing the package, obtain the facial feature image information B1 of the passenger, and establish the corresponding identity ID, and send it to the server at the same time;

[0049] The terahertz security inspection machine is used to scan the passenger's body information and extract the passenger's body security inspection image information C1 and the information of suspicious objects carried by the human body C2;

[0050] The second camera is used to dynamically collect, extract and identify the facial information of the inspected passenger, obtain the passenger's facial feature picture information B2, and send it to the server.

[0051] The server includes a first image processing unit and a second image processing unit:

[0052] The first image processing unit is used to receive the X-ray image information A1 of the package and the suspected luggage information A2 uploaded by the X-ray machine, and at the same time receive and identify the facial feature picture information B1 of the passenger uploaded by the first camera, and associate the passenger's identity ID with the X-ray image information A1 of the package and the suspected luggage information A2 using the time attribute of the facial feature picture information B1 as an association medium;

[0053] The second image processing unit is used to receive the passenger body security inspection image information C1 and the human body suspicious object information C2 uploaded by the terahertz security inspection machine, and at the same time receive and identify the facial feature picture information B2 of the inspected passenger uploaded by the second camera; and perform real-time face recognition comparison between the facial feature picture information B2 of the inspected passenger and the facial feature picture information B1 of the passenger. After the comparison and recognition is successful, the identity ID of the corresponding passenger is searched, and the identity ID of the passenger is associated with the passenger body security inspection image information C1 and the human body suspicious object information C2.

[0054] Dynamic face recognition system: Building a residual network ResNet-50 model

[0055] Implement the ResNet-50 residual network algorithm structure on the pytorch platform; import the data set for training and parameter adjustment, and verify the model on the test set after convergence. Figure 2 shown.

[0056] Specifically, face angle, face missing, face lighting, and face expression calculation:

[0057] (1) Based on face angle: The model scores from 0 from front face to side face, and the absolute front face score is 10 points;

[0058] (2) Based on face illumination: A histogram is calculated for the position of the recognized face, and the average regional illumination score is calculated. The score is scored from 0 to 10 in the range of too bright or too dark, with 0 for both too bright and too dark.

[0059] (3) Based on face missing: extract face information through the residual network model to detect whether there are organs missing in the face image, which is used as a pre-judgment standard for quality detection. If there are missing organs, face extraction will not be performed;

[0060] (4) Based on facial expressions: The geometric features are calculated through the information extracted by the residual network model. The organs on different faces, such as eyes, noses, and other facial contours, have different shapes, sizes, and relative positions. A vector is used to represent the extracted geometric parameters. The relative distance between them is calculated to determine whether the expression meets the recognition criteria. The score starts from 10 points for neutral and decreases to 0 points according to the degree of exaggeration of the expression of each part.

[0061] Using an open source face recognition dataset, most face images are rotated within a range of 45 degrees to the left and right, with a data volume of 60,000. The dataset format is organized, the face images are cropped, the images are aligned, and the dataset is divided into a training set of 48,000 and a test set of 12,000. The ResNet-50 residual network algorithm structure is implemented on the pytorch platform. The dataset is imported for training and parameter adjustment. After convergence, the model is verified on the test set, and the recognition accuracy rate detected by the verification set is 97.67%.

[0062] like Figure 3 , 4 As shown in the figure, the trained ResNet-50 network model is used for dynamic face recognition. The method for person detection based on dynamic face recognition has the following specific steps:

[0063] (1) The X-ray machine scans the passenger's baggage and extracts X-ray image information A1 of the items in the passenger's baggage and information A2 of the suspected luggage, and uploads the extracted image information to the server.

[0064] (2) When the passenger places the package at the X-ray machine, the first camera collects facial image information, and the dynamic face recognition method is used to detect, extract features, and identify and compare the facial image information to obtain the passenger's facial feature image information B1 and establish a corresponding identity ID.

[0065] (3) Using the time attribute of the facial image information collected by the first camera as the association medium, the passenger's ID is associated with the X-ray image information A1 of the package and the suspected luggage information A2, and the association is then stored in the server;

[0066] The time attribute refers to: the initial time of the passenger's facial information collected by the first camera and the time period when the X-ray machine conveys the package. After receiving the X-ray image of the package, because the conveyor belt speed is uniform, the facial image within a certain time period can be inferred, the corresponding facial information can be found, and the X-ray image information A1, the suspected baggage information A2 and the facial information are bound.

[0067] (4) The passenger enters the terahertz security inspection machine for security inspection, and the body security inspection image information C1 and the information C2 of suspicious objects carried by the passenger are extracted; at the same time, the facial image information of the passenger is collected through the second camera, and the facial image information is detected, feature extracted, and identified and compared using a dynamic face recognition method to obtain the facial feature image information B2 of the passenger.

[0068] (5) Perform real-time face recognition comparison on the facial feature image information B2 of the passenger being inspected and the facial feature image information B1 of the passenger. When the facial feature comparison similarity exceeds 90%, it is determined that they are the same person. The passenger's ID is searched and the passenger's ID is used as the association point to associate with the human body security inspection image information C1 and the human body suspicious object information C2.

[0069] (6) storing the associated data—the X-ray image information A1 of the package, the suspected luggage information A2, the human body security inspection image information C1, and the suspected human body information C2—on the server;

[0070] The data that have not been successfully associated - X-ray image information A1, suspected baggage information A2 and human body inspection image information C1, suspected human body information C2 are stored in the corresponding X-ray database and terahertz database respectively.

[0071] Working principle: When a passenger places a bag at the X-ray machine, the first camera collects facial information, uses dynamic facial recognition methods to perform facial detection, quality inspection, feature extraction, and identification comparison to identify different faces, and then compiles different identity IDs. At the same time, the time attribute of the face screenshot of the first camera is used, and the algorithm of time association and item feature extraction is used to associate the passenger's identity ID with the X-ray image of the baggage package and store it in the server; when the passenger walks to the terahertz security inspection machine, the real-time facial comparison is performed with the facial information collected by the second camera to find the corresponding identity ID, and the information of suspected objects carried by the human body collected by the terahertz security inspection machine is bound to the inspected passenger; the effect of binding and displaying the baggage security inspection results and the human body security inspection results through facial information without stopping during the security inspection process is achieved. The system binds the collected X-ray images, baggage suspected object information, terahertz human body security inspection images, and human body suspected object information to their identity IDs, and stores them in the server database.

[0072] Although the embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and the embodiments. They can be applied to multiple fields to which the present invention is applicable. For those familiar with the art, additional modifications can be easily implemented. Therefore, without departing from the general concept defined by the claims and the scope of equivalents, the present invention is not limited to the specific details and the illustrations shown and described herein.

Claims

1. A person detection method based on dynamic face recognition, characterized in that: The specific steps are as follows: The X-ray machine scans the passenger's baggage, extracts X-ray image information A1 of the items in the passenger's baggage, and information A2 of the suspected luggage, and uploads the extracted image information to the server; When the passenger places the package at the X-ray machine, the first camera collects the facial image information, and the dynamic face recognition method is used to detect, extract features, and identify and compare the facial image information to obtain the passenger's facial feature image information B1, and establish the corresponding identity ID; Using the time attribute of the facial image information collected by the first camera as the association medium, the passenger's identity ID is associated with the X-ray image information A1 of the package and the suspected luggage information A2, and stored in the server; The passenger enters the terahertz security inspection machine for security inspection, and the body security inspection image information C1 and the body suspected object information C2 of the inspected passenger are extracted; at the same time, the facial image information of the inspected passenger is collected through the second camera, and the facial image information is detected, feature extracted, and identified and compared using a dynamic face recognition method to obtain the facial feature image information B2 of the inspected passenger; Perform real-time face recognition comparison on the facial feature image information B2 of the passenger being inspected and the facial feature image information B1 of the passenger. When the facial feature comparison similarity exceeds 90%, it is determined that they are the same person. The passenger's ID is searched and the passenger's ID is used as the association point to associate with the human body security inspection image information C1 and the human body suspected object information C2. The associated data - the X-ray image information A1 of the package, the suspected luggage information A2, the human body security inspection image information C1, and the suspected human body information C2 are stored in the server; The dynamic face recognition method uses Pytorch to build a residual network ResNet-50, uses a convolutional neural network model, trains the model through a deep learning algorithm, uses scored face image information to train and test the model, and uses the trained network model to identify and analyze face information.

2. The method for detecting people based on dynamic face recognition according to claim 1, characterized in that: The training of the network model of the dynamic face recognition method includes the following factors: Face angle, face missing, face lighting and face expression.

3. The method for detecting people based on dynamic face recognition according to claim 1, characterized in that: The data that have not been successfully associated - X-ray image information A1, suspected baggage information A2 and human body inspection image information C1, suspected human body information C2 are stored in the corresponding X-ray database and terahertz database respectively.

4. A person co-detection system based on dynamic face recognition, the system is used to implement the person co-detection method based on dynamic face recognition as claimed in claim 1, characterized in that: include: An X-ray machine is used to scan passenger baggage information, extract X-ray image information A1 of items in the passenger baggage, and information A2 of suspected luggage, and upload the extracted image information to a server; including: The first camera is used to dynamically collect, extract and identify the facial information of different passengers when placing the package, obtain the facial feature image information B1 of the passenger, and establish the corresponding identity ID, and send it to the server at the same time; The terahertz security inspection machine is used to scan the passenger's body information and extract the passenger's body security inspection image information C1 and the information of suspicious objects carried by the human body C2; including: The second camera is used to dynamically collect, extract and identify the facial information of the inspected passenger, obtain the passenger's facial feature picture information B2, and send it to the server; Dynamic face recognition system; The server binds the collected X-ray image information A1 of the package, the suspected luggage information A2, the passenger's body inspection image information C1, and the suspected body information C2 to the passenger's identity ID, and stores the data in the server's database and storage medium by category.

5. The person detection system based on dynamic face recognition according to claim 4 is characterized in that: The server comprises: The first image processing unit is used to receive the X-ray image information A1 of the package and the suspected luggage information A2 uploaded by the X-ray machine, and at the same time receive and identify the facial feature picture information B1 of the passenger uploaded by the first camera, and associate the passenger's identity ID with the X-ray image information A1 of the package and the suspected luggage information A2 using the time attribute of the facial feature picture information B1 as an association medium; The second image processing unit is used to receive the passenger body security inspection image information C1 and the human body suspicious object information C2 uploaded by the terahertz security inspection machine, and at the same time receive and identify the facial feature picture information B2 of the inspected passenger uploaded by the second camera; and perform real-time face recognition comparison between the facial feature picture information B2 of the inspected passenger and the facial feature picture information B1 of the passenger. After the comparison and recognition is successful, the identity ID of the corresponding passenger is searched, and the identity ID of the passenger is associated with the passenger body security inspection image information C1 and the human body suspicious object information C2.

6. A person detection system based on dynamic face recognition according to claim 5, characterized in that: The time attribute of the facial feature image information B1 refers to: the initial time of the passenger's facial information collected by the first camera and the time period when the X-ray machine transmits the package.

7. The person detection system based on dynamic face recognition according to claim 5, characterized in that: The database includes a relational database, an X-ray database and a terahertz database.

8. The person detection system based on dynamic face recognition according to claim 7, characterized in that: The X-ray database is used to store the X-ray image information A1 and the luggage suspicious object information A2 uploaded by the X-ray machine; the terahertz database is used to store the human body security image information C1 and the human body carrying suspicious object information C2 uploaded by the terahertz security inspection machine.

Images