An intelligent recognition system

Abstract

The present invention discloses an intelligent recognition system, which relates to the field of face recognition technology, specifically S1, face feature input; S2, camera installation; S3, monitoring and S4, analysis. The intelligent recognition system, based on face recognition technology of facial features, is helpful to distinguish the number and frequency of people who are not paying attention and generate images by photographing and comparing the face, and can record the people who are not in the attendance space through comparison, which is convenient for the administrator to check, and use this as a basis to analyze the attendance of the personnel, score the attendance of each person, reduce the burden of manual assessment, and help solve the problem of personnel impersonation and leaving midway through intelligent face recognition.

Description

Technical Field

[0001] The present invention relates to the technical field of face recognition, in particular to an intelligent recognition system. Background Art

[0002] The existing attendance system is both energy-consuming and time-consuming, and it is impossible to effectively monitor attendance due to the occurrence of impersonation, leaving midway, etc. Summary of the invention

[0003] In view of the deficiencies in the prior art, the present invention provides an intelligent recognition system, which solves the problem in the above background technology that the existing attendance system is both labor-intensive and time-consuming, and cannot effectively supervise attendance due to the occurrence of impersonation, leaving midway, etc.

[0004] To achieve the above objectives, the present invention is implemented through the following technical solutions: an intelligent recognition system, including the following operations:

[0005] S1. Facial feature input:

[0006] Input facial data and extract facial feature points, then store the facial data and facial feature points into the corresponding database;

[0007] S2. Camera installation:

[0008] Install the camera in a suitable location to ensure that the face is not obscured during normal use;

[0009] S3. Monitoring:

[0010] The camera monitors the situation in real time and streams the video back to the server;

[0011] S4. Analysis:

[0012] The central server receives the video and performs analysis and processing, which mainly includes the following parts:

[0013] A. According to the device information of the input video stream, determine the number of the corresponding monitoring area and obtain the corresponding face information database;

[0014] B. Detect faces appearing in the video stream, compare and identify faces stored in the corresponding database, and count the attendance of personnel;

[0015] Face detection and recognition mainly includes the following steps:

[0016] Face detection, face alignment, and face similarity measurement are called face recognition.

[0017] Optionally, in step S1, the facial feature points include eye opening and closing degrees and mouth opening and closing degrees.

[0018] Optionally, the eye opening degree is calculated as follows: based on face recognition, among the facial feature points obtained, there are six feature points marking the positions of the eyes, and the coordinates of the six feature points of a certain eye are (x1, y1), (x2, y2), (x3, y3), (x4, y4), (x5, y5), and (x6, y6) in sequence;

[0019] Let the initial minx of the point cluster be min{x1,x2,x3,x4,x5,x6}, maxx = max{x1,x2,x3,x4,x5,x6}, miny = min{y1,y2,y3,y4,y5,y6}, maxy = max{y1,y2,y3,y4,y5,y6}, determine the initial enclosing rectangle, calculate the area of ​​the enclosing rectangle, and then rotate the point cluster according to the following formula:

[0020] x′=x×cos by×sin b

[0021] y′=x×sin b+y×cos b

[0022] Where b is the rotation angle;

[0023] After the rotation, find the new minx, maxx, miny, maxy, and calculate the area at this time until the area reaches the minimum.

[0024] f=(maxy-miny) / (maxx-minx)

[0025] After calculating the two f-values, take the average value as the eye opening degree.

[0026] Optionally, the calculation of the mouth opening degree: based on face recognition, among the facial feature points obtained, there are twenty mouth feature points. The calculation of the mouth opening degree is similar to the calculation of the eye opening degree, the difference is that the initial minx = min{x1,x2,x3,…,x 19 ,x 20}, maxx= max{x1,x2,x3,…,x 19 ,x 20}, miny=min{y1,y2,y3,…,y 19 ,y 20}, maxy= max{y1,y2,y3,…,y 19 ,y 20}.

[0027] Optionally, in step S2, the number of cameras is 2-3, and the cameras are installed on the top of the wall close to the classroom podium. The preferred installation position is the corner between the wall and the other walls, and then the middle of the wall.

[0028] Optionally, in step S4, face detection uses an MTCNN algorithm, which includes a three-layer network:

[0029] (1) The first layer of P-Net will undergo convolution and pooling operations and then output the classification, that is, whether there is a face at the corresponding pixel and the regression box result.

[0030] (2) The second layer of the network uses non-maximum suppression to remove highly overlapping candidate boxes from the output of the first layer, and puts these candidate boxes into R-Net for fine operations, rejecting a large number of error boxes, and then correcting the regression boxes and using NMS to remove overlapping boxes. The output branches also have two classification and regression branches;

[0031] (3) Finally, the candidate frames that are considered to be faces by R-Net are input into O-Net for further refinement, and the wrong frames are rejected.

[0032] Optionally, in the step S4, face alignment means that after the face is detected, each component of the face can be located and the corresponding component features can be extracted. The face alignment adopts the ASM algorithm, and the algorithm is divided into a training process and a search process.

[0033] Optionally, the training process is as follows:

[0034] In the training process, first collect T training samples, the number of samples is 100, then manually mark n facial feature points, the number of facial feature points is 68, that is, n is 68, the positions of the recorded coordinate points are connected into vectors in order, and each sample is described as a vector, as follows:

[0035] X i =[x i0 ,y i0 ,x i1 ,y i1 ,…x i(n-1) ,y i(n-1) ] T

[0036] Among them, x and y represent the two-dimensional position information of the coordinate point.

[0037] Finally, without changing the point distribution model, simple translation, rotation and scaling are used to normalize multiple faces in the sample to a unified standard.

[0038] Finally, a local feature is constructed for each feature point. The method used in the present invention is to search in a rectangular box near the feature point to find a new feature point matching position in an iterative manner. The feature points taken are described as follows:

[0039] For the i-th (i < T) sample and the k-th (k < n) point, along the vertical direction perpendicular to the line connecting its two adjacent points, m points are selected on each side to form a vector of length (2m + 1). The derivative of the pixel grayscales included in this vector is calculated to obtain a local texture value g ik , and the same operation is performed on the corresponding points in other samples, then the local texture sequence corresponding to the k-th feature point is obtained:

[0040] {g i0 , g i1 , g i2 , …, g ik , …, g in}

[0041] The local features corresponding to the k-th feature point are described by the experimental mean and variance:

[0042]

[0043]

[0044] The comparison between the feature point and the sample features uses the Mahalanobis distance:

[0045]

[0046] Optionally, the search process is as follows:

[0047] In the search process, the aligned face needs to be initialized first. That is, according to the approximate face position detected by face detection, the calculated average face is subjected to an affine transformation to obtain an initial feature point model, and then the feature points are searched for;

[0048] The steps to search for feature points are as follows.

[0049] 1) For each feature point:

[0050] Iterative search is performed within the neighborhood of the feature point. Through the feature matching of local feature points, the new positions of the feature points are obtained;

[0051] Through the feature point selection algorithm, multiple candidate points are selected. The candidate points are subjected to feature extraction using the same method as above, and then matched with the sample feature point model. The matching uses the Mahalanobis distance, and the candidate point with the smallest distance is used as the center of the new feature point.

[0052] 2) The matching result is corrected using the average face model

[0053] Steps 1) and 2) are repeatedly executed until convergence;

[0054] In face similarity measurement, the cosine distance is used to determine whether the detected face and a face in the database are the same person. The calculation formula is:

[0055]

[0056] Among them, n represents the number of all features, A i represents the i-th feature of the recognized face, B i Represents the i-th feature of a face in the sample library;

[0057] The calculated cosθ is compared with the threshold given by the system. The larger the cosθ value is, the greater the possibility that the two faces are the same person.

[0058] For the person identified in the face database, based on the identification, the eye opening and mouth opening degrees are calculated, and the data stored in the two databases at the time of collection are compared. If the eye opening degree is less than the eye opening degree at the time of collection or the mouth opening degree is greater than the mouth opening degree at the time of collection, the image at this moment will be recorded;

[0059] The recorded images are stored separately in a database, which records the names of the people or other information that can indicate the identity of the people. This information is the information uniformly entered when the images are collected, the time of occurrence, and the screenshots of the screen at that time;

[0060] Based on the processing results, the system counts the total number of people who appear within a certain period of time, the number of times each person appears, and the image of each appearance. The administrator will review them and use them as the basis.

[0061] Optionally, the intelligent recognition system is applied to the field of face recognition.

[0062] The present invention provides an intelligent recognition system, which has the following beneficial effects:

[0063] This intelligent recognition system uses facial recognition technology based on facial features. By photographing and comparing the face, it is helpful to distinguish the number and frequency of inattentive people and generate images. Through comparison, it can record the people who are not in the attendance space, which is convenient for the administrator to view. Based on this, the attendance of the personnel is analyzed and scored for each person, reducing the burden of manual assessment. Intelligent face recognition is helpful to solve the problems of impersonation and leaving the team midway. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 It is a schematic diagram of the overall process of the present invention. DETAILED DESCRIPTION

[0065] See also Figure 1 The present invention provides a technical solution: an intelligent recognition system, comprising the following operations:

[0066] S1. Facial feature input:

[0067] Input facial data and extract facial feature points, then store the facial data and facial feature points into the corresponding database;

[0068] S2. Camera installation:

[0069] Install the camera in a suitable location to ensure that the face is not obscured during normal use;

[0070] S3. Monitoring:

[0071] The camera monitors the situation in real time and streams the video back to the server;

[0072] S4. Analysis:

[0073] The central server receives the video and performs analysis and processing, which mainly includes the following parts:

[0074] A. According to the device information of the input video stream, determine the number of the corresponding monitoring area and obtain the corresponding face information database;

[0075] B. Detect faces appearing in the video stream, compare and identify faces stored in the corresponding database, and count the attendance of personnel;

[0076] Face detection and recognition mainly includes the following steps:

[0077] Face detection, face alignment, and face similarity measurement are called face recognition.

[0078] In step S1, the facial feature points include the degree of eye opening and the degree of mouth opening.

[0079] Calculation of eye opening degree: Based on face recognition, among the facial feature points, there are six feature points marking the position of the eyes. The coordinates of the six feature points of one eye are (x1, y1), (x2, y2), (x3, y3), (x4, y4), (x5, y5), (x6, y6);

[0080] Let the initial minx of the point cluster be min{x1,x2,x3,x4,x5,x6}, maxx = max{x1,x2,x3,x4,x5,x6}, miny = min{y1,y2,y3,y4,y5,y6}, maxy = max{y1,y2,y3,y4,y5,y6}, determine the initial enclosing rectangle, calculate the area of ​​the enclosing rectangle, and then rotate the point cluster according to the following formula:

[0081] x'=x×cos by×sin b

[0082] y'=x×sin b+y×cos b

[0083] Where b is the rotation angle;

[0084] After the rotation, find the new minx, maxx, miny, maxy, and calculate the area at this time until the area reaches the minimum.

[0085] f=(maxy-miny) / (maxx-minx)

[0086] After calculating the two f-values, take the average value as the eye opening degree.

[0087] Calculation of mouth opening degree: Based on face recognition, among the facial feature points, there are twenty mouth feature points. The calculation of mouth opening degree is similar to that of eyes opening degree, except that the initial minx = min{x1,x2,x3,…,x 19 ,x 20}, maxx=max{x1,x2,x3,…,x 19 ,x 20}, miny=min{y1,y2,y3,…,y 19 ,y 20}, maxy=max{y1,y2,y3,…,y 19 ,y 20}.

[0088] In step S2, the number of cameras is 2-3, and the cameras are installed on the top of the wall close to the classroom podium. The preferred installation position is the corner between the wall and the other walls, and then the middle of the wall.

[0089] In step S4, face detection uses the MTCNN algorithm, which consists of a three-layer network:

[0090] (1) The first layer of P-Net will undergo convolution and pooling operations and then output the classification, that is, whether there is a face at the corresponding pixel and the regression box result.

[0091] (2) The second layer of the network uses non-maximum suppression to remove highly overlapping candidate boxes from the output of the first layer, and puts these candidate boxes into R-Net for fine operations, rejecting a large number of error boxes, and then correcting the regression boxes and using NMS to remove overlapping boxes. The output branches also have two classification and regression branches;

[0092] (3) Finally, the candidate frames that are considered to be faces by R-Net are input into O-Net for further refinement, and the wrong frames are rejected.

[0093] In step S4, face alignment means that after detecting a face, each component of the face can be located, and the corresponding component features can be extracted. The Active Shape Model (ASM) algorithm is used for face alignment, and this algorithm is divided into a training process and a search process.

[0094] The training process is as follows:

[0095] In the training process, first collect T training samples. For example, the number of samples is 100. Then manually mark n facial feature points. For example, the number of facial feature points is 68, that is, n is taken as 68 here. Connect the positions of the recorded coordinate points in sequence into a vector, and each sample is described as a vector, as follows:

[0096] X i =[x i0 ,y i0 ,x i1 ,y i1 ,…x i(n-1) ,y i(n-1) T

[0097] Among them, x and y represent the two-dimensional position information of the coordinate points.

[0098] Finally, without changing the point distribution model, use simple translation, rotation, and scaling to normalize multiple faces in the sample to a unified standard.

[0099] Finally, construct local features for each feature point. The method selected in the present invention is to search in a rectangular box near the feature point and find a new matching position of the feature point in an iterative manner. The description of the feature point is as follows:

[0100] For the i-th (i < T) sample and the k-th (k < n) point, along the vertical direction perpendicular to the line connecting its two adjacent points, select m points on each side to form a vector with a length of (2m + 1). Take the derivative of the pixel grayscale included in this vector to obtain a local texture value g ik , and perform the same operation on the corresponding points in other samples to obtain the local texture sequence corresponding to the k-th feature point:

[0101] {g i0 ,g i1 ,g i2 ,…,g ik ,…,g in}

[0102] Use the experimental mean and variance to describe the local features corresponding to the k-th feature point:

[0103]

[0104] ​

[0105] The comparison between feature points and sample features uses Mahalanobis distance:

[0106]

[0107] The search process is as follows:

[0108] In the search process, the face alignment must be initialized first, that is, according to the approximate face position detected by the face, the average face is affine transformed to obtain an initial feature point model, and then the feature points are searched;

[0109] The steps for searching feature points are as follows.

[0110] 1) For each feature point:

[0111] Perform iterative search in the neighborhood of feature points and obtain new feature point positions through feature matching of local feature points;

[0112] Through the feature point selection algorithm, multiple candidate points are selected. The candidate points are extracted using the same method as above, and then matched with the sample feature point model. The matching uses the Mahalanobis distance, and the candidate point with the smallest distance is used as the new feature point center.

[0113] 2) Use the average face model to correct the matching results

[0114] Repeat 1) and 2) until convergence;

[0115] In the face similarity measurement, the cosine distance is used to determine whether the detected face is the same person as a face in the database. The calculation formula is:

[0116]

[0117] Among them, n represents the number of all features, A i represents the i-th feature of the recognized face, B i Represents the i-th feature of a face in the sample library;

[0118] The calculated cosθ is compared with the threshold given by the system. The larger the cosθ value is, the greater the possibility that the two faces are the same person.

[0119] For the person identified in the face database, based on the identification, the eye opening and mouth opening degrees are calculated, and the data stored in the two databases at the time of collection are compared. If the eye opening degree is less than the eye opening degree at the time of collection or the mouth opening degree is greater than the mouth opening degree at the time of collection, the image at this moment will be recorded;

[0120] The recorded images are stored separately in a database, which records the names of the people or other information that can indicate the identity of the people. This information is the information uniformly entered when the images are collected, the time of occurrence, and the screenshots of the screen at that time;

[0121] Based on the processing results, the system counts the total number of people who appear within a certain period of time, the number of times each person appears, and the image of each appearance. The administrator will review them and use them as the basis.

[0122] Intelligent recognition systems are applied in the field of face recognition.

[0123] This intelligent recognition system uses facial recognition technology based on facial features. By photographing and comparing the face, it is helpful to distinguish the number and frequency of inattentive people and generate images. Through comparison, it can record the people who are not in the attendance space, which is convenient for the administrator to view. Based on this, the attendance of the personnel is analyzed and scored for each person, reducing the burden of manual assessment. Intelligent face recognition is helpful to solve the problems of impersonation and leaving the team midway.

[0124] In summary, when using the intelligent recognition system, firstly, before using the system, face images are collected, face data is input according to the actual situation, face feature points are extracted, and face feature points and data such as eye opening and closing degree and mouth opening and closing degree are stored in the database corresponding to each class;

[0125] Then install the camera in a suitable position, and ensure that the face is not blocked during use;

[0126] After the camera is installed, it can be used to transmit the real-time monitoring images back to the server, and the central server will process and identify the video stream;

[0127] Finally, based on the server processing results, the system counts the total number of inattentive people in the space, the number of times each person appears, and the image of each appearance. The administrator will check and use this as a basis to analyze the attendance situation;

[0128] The core of the recognition processing of the central server is to use face recognition technology to collect attendance information, and on this basis calculate the degree of eye opening and mouth opening, and use the calculated data as the basis for judging whether the person is focused. The specific processing flow is as follows:

[0129] A.According to the device information of the input video stream, obtain the corresponding face information database;

[0130] B. Detect faces appearing in the video stream, compare and identify faces stored in the database, and count attendance information;

[0131] C. For the person identified in the face database, on the basis of identification, calculate the degree of eye opening and mouth opening, and compare the data stored in the two databases at the time of collection. If the degree of eye opening is less than the degree of eye opening at the time of collection or the degree of mouth opening is greater than the degree of mouth opening at the time of collection, then it will be recorded at this moment that the person is not paying attention;

[0132] D. The images showing inattention are saved separately in a database, which records the name of the inattention subject (or other information that can indicate the identity, which is the information uniformly entered when the image is collected), the time when the inattention occurred, and the screenshot of the screen at that time;

[0133] This intelligent recognition system uses facial recognition technology based on facial features. By photographing and comparing the face, it is helpful to distinguish the number and frequency of inattentive people and generate images. Through comparison, it can record the people who are not in the attendance space, which is convenient for the administrator to view. Based on this, the attendance of the personnel is analyzed and scored for each person, reducing the burden of manual assessment. Intelligent face recognition is helpful to solve the problems of impersonation and leaving the team midway.

Claims

1. An intelligent recognition system, characterized in that: The following operations are included: S1. Facial feature input: Input facial data and extract facial feature points, then store the facial data and facial feature points into the corresponding database; S2. Camera installation: Install the camera in a suitable location to ensure that the face is not obscured during normal use; S3. Monitoring: The camera monitors the situation in real time and streams the video back to the server; S4. Analysis: The central server receives the video and performs analysis and processing, which includes the following parts: A. According to the device information of the input video stream, determine the number of the corresponding monitoring area and obtain the corresponding face information database; B. Detect faces appearing in the video stream, compare and identify faces stored in the corresponding database, and count the attendance of personnel; Face detection and recognition includes the following steps: Face detection, face alignment, and face similarity measurement are face recognition. Facial feature points include the degree of eye opening and mouth opening. Based on face recognition, six feature points are obtained to mark the position of the eyes. The coordinates of the six feature points of one eye are: , , , , , ; Let the cluster of points initially ; ; ； , determine the initial bounding rectangle, find the area of ​​the bounding rectangle, and then rotate the point cluster according to the following formula: ； ； Where b is the rotation angle; After the rotation, find the new minx, maxx, miny, maxy, and calculate the area at this time until the area reaches the minimum. ； After calculating the two f values, take the average value as the eye opening degree. The search process is as follows: In the search process, the face alignment must be initialized first, that is, according to the approximate face position detected by the face, the average face is affine transformed to obtain an initial feature point model, and then the feature points are searched; The steps for searching feature points are as follows: 1) For each feature point: Perform iterative search in the neighborhood of feature points and obtain new feature point positions through feature matching of local feature points; Through the feature point selection algorithm, multiple candidate points are selected. The candidate points are extracted using the same method as above, and then matched with the sample feature point model. The matching uses the Mahalanobis distance, and the candidate point with the smallest distance is used as the new feature point center. 2) Use the average face model to correct the matching results Repeat 1) and 2) until convergence; In the face similarity measurement, the cosine distance is used to determine whether the detected face is the same person as a face in the database. The calculation formula is: ； Where n represents the number of all features. represents the i-th feature of the recognized face, shows the i-th feature of a face in the sample library; Calculated Compared with the threshold given by the system, The larger the value, the greater the possibility that the two faces are of the same person; For the person identified in the face database, based on the identification, the eye opening and mouth opening degrees are calculated, and the data stored in the two databases at the time of collection are compared. If the eye opening degree is less than the eye opening degree at the time of collection or the mouth opening degree is greater than the mouth opening degree at the time of collection, the image at this moment will be recorded; The recorded images are stored separately in a database, which records the names of the people or other information that can indicate the identity of the people. This information is the information uniformly entered when the images are collected, the time of occurrence, and the screenshots of the screen at that time; Based on the processing results, the system counts the total number of people who appear within a certain period of time, the number of times each person appears, and the image of each appearance. The administrator will review them and use them as the basis.

2. An intelligent identification system according to claim 1, characterized in that: The calculation of the mouth opening degree: Based on face recognition, among the facial feature points, there are twenty mouth feature points. The calculation of the mouth opening degree is similar to the calculation of the eye opening degree. The difference is that the initial ； ； ； 。 3. An intelligent identification system according to claim 2, characterized in that: In the step S2, the number of cameras is 2-3, and the cameras are installed on the top of the wall close to the classroom podium. The preferred installation position is the corner between the wall and the other walls, and then the middle of the wall.

4. An intelligent identification system according to claim 3, characterized in that: In step S4, face detection uses the MTCNN algorithm, which includes a three-layer network: (1) The first layer of P-Net will undergo convolution and pooling operations and then output the classification, that is, whether there is a face at the corresponding pixel and the regression box result; (2) The second layer of the network uses non-maximum suppression to remove highly overlapping candidate boxes from the output of the first layer, and puts these candidate boxes into R-Net for fine operations, rejecting a large number of error boxes, and then correcting the regression boxes. It also uses NMS to remove overlapping boxes. The output branches also have two classification and regression branches. (3) Finally, the candidate frames that are considered to be faces by R-Net are input into O-Net for further refinement, and the wrong frames are rejected.

5. An intelligent identification system according to claim 4, characterized in that: In the step S4, face alignment means that after the face is detected, each component of the face can be located and the corresponding component features can be extracted. The face alignment adopts the ASM algorithm, which is divided into a training process and a search process.

6. An intelligent identification system according to claim 5, characterized in that: The training process is as follows: During the training process, we first collect T training samples, the number of samples is 100, and then manually mark n facial feature points, the number of facial feature points is 68, that is, n here is 68, and the positions of the recorded coordinate points are connected into vectors in order. Each sample is described as a vector, as follows: ;in, Represents the two-dimensional position information of the coordinate point; Finally, without changing the point distribution model, multiple faces in the sample are normalized to a unified standard using simple translation, rotation, and scaling; Finally, a local feature is constructed for each feature point, and a rectangular box search is performed near the feature point to find a new matching position of the feature point in an iterative manner. The feature points taken are described as follows: For Samples, points, along the perpendicular direction to the line connecting the two adjacent points, each of the two selects m points to form a line with a length of The vector contains the pixel grayscale derivative to obtain a local texture value , perform the same operation on the corresponding points in other samples, and you will get the local texture sequence corresponding to the kth feature point: ； Experimental mean and variance are used to describe the local features corresponding to the kth feature point: ; ; The comparison between feature points and sample features uses Mahalanobis distance: 。 7. An intelligent identification system according to any one of claims 1 to 6, characterized in that: The intelligent recognition system is applied to the field of face recognition.

Images