Multi-spectral face liveness detection method and system based on neural network

Abstract

The application provides a kind of multispectral face living body detection method and system based on neural network, comprising: obtaining multispectral image of real person and simulation head model;The dimension conversion is carried out to multispectral image;Spectral feature extraction;The data correction is carried out to the extracted spectral feature;Class calibration is carried out to the data after correction;Obtain n group spectral feature data and n group label;Design neural network infrastructure;Select effective spectral dimension in n group spectral feature data;Adjust the input layer of neural network infrastructure, determine the final neural network structure;The final training data is used to train the final neural network structure, and the final face living body detection model is saved after training;Complete multispectral face living body detection according to the final face living body detection model.The technical scheme of the application is used to solve the technical problems of low classification accuracy, large data volume, poor detection effect and slow detection speed in the prior art.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of face recognition, and particularly relates to a multispectral face living body detection method and system based on a neural network. BACKGROUND

[0002] Face recognition is a kind of biometric technology based on facial feature information for identity recognition, which has a wide application prospect in access control system, monitoring, identity authentication, secure payment, etc.Among them, the living body detection is an important part of face recognition. At present, the face recognition test system will design the face movement as the condition of face living body detection (Shakhnarovich G, Moghaddam B. Handbook of Face Recognition. 2004.). The living body detection can pass through the blink of an eye (Gang P, Lin S, Wu Z, et al. Eye blink-based Anti-Spoofing in Face Recognition from a Generic Web camera [C] / / IEEE 11th International Conference on Computer Vision, ICCV 2007, Rio de Janeiro, Brazil, October 14-20, 2007. IEEE, 2007.) (Kollreider K, Fronthaler H, Bigun J. Verifying Liveness by Multiple Experts in Face Biometrics [C] / / 2008.), shaking head, nodding (Kollreider K, Fronthaler H, Bigun J. Verifying Liveness by Multiple Experts in Face Biometrics [C] / / 2008.) (K. Kollreider, H. Fronthaler and J. Bigun, "Evaluating liveness by face images and the structure tensor," Fourth IEEE Workshop on Automatic Identification Advanced Technologies (AutoID'05), 2005, pp. 75-80, doi: 10.1109 / AUTOID.2005.20.), opening mouth (Wagner M, Chetty G. "liveness" verification in audio-video authentication [C] / / INTERSPEECH 2004-ICSLP, 8th International Conference on Spoken Language Processing, Jeju Island, Korea, October 4-8, 2004. 2004.) and other combined actions. This method needs good visible light and enough time, and has a certain negative psychological impact on the user. In addition, this method is mainly suitable for flat image detection, and is difficult to identify simulated portraits, masks, 3D animations and other counterfeit articles, and has certain limitations.

[0003] A large number of researches have been done at home and abroad, among which a typical way is to try to use human face skin color as the basis of portrait (Lei Ming, Zhang Junying, Dong Jiyang. A skin color detection algorithm under variable lighting conditions [J]. Computer Engineering and Applications, 2002, 38(24): 3.) because the RGB data is very common and easy to collect, so it has been widely used.

[0004] In view of the problem of less RGB information data, the subsequent traditional methods such as svm are used to extract and classify the features of multispectral face line (Hou YL, Hao X, Wang Y, et al. Multispectral face liveness detection method based on gradient features [J]. Optical Engineering, 2013, 52(11): 113102.) (Zhang Z, Dong Y, Zhen L, et al. Face liveness detection by learning multispectral reflectance distributions [C] / / Automatic Face & Gesture Recognition and Workshops (FG2011), 2011 IEEE International Conference on. IEEE, 2011.) (Song X, Gao S, Chen C. A multispectral feature fusion network for robust pedestrian detection [J]. AEJ-Alexandria Engineering Journal, 2020, 60(1).).

[0005] With the development of deep learning, deep learning methods have been used at home and abroad (Zhang P, Zou F, Wu Z, et al. FeatherNets: Convolutional Neural Networks as Light as Feather for Face Anti-spoofing [C] / / 2019 IEEE / CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). IEEE, 2019.) (—Phan, H., & Nguyen, A. (2022). DeepFace-EMD: Re-Ranking Using Patch-Wise Earth Mover’s Distance Improves Out-of-Distribution Face Identification. In Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition (pp. 20259-20269).) (—Dhar, P., Kumar, A., Kaplan, K., Gupta, K., Ranjan, R., & Chellappa, R. (2022). EyePAD++: A Distillation-based approach for joint Eye Authentication and Presentation Attack Detection using Periocular Images. In Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition (pp. 20218-20227).), which requires a large amount of training data to train a relatively complex neural network to extract features from deep features for identifying the authenticity of images.

[0006] However, if the skin is used as a criterion, when the light environment changes or a simulated human model is used, the visible light RGB plane information is less, the RGB image cannot distinguish whether it is a real person image, and is easily affected by environmental factors, resulting in poor detection effect. Using traditional manual feature extraction method for feature extraction, the accuracy is difficult to achieve the expected target. Using the rgb image method of deep learning, 1) dataset: first, a large amount of training data under various environments is needed, which needs a lot of manpower cost for collection and calibration; 2) model: a relatively complex network structure is needed for training and testing, which takes a long time to test and needs a certain standard hardware facility. SUMMARY

[0007] The application provides a multi-spectral face living body detection method and system based on a neural network, which can solve the technical problems of low classification accuracy, large data volume, poor detection effect and slow detection speed in the prior art.

[0008] According to an aspect of the application, a multi-spectral face living body detection method based on a neural network is provided, which comprises the following steps: step one, collecting spectral data of real people and simulated head models to obtain real multi-spectral images and simulated multi-spectral images; step two, converting the dimensions of the collected real multi-spectral images and simulated multi-spectral images; step three, for any dimensionally converted multi-spectral image in the real multi-spectral images and the simulated multi-spectral images, extracting the spectral features of the target region pixels in the dimensionally converted multi-spectral image; step four, for any dimensionally converted multi-spectral image, correcting the extracted spectral features to obtain spectral feature data of any dimensionally converted multi-spectral image; step five, for any dimensionally converted multi-spectral image, classifying the corrected data to obtain the labels of the spectral feature data; step six, for other dimensionally converted multi-spectral images in the real multi-spectral images and the simulated multi-spectral images, repeating steps three to five to obtain n sets of spectral feature data and n sets of labels of the dimensionally converted real multi-spectral images and the simulated multi-spectral images; step seven, designing a neural network infrastructure based on the n sets of spectral feature data and the n sets of labels; step eight, using the n sets of spectral feature data and the n sets of labels collected in step six as training data, using the neural network infrastructure in step seven as a training model, selecting effective spectral dimensions from the n sets of spectral feature data to form the final training data; step nine, adjusting the input layer of the neural network infrastructure in step seven according to the final training data in step eight to determine the final neural network structure; step ten, training the final neural network structure using the final training data, and saving it as a final face living body detection model after training; step eleven, completing multi-spectral face living body detection according to the final face living body detection model.

[0009] Further, in step three, the spectral feature extraction of the target region pixels in the dimensionally converted multispectral image specifically includes: positioning the face region in the dimensionally converted multispectral image, and randomly selecting a region block of a set size in the face region, averaging the region block of the set size in the plane dimension to obtain the spectral information in the set region range, and completing the spectral feature extraction of the target region pixels.

[0010] Further, in step four, the data correction of the extracted spectral features specifically includes: multiplying the spectral information in the set region range with a set matrix to obtain the spectral feature data of the set region points.

[0011] Further, in step five, for any dimensionally converted multispectral image, the spectral feature data obtained in step four is labeled true or false according to the true or false of the portrait where the feature is located, and the label of the spectral feature data is obtained.

[0012] Further, in step seven, the neural network infrastructure is based on a lightweight BP neural network, and the neural network infrastructure includes an input layer, a hidden layer, and an output layer.

[0013] Further, the hidden layer includes a first hidden sub-layer, a second hidden sub-layer, and a third hidden sub-layer, the number of neurons of the first hidden sub-layer is 36, the number of neurons of the second hidden sub-layer is 48, and the number of neurons of the third hidden sub-layer is 36.

[0014] Further, in step eight, the neural network infrastructure in step seven is used as a training model to train each spectral feature dimension data in the n groups of spectral feature data collected in step six, to determine whether the classification accuracy of any spectral feature dimension data is greater than or equal to a set classification accuracy, and to use the spectral feature dimension greater than or equal to the set classification accuracy as an effective spectral dimension. All effective spectral dimension data and corresponding labels form the final training data.

[0015] Further, in step one, an ximea multispectral image collector is used to collect multispectral image data of real portraits and simulation head models respectively, and real multispectral images and simulation head model multispectral images are collected and stored in bayer format.

[0016] Further, in step two, the real multispectral images and simulation head model multispectral images in bayer format are converted into an image format of (h / 5)*(w / 5)*25 according to the spectral range.

[0017] According to another aspect of the present application, a neural network-based multispectral face living body detection system is provided, which uses the neural network-based multispectral face living body detection method as described above to perform multispectral face living body detection.

[0018] The technical solution of the present application provides a neural network-based multispectral face living body detection method, which combines multispectral imaging and spectral detection, can not only analyze the internal component content of an object, but also visually analyze the distribution thereof, and therefore uses multispectral imaging technology as an information collection means for living body portrait recognition, can enrich the diversity of data information, and improve the living body portrait recognition accuracy. However, the multi-dimensional spectral data not only provides more information, but also brings higher feature extraction difficulty and computation amount. The method selects effective spectral dimensions from n groups of spectral feature data to form the final training data by screening the multi-dimensional spectral data, which greatly reduces the data amount, reduces the labor cost of data collection, and improves the calculation speed. In addition, the method uses multispectral images as collection data and uses a neural network method to quickly and efficiently perform living body face recognition, which does not require high hardware equipment requirements, reduces the project cost, and has high accuracy. Therefore, compared with the prior art, the multispectral face living body detection method provided by the present application uses multispectral images instead of rgb images for data collection, and the collected data is in the form of a multi-channel dimensional vector instead of the spatial dimension of an rgb image, which can not only reduce the complex spatial dependence on rgb images, but also make the recognition accuracy more obvious due to the advantages of the multi-channel dimensional information of the multispectral images. After single-dimensional training and testing, the part with higher recognition rate is selected to form new training and testing samples, which can eliminate redundant features, reduce data amount, and improve precision. The multispectral face living body detection method provided by the present application can effectively solve the technical problems of low classification accuracy, large data amount, poor detection effect, and slow detection speed in the prior art. BRIEF DESCRIPTION OF DRAWINGS

[0019] The accompanying drawings included to provide a further understanding of the embodiments of the present application, constitute a part of the specification and illustrate embodiments of the present application, and together with the text description, explain the principles of the present application. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0020] Figure 1 A structural schematic diagram of a neural network infrastructure provided according to a specific embodiment of the present application is shown;

[0021] Figure 2A schematic diagram of setting a matrix is shown. DETAILED DESCRIPTION

[0022] It should be noted that the embodiments and features of the embodiments in the present application can be combined with each other without conflict. The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings of the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. The description of the at least one example embodiment is actually only illustrative, but not as any limitation on the present application and its application or use. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor are within the scope of protection of the present application.

[0023] It should be noted that the terms used herein are only intended to describe specific embodiments, and are not intended to limit the example embodiments according to the present application. As used herein, the singular form is intended to include the plural form, unless the context clearly indicates otherwise, and it should also be understood that when the terms "comprise" and / or "include" are used in the specification, there is a presence of the features, steps, operations, devices, components and / or combinations thereof.

[0024] Unless specifically stated otherwise, the relative arrangement of components and steps, numerical expressions, and numerical values set forth in the examples contained herein are not limiting. It should be understood that the various parts shown in the drawings are not necessarily drawn to scale in proportion. The techniques, methods, and devices known to those of ordinary skill in the relevant art can not be discussed in detail, but should be considered as part of the authorized description. In all examples shown and discussed herein, any specific value should be interpreted as merely exemplary, and not as a limitation. Therefore, other examples of example embodiments can have different values. It should be noted that similar reference numbers and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further discussed in subsequent drawings.

[0025] As Figure 1 and Figure 2As shown, the specific embodiment according to the present application provides a neural network-based multispectral face liveness detection method, which comprises the following steps: Step 1, spectral data of real people and simulation head models are collected to obtain multispectral images of real people and multispectral images of simulation head models; Step 2, the collected multispectral images of real people and multispectral images of simulation head models are respectively subjected to dimension conversion; Step 3, for any dimension-converted multispectral image in the multispectral images of real people and simulation head models, spectral feature extraction is performed on the target region pixels in the dimension-converted multispectral image; Step 4, for any dimension-converted multispectral image, data correction is performed on the extracted spectral features to obtain spectral feature data of any dimension-converted multispectral image; Step 5, for any dimension-converted multispectral image, class labeling is performed on the corrected data to obtain labels of the spectral feature data; Step 6, for other dimension-converted multispectral images in the multispectral images of real people and simulation head models, steps 3 to 5 are repeated to obtain n sets of spectral feature data and n sets of labels of the dimension-converted multispectral images of real people and simulation head models; Step 7, a neural network infrastructure is designed based on the n sets of spectral feature data and n sets of labels; Step 8, using the n sets of spectral feature data and n sets of labels collected in Step 6 as training data and using the neural network infrastructure in Step 7 as a training model, effective spectral dimensions are selected from the n sets of spectral feature data to form final training data; Step 9, according to the final training data in Step 8, the input layer of the neural network infrastructure in Step 7 is adjusted to determine the final neural network structure; Step 10, the final neural network structure is trained using the final training data, and after training, it is saved as a final face liveness detection model; Step 11, multispectral face liveness detection is completed according to the final face liveness detection model.

[0026] By using the configuration mode, a multispectral face living body detection method based on a neural network is provided. The method combines multispectral imaging and spectral detection, can analyze the internal component content of an object, and can visually analyze the distribution of the object. Therefore, the multispectral imaging technology is used as an information collection means for living body portrait recognition, the diversity of data information is enriched, and the living body portrait recognition accuracy is improved. However, the multispectral data provides more information and brings higher feature extraction difficulty and calculation amount. The method selects effective spectral dimensions from n groups of spectral feature data to form final training data, thereby greatly reducing the data amount, reducing the labor cost of data collection, and improving the calculation speed. In addition, the method uses multispectral images as collection data and uses a neural network method to quickly and efficiently perform living body face recognition. This method does not require high hardware equipment requirements, reduces project costs, and has high accuracy. Therefore, compared with the prior art, the multispectral face living body detection method provided by the present application uses multispectral images instead of rgb images for data collection. The collected data is in the form of a multi-channel dimension vector instead of the spatial dimension of an rgb image. The method can reduce the complex spatial dependence on rgb images, and the advantages of the multi-channel dimension information of the multispectral image make the recognition accuracy more obvious. After single dimension training and testing, the part with higher recognition rate is selected to form a new training and testing sample, so that redundant features can be eliminated, the data amount can be reduced, and the precision can be improved. The multispectral face living body detection method provided by the present application can effectively solve the technical problems of low classification accuracy, large data amount, poor detection effect, and slow detection speed in the prior art.

[0027] Specifically, in the present application, in order to realize multispectral face living body detection, first, spectral data of real people and simulation head models need to be collected to obtain real person multispectral images and simulation head model multispectral images. As a specific embodiment of the present application, in step one, an existing ximea multispectral image collector is used to collect multispectral image data of real people and simulation head models. The machine can collect spectral information of 25 spectral bands, and collect and store real person multispectral images and simulation head model multispectral images in bayer format.

[0028] Further, after obtaining the real person multispectral images and the simulation head model multispectral images, the collected real person multispectral images and the simulation head model multispectral images can be converted in dimension. In the present application, the collected real person multispectral images and the simulation head model multispectral images are converted in dimension, that is, the real person multispectral images and the simulation head model multispectral images in bayer format (that is, a planar image of h*w, wherein h represents the length of the planar image, and w represents the width of the planar image) are converted into an image format of (h / 5)*(w / 5)*25 according to the spectral bands, wherein 25 is the longitudinal spectral dimension.

[0029] In the present application, after the collected real person multi-spectral images and the simulated head model multi-spectral images are dimensionally converted, the spectral feature extraction of the target region pixels in the dimensionally converted multi-spectral images can be performed for any dimensionally converted multi-spectral image. Specifically, in step three, the spectral feature extraction of the target region pixels in the dimensionally converted multi-spectral images specifically includes: positioning the face region in the dimensionally converted multi-spectral images, and randomly selecting a region block of a set size in the face region, and averaging the region block of the set size in the plane dimension to obtain the spectral information in the set region range, and completing the spectral feature extraction of the target region pixels.

[0030] As a specific embodiment of the present application, the face region in the dimensionally converted multi-spectral images is positioned, and a 5*5*25 region block is randomly selected in the face region, and then averaged in the plane dimension, that is, a 1*25-dimensional feature spectral data is obtained through the 5*5*25 region block, and the data in 1*25 is the summation average value of each dimension point in the 5*5 range, and at this time the 1*25-dimensional feature spectral data represents the spectral information in the region range.

[0031] Further, after the spectral feature is extracted, the data correction of the extracted spectral feature can be performed for any dimensionally converted multi-spectral image to obtain the spectral feature data of any dimensionally converted multi-spectral image. Specifically, in step four, the data correction of the extracted spectral feature specifically includes: multiplying the spectral information in the set region range with a set matrix to obtain the spectral feature data of the set region point.

[0032] As a specific embodiment of the present application, due to the ximea camera characteristics, the 1*25 spectral feature data extracted needs to be corrected, and a fixed 24*25 matrix (matrix as shown in Figure 2 ) is multiplied with the camera to finally obtain a 1*24-dimensional feature, and at this time the feature is a group of 1*24-dimensional spectral features of a region point on a picture.

[0033] Further, after the spectral feature data of any dimensionally converted multi-spectral image is obtained, the corrected data can be classified for any dimensionally converted multi-spectral image to obtain the label of the spectral feature data. Specifically, in step five, for any dimensionally converted multi-spectral image, the spectral feature data obtained in step four is true or false labeled according to the true or false of the portrait where the feature is located to obtain the label of the spectral feature data.

[0034] As a specific embodiment of the present application, the 1*24-dimensional spectral feature obtained in step four is labeled as true or false according to the true or false of the portrait where the feature is located, and finally a 1*2 label (for example: real person 01, fake person 10) is obtained.

[0035] Further, after obtaining the label of the spectral feature data, the steps three to five can be repeated for the multi-spectral images of the real person and the multi-spectral images of the simulation head model after the conversion of other dimensions, to obtain n sets of spectral feature data and n sets of labels of the multi-spectral images of the real person and the simulation head model after the conversion of dimensions. The n sets of spectral feature data are n sets of real and fake face spectral data, each set being 24-dimensional, i.e. n*24. At the same time, each set of data has a corresponding real and fake face label, i.e. n*2.

[0036] In the present application, after obtaining n sets of spectral feature data and n sets of labels of the multi-spectral images of the real person and the simulation head model, a neural network basic framework can be designed based on the n sets of spectral feature data and the n sets of labels. In step seven, the neural network basic framework is based on a lightweight BP neural network, and the neural network basic framework includes an input layer, a hidden layer and an output layer.

[0037] As a specific embodiment of the present application, the neural network basic framework is designed in step S7. The neural network structure designed for the task of face liveness detection is as follows: Figure 1 The network is an end-to-end lightweight neural network, mainly including an input layer (such as 24-dimensional), a hidden layer, and an output layer (such as 2-dimensional, real and fake face label). The detailed information of the neural network is as follows: in order to improve the speed, the present patent is designed to use a lightweight BP neural network as the basis, and the final network layer setting is verified by experiments as follows: the input layer is adjustable according to the dimension of the input data (such as 24 neurons), the output layer has 2 neurons (real or fake face label), the middle hidden layer has three layers: the first hidden layer, the second hidden layer and the third hidden layer, the number of neurons of the first hidden layer is 36, the number of neurons of the second hidden layer is 48, and the number of neurons of the third hidden layer is 36. The number of hidden layers can be adjusted experimentally according to different tasks and spectral bands, and the input and output are determined according to the input requirements and prediction results of different tasks.

[0038] Further, after completing the design of the neural network basic framework, n sets of spectral feature data and n sets of labels collected in step six can be used as training data, and the neural network basic framework in step seven can be used as a training model to select effective spectral dimensions from the n sets of spectral feature data to form the final training data.

[0039] Specifically, in step eight, the neural network infrastructure in step seven is used as a training model to train each spectral feature dimension data in the n sets of spectral feature data collected in step six, to determine whether the classification accuracy of any spectral feature dimension data is greater than or equal to the set classification accuracy, and to take the spectral feature dimension greater than or equal to the set classification accuracy as an effective spectral dimension. The effective spectral dimension data and the corresponding labels form the final training data.

[0040] As a specific embodiment of the present application, the n sets of preprocessed images collected in step six are used as training data in step eight to screen the effective spectral dimension. After the camera acquisition and preprocessing in steps one to five, the 24-dimensional spectrum in the n*24-dimensional spectral data obtained finally is not all spectral information, and in order to improve efficiency and eliminate redundant spectral dimensions, the present application attempts to screen the spectral dimensions in the n sets of 24-dimensional data, that is, the n*24 input is divided into 24 sets of n*1 data, and the corresponding label output is also 24 sets of n*2 output data, thereby forming the preprocessed feature data for dimension screening.

[0041] In step eight, the base neural network in step seven is used as a training model. According to the preprocessed input feature of dimension screening, there are 24 categories, each category has n*1 feature dimension, n is the number of data groups, and 1 is the feature dimension of data, so that the input layer neuron dimension is defined as 1, and the number of other neurons (hidden layer, output layer) is unchanged. The effective spectral dimension is selected from the preprocessed spectral data. Because the data is divided into 24 sets of n*1 data, the training is performed 24 times. The m spectral feature dimensions with a classification accuracy >=70% in the 24 training results are retained (for example, if 7 training classification accuracies >70% are obtained in the 24 training results, the 7 spectra are considered to be effective spectral dimensions for real and false face recognition), and the accuracy of 70% can be determined according to the task performance requirement.

[0042] According to the number of effective spectral dimensions m obtained from the 24 training results, the corresponding m-dimensional data is selected from the n*24 preprocessed data obtained in step five as the final experimental training data, that is, the training data is changed from the preprocessed 24-dimensional data to the effective m-dimensional data, and the final training data is n*m, wherein n is the number of data groups, and m is the effective spectral dimension. As a specific embodiment of the present application, if 7 training classification accuracies >70% are obtained in the 24 training results, m is equal to 7.

[0043] Further, after the final training data is obtained, the input layer of the neural network framework in step seven is adjusted according to the final training data in step eight to determine the final neural network structure. As a specific embodiment of the present application, assuming that the classification accuracy of 7 out of 24 training results is > 70%, m is equal to 7, and the final training data is n*7. According to the input data dimension 7 obtained in step eight, the neural network framework is adjusted, the input layer of the final neural network framework is adjusted to 7 dimensions, the output layer is 2 dimensions, the intermediate hidden layer has three layers, and the number of neurons is 36, 48, and 36, respectively.

[0044] Then, the final neural network structure is trained using the final training data, and after the training is completed, it is saved as a final face liveness detection model. After obtaining the final face liveness detection model, for any to-be-tested data, m-dimensional data in the to-be-tested data is selected and input into the face liveness detection model, and multi-spectral face liveness detection is completed.

[0045] According to another aspect of the present application, a neural network-based multi-spectral face liveness detection system is provided, which uses the neural network-based multi-spectral face liveness detection method as described above to perform multi-spectral face liveness detection.

[0046] By using the configuration mode, a multi-spectral face living body detection system based on a neural network is provided, which combines multi-spectral imaging and spectral detection, can not only analyze the internal component content of an object, but also can visually analyze the distribution of the object, and therefore, using multi-spectral imaging technology as an information collection means for living body portrait recognition can enrich the diversity of data information and improve the living body portrait recognition accuracy. However, the multi-dimensional spectral data not only provides more information, but also brings higher feature extraction difficulty and calculation amount. The method selects effective spectral dimensions from n groups of spectral feature data to form the final training data, which greatly reduces the data amount, reduces the labor cost of data collection, and improves the calculation speed. In addition, the method uses multi-spectral images as collection data and uses a neural network method to quickly and efficiently perform living body face recognition, which does not require high hardware equipment requirements, reduces the project cost, and has high accuracy. Therefore, compared with the prior art, the multi-spectral face living body detection system provided by the present application uses multi-spectral images instead of rgb images for data collection, and the collected data is in the form of a multi-channel dimension vector instead of the spatial dimension of an rgb image, which can not only reduce the complex spatial dependence on rgb images, but also make the recognition accuracy more obvious due to the advantages of multi-spectral image channel dimension information. After single-dimension training and testing, the part with higher recognition rate is selected to form a new training and testing sample, which can eliminate redundant features, simplify the data amount, and improve the precision. The multi-spectral face living body detection system provided by the present application can effectively solve the technical problems of low classification accuracy, large data amount, poor detection effect, and slow detection speed in the prior art.

[0047] In order to have a further understanding of the present application, the following Figure 1 and Figure 2 The multi-spectral face living body detection method based on a neural network provided by the present application will be described in detail

[0048] As shown in Figure 1 and Figure 2 According to the embodiment of the present application, a multi-spectral face living body detection method based on a neural network is provided, which specifically includes the following steps.

[0049] Step one, collect spectral data of real people and simulation head models to obtain multi-spectral images of real people and simulation head models. In this embodiment, an existing ximea multi-spectral image collector is used to collect multi-spectral image data of real people and simulation head models. The machine can collect spectral information of 25 spectral bands and collect and store in bayer format.

[0050] Step two, the collected multi-spectral images of real people and the multi-spectral images of the simulation head model are respectively converted in dimension. In this embodiment, the planar image in bayer format of h*w is converted into an image format of (h / 5)*(w / 5)*25 according to the spectral band, where 25 is the vertical spectral dimension.

[0051] Step three, the spectral feature of the target region pixel in the multi-spectral image converted in dimension is extracted for any multi-spectral image converted in dimension in the multi-spectral images of real people and the multi-spectral images of the simulation head model. In this embodiment, the face region in the multi-spectral image converted in dimension is located, and a 5*5*25 region block is randomly selected in the face region, then the 5*5*25 region block is averaged in the planar dimension to obtain a 1*25-dimensional feature spectral data, and each dimension point in the 1*25 is the sum average value of the data in the range of 5*5. At this time, the 1*25-dimensional feature spectral data represents the spectral information in the region range.

[0052] Step four, the extracted spectral feature is data-corrected to obtain the spectral feature data of the multi-spectral image converted in dimension for any multi-spectral image converted in dimension. In this embodiment, a fixed 24*25 matrix (the matrix is shown as Figure 2 ) is multiplied with the matrix to finally obtain a 1*24-dimensional feature. At this time, the feature is a group of 1*24-dimensional spectral features of a region point on a picture.

[0053] Step five, the corrected data is classified to obtain the label of the spectral feature data for any multi-spectral image converted in dimension. In this embodiment, the 1*24-dimensional spectral feature obtained in step four is true or false labeled according to the true or false of the portrait where the feature is located, and finally a 1*2 label (for example: real person 01, fake person 10) is obtained.

[0054] Step six, the steps three to five are repeated for other multi-spectral images converted in dimension in the multi-spectral images of real people and the multi-spectral images of the simulation head model, and n groups of spectral feature data and n groups of labels of the multi-spectral images of real people and the multi-spectral images of the simulation head model converted in dimension are obtained.

[0055] Step seven, the neural network basic framework is designed based on the n groups of spectral feature data and the n groups of labels. In this embodiment, a lightweight BP neural network is used as the basis, and the network layer is finally set as follows through experimental verification: the input layer is adjustable according to the input data dimension (such as: 24 neurons), the output layer has 2 neurons (real or fake face label), the middle hidden layer has three layers: the first hidden layer, the second hidden layer and the third hidden layer, the number of neurons in the first hidden layer is 36, the number of neurons in the second hidden layer is 48, and the number of neurons in the third hidden layer is 36.

[0056] Step eight, according to the n sets of spectral feature data and n sets of labels collected in step six as training data, using the neural network infrastructure in step seven as a training model, selecting effective spectral dimensions from the n sets of spectral feature data to form the final training data. In this embodiment, the n*24 input is divided into 24 types of n*1 data, and the corresponding label output is 24 types of n*2 output data, thereby forming the preprocessed feature data for dimension screening. According to the preprocessed input feature of dimension screening, the feature dimension of each type of n*1 data is 24, and n is the number of data groups and 1 is the feature dimension of the data, thereby defining the input layer neuron dimension as 1, and the number of other neurons (hidden layer, output layer) remains unchanged. Effective spectral dimensions are selected from the preprocessed spectral data. Because the data is divided into 24 types of n*1 data, it is trained 24 times. The m spectral feature dimensions with classification accuracy >=70% in the 24 training results are retained (for example, if 7 of the 24 training results have a classification accuracy >70%, the 7 spectra are considered to be effective spectral dimensions for real and fake face recognition), wherein the accuracy of 70% can be determined according to the performance requirements of the task. According to the number of effective spectral dimensions m obtained from the 24 training results, then from the n*24 preprocessed data obtained in step five, select the corresponding m-dimensional data from the 24-dimensional data as the final experimental training data, that is, the training data is changed from 24-dimensional preprocessed data to effective m-dimensional data, and the final training data is n*m, wherein n is the number of data groups and m is the effective spectral dimension.

[0057] Step nine, according to the final training data in step eight, adjust the input layer of the neural network infrastructure in step seven to determine the final neural network structure. In this embodiment, according to the input data dimension 7 obtained in step eight, the neural network infrastructure is adjusted, and the final neural network infrastructure is adjusted to have an input layer of 7 dimensions, an output layer of 2 dimensions, and three hidden layers in the middle with 36, 48, and 36 neurons respectively.

[0058] Step ten, use the final training data to train the final neural network structure, and save it as the final face liveness detection model after training.

[0059] Step eleven, complete multispectral face liveness detection according to the final face liveness detection model.

[0060] In summary, the present application is aimed at the complex illumination conditions of the traditional imaging system, the complex condition requirements of the living face, and the low recognition rate. A method for living face recognition using multispectral images as acquisition data and using neural network method is proposed. The method combines multispectral imaging and spectral detection, which can not only analyze the internal component content of the object, but also visualize the distribution. Therefore, using multispectral imaging technology as the information acquisition means of living face recognition can enrich the diversity of data information and improve the accuracy of living face recognition. However, the multi-dimensional spectral data not only provides more information, but also brings higher difficulty in feature extraction and larger calculation amount. The method selects effective spectral dimensions from n groups of spectral feature data to form the final training data, which greatly reduces the data amount, reduces the labor cost of data acquisition, and improves the calculation speed. In addition, the method uses multispectral images as acquisition data and uses neural network method to quickly and efficiently recognize living face. This method does not require high hardware equipment requirements, reduces project costs, and has high accuracy. Therefore, compared with the prior art, the multispectral face living detection method provided by the present application uses multispectral images instead of rgb images for data acquisition. The data collected in the form of multi-channel dimension vector instead of the spatial dimension of rgb image can reduce the complex spatial dependence of rgb image. The advantages of multi-channel dimension information of multispectral image make the recognition accuracy more obvious. After single dimension training and testing, the part with higher recognition rate is selected to form new training and testing samples, which can eliminate redundant features, simplify data amount, and improve precision. It does not require a large amount of data, reduces the labor cost of data acquisition, does not require high hardware equipment requirements, reduces project costs, has a speed of about 90ms, and has an accuracy of about 95%.

[0061] For the purposes of this description, spatially relative terms, such as "above", "below", "up", "down", "between", "within", "lower", "upper", and the like, can be used herein for ease of description to describe one element's or feature's relationship to another element(s) or feature(s) as illustrated in the figures. It will be understood that the spatially relative terms are intended to encompass different orientations of the device in use or operation in addition to the orientations depicted in the figures. For example, if a device is inverted or rotated by 90 degrees, then an element described as "above" or "up" another element or feature would now be oriented "below" or "down" the other element or feature. Thus, the example term "above" can encompass both an orientation of above and below. The device can be otherwise oriented (rotated 90 degrees or at other orientations) and the spatially relative descriptors used herein interpreted accordingly. It will be understood that the spatially relative terms are intended to encompass different orientations of the device in use or operation in addition to the orientations depicted in the figures. For example, if a device is inverted or rotated by 90 degrees, then an element described as "above" or "up" another element or feature would now be oriented "below" or "down" the other element or feature. Thus, the example term "above" can encompass both an orientation of above and below. The device can be otherwise oriented (rotated 90 degrees or at other orientations) and the spatially relative descriptors used herein interpreted accordingly.

[0062] In addition, it should be noted that the use of "first", "second", and the like words to qualify parts is merely for the convenience of distinguishing the corresponding parts, and the above words have no special meaning unless otherwise stated, and therefore cannot be understood as limiting the scope of protection of the present application.

[0063] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. Various modifications and changes can be made by those skilled in the art based on the spirit and principles of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A multispectral face liveness detection method based on neural networks, characterized in that, The neural network-based multispectral face liveness detection method includes: Step 1: Collect spectral data from both the real person and the simulated head model to obtain multispectral images of the real person and the simulated head model respectively; Step 2: Perform dimensionality transformation on the acquired real-person multispectral images and simulated head model multispectral images respectively; Step 3: For any dimension-converted multispectral image of the real person multispectral image and the simulated head model multispectral image, extract spectral features from the pixels of the target region in the dimension-converted multispectral image. Step 4: For the multispectral image after transformation in any dimension, perform data correction on the extracted spectral features to obtain the spectral feature data of the multispectral image after transformation in any dimension. Step 5: For the multispectral image after transformation in any dimension, classify the corrected data and obtain the label of the spectral feature data; Step six: For the multispectral images of the real person multispectral image and the simulated head model multispectral image after other dimension transformations, repeat steps three to five to obtain n sets of spectral feature data and n sets of labels for the multispectral images of the real person multispectral image and the simulated head model multispectral image after dimension transformation; Step 7: Design the neural network infrastructure based on the n sets of spectral feature data and n sets of labels; Step 8: Using the n sets of spectral feature data and n sets of labels collected in Step 6 as training data, and the neural network infrastructure in Step 7 as the training model, select effective spectral dimensions from the n sets of spectral feature data to form the final training data. Step 9: Based on the final training data from Step 8, adjust the input layer of the neural network infrastructure from Step 7 to determine the final neural network structure. Step 10: Use the final training data to train the final neural network structure, and save it as the final face liveness detection model after training. Step 11: Perform multispectral face liveness detection based on the final face liveness detection model.

2. The multispectral face liveness detection method based on neural networks according to claim 1, characterized in that, In step three, the spectral feature extraction of the target region pixels in the dimensionally transformed multispectral image specifically includes: locating the face region in the dimensionally transformed multispectral image, randomly selecting a region block of a set size in the face region, averaging the region block of the set size in the planar dimension to obtain spectral information within the set region range, and completing the spectral feature extraction of the target region pixels.

3. The multispectral face liveness detection method based on neural networks according to claim 2, characterized in that, In step four, data correction of the extracted spectral features specifically includes: performing matrix multiplication of the spectral information within a set region with a set matrix to obtain spectral feature data of the set region points.

4. The multispectral face liveness detection method based on neural networks according to claim 3, characterized in that, In step five, for the multispectral image after transformation of any dimension, the spectral feature data obtained in step four is labeled as real or fake based on the authenticity of the human image containing the feature, and the label of the spectral feature data is obtained.

5. The multispectral face liveness detection method based on neural networks according to claim 4, characterized in that, In step seven, the neural network infrastructure is based on a lightweight backpropagation (BP) neural network, which includes an input layer, a hidden layer, and an output layer.

6. The multispectral face liveness detection method based on neural networks according to claim 5, characterized in that, The hidden layer includes a first hidden layer, a second hidden layer, and a third hidden layer. The first hidden layer has 36 neurons, the second hidden layer has 48 neurons, and the third hidden layer has 36 neurons.

7. The multispectral face liveness detection method based on neural networks according to claim 6, characterized in that, In step eight, the neural network infrastructure from step seven is used as the training model to train the data of each spectral feature dimension in the n sets of spectral feature data collected in step six. It is determined whether the classification accuracy of any spectral feature dimension data is greater than or equal to the set classification accuracy. The spectral feature dimensions that are greater than or equal to the set classification accuracy are taken as valid spectral dimensions. All valid spectral dimension data and their corresponding labels are used to form the final training data.

8. The multispectral face liveness detection method based on neural networks according to claim 7, characterized in that, In step one, the Ximea multispectral image acquisition device is used to acquire multispectral image data of the real human figure and the simulated head model respectively, and the real human multispectral image and the simulated head model multispectral image are acquired and stored in Bayer format.

9. The multispectral face liveness detection method based on neural networks according to claim 8, characterized in that, In step two, the Bayer format real-person multispectral images and simulated head model multispectral images are converted into an image format of (h / 5)*(w / 5)*25 according to the spectral bands.

10. A multispectral face liveness detection system based on neural networks, characterized in that, The neural network-based multispectral face liveness detection system uses the neural network-based multispectral face liveness detection method as described in any one of claims 1 to 9 to perform multispectral face liveness detection.

Images