A biometric-based intelligent signboard verification management method

Abstract

The application relates to the technical field of biometric feature recognition, and discloses a smart signboard verification management method based on biometric features, which comprises the following steps: establishing a living body recognition model based on historical living body feature data; combining the acquired living body feature data of a current user and the living body recognition model to generate a living body recognition result H3 of the current user; establishing a biometric feature information base based on the fingerprint feature information and the face feature information of all users; searching the biometric feature information base based on the acquired biometric feature information of the current user to generate an identity recognition result F of the current user; and combining the identity recognition result F and the living body recognition result H3 of the current user to generate a signboard verification result. The application can accurately recognize living bodies according to the comprehensive features of facial movements and illumination, and improves the adaptability of the model in various practical application scenarios.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of biometric identification, in particular to a smart sign verification management method based on biometric features. BACKGROUND

[0002] With the development of the Internet of Things and smart home fields, biometric identification technology has been widely applied in various devices and systems, such as smart door locks, surveillance cameras, access control systems, etc. The demand of these application scenarios has driven the continuous innovation and development of biometric identification technology.

[0003] In the smart sign verification management method, the application of biometric identification technology can improve the accuracy and security of sign verification. In biometric identification, live detection is a key issue. Attackers may use fake biometric features (such as photos, videos or 3D models) to deceive the identification system. Therefore, how to accurately detect the live state of biometric features is a challenge. SUMMARY

[0004] The purpose of the present application is to prevent attacks by fake biometric features through the combination of a live identification model and a biometric feature database, the monitoring of temperature, pulse, capacitance, etc. and the judgment of facial movements and lighting; a fast and accurate identity verification process reduces user waiting time and improves user experience.

[0005] To achieve the above purpose, the present application provides a smart sign verification management method based on biometric features, comprising:

[0006] establishing a live identification model based on historical live feature data;

[0007] combining the acquired live feature data of the current user and the live identification model to generate a live identification result H3 of the current user;

[0008] entering the fingerprint feature information and the facial feature information of all users to establish a biometric feature database; based on the acquired fingerprint feature information and the facial feature information of the current user, searching the biometric feature database to generate an identity recognition result F of the current user;

[0009] combining the identity recognition result F and the live identification result H3 of the current user to generate a sign verification result;

[0010] The historical live feature data includes historical first live feature data and historical second live feature data.

[0011] In some embodiments of the present application, when generating the live identification result H3 of the current user, it includes:

[0012] The historical first living body feature data includes historical temperature data, historical pulse data and historical capacitance data.

[0013] The fingerprint living body recognition model is established based on the first living body feature data, and a first-level living body recognition result H1 is generated by combining the fingerprint living body recognition model and the acquired real-time first living body feature data.

[0014] The historical second living body feature data includes historical face illumination data and historical face action data.

[0015] The face living body recognition model is established based on the second living body feature data, and a second-level living body recognition result H2 is generated by combining the established face living body recognition model and the acquired real-time second living body feature data.

[0016] The living body recognition result H3 of the current user is generated based on the first-level living body recognition result H1 and the second-level living body recognition result H2.

[0017] When H1=1 and H2=1, H3=1; when H1≠1 or H2≠1, H3=0.

[0018] In some embodiments of the present application, when the fingerprint living body recognition model is established, the following steps are included.

[0019] The historical temperature data, the historical pulse data and the historical capacitance data.

[0020] The first living body temperature interval (a1, a2) is generated based on the historical temperature data and a preset temperature interval.

[0021] The temperature interval correction coefficient b is generated based on the current environment temperature, and the second living body temperature interval (a1*(1-b), a2*(1+b)) is generated by combining the temperature interval correction coefficient and the first living body temperature interval.

[0022] The pulse monitoring interval (c1, c2) is set based on the historical pulse data.

[0023] The capacitance monitoring interval (d1, d2) is set based on the historical capacitance data.

[0024] Wherein, a1 represents a first preset value of living body temperature, a2 represents a second preset value of living body temperature, c1 represents a first preset value of pulse monitoring, c2 represents a second preset value of pulse monitoring, d1 represents a first preset value of capacitance monitoring, and d2 represents a second preset value of capacitance monitoring.

[0025] The fingerprint living body recognition model is generated by combining the second living body temperature interval (a1*(1-b), a2*(1+b)), the pulse monitoring interval (c1, c2) and the capacitance monitoring interval (d1, d2).

[0026] In some embodiments of the present application, when the first-level living body recognition result H1 is generated, the method comprises the following steps:

[0027] A temperature sensor, a light-emitting diode, a photodetector, and an alternating current electric field are arranged on the fingerprint sensor.

[0028] Real-time temperature data ai of the user is generated based on the temperature sensor;

[0029] Real-time pulse data ci of the user is generated based on the light-emitting diode and the photodetector;

[0030] Real-time capacitance data di of the user is generated based on the alternating current electric field to monitor the electric field change data when the user uses the fingerprint sensor;

[0031] Real-time first living body feature data is generated, which comprises the real-time temperature data ai of the user, the real-time pulse data ci of the user, and the real-time capacitance data di of the user;

[0032] The first-level living body recognition result H1 is generated based on the real-time first living body feature data;

[0033] If ai∈(a1*(1-b), a2*(1+b)) and (ci∈(c1, c2)) and (di∈(d1, d2), then H1=1;

[0034] If ai∈(1-(a1*(1-b), a2*(1+b)) or ci∈1-(c1, c2) or di∈1-(d1, d2), then H1=0.

[0035] In some embodiments of the present application, when the real-time pulse data ci of the user is generated based on the light-emitting diode and the photodetector, the method comprises the following steps:

[0036] The light emitted by the light-emitting diode and penetrating the finger is monitored by the photodetector;

[0037] The periodic change of the light intensity of the light with the pulse beat is judged to generate the real-time pulse data ci of the user.

[0038] In some embodiments of the present application, when the face living body recognition model is established based on the second living body feature data, the method comprises the following steps:

[0039] A face feature data set is extracted based on the historical face illumination data and the historical face motion data, and the face feature data set comprises state information of a plurality of data points;

[0040] The state information of the data points comprises position information of the data points and illumination information of the data points;

[0041] Obtain monitoring data of position information of a plurality of data points corresponding to historical facial action data, generate a facial action judgment model;

[0042] Obtain monitoring data of illumination information of a plurality of data points corresponding to historical facial illumination data, generate a facial illumination judgment model;

[0043] Establish a facial liveness recognition model based on the facial action judgment model and the facial illumination judgment model.

[0044] In some embodiments of the present application, when the facial illumination judgment model is generated, the following steps are included:

[0045] Conduct an illumination test based on the obtained historical facial illumination data;

[0046] Obtain the light reflection intensity of the test target facial data points under different color illuminations;

[0047] Calculate the variance of the light reflection intensity of all test target facial data points under the current color illumination, and generate a light reflection intensity consistency evaluation value based on the variance of the light reflection intensity of all facial data points;

[0048] Calculate the corresponding light reflection intensity consistency evaluation values under all color illuminations, and compare them with a preset value to generate a facial illumination judgment result;

[0049] Generate a facial illumination judgment model by integrating the illumination color type and the facial illumination judgment result.

[0050] In some embodiments of the present application, when the secondary liveness recognition result H2 is generated, the following steps are included:

[0051] Set the facial action in the facial recognition process, and obtain a set of position information E of the data points corresponding to the facial action performed by the current user, E={e1, e2…ei…en};

[0052] Wherein, ei represents the position information of the i-th data point, and n represents the total number of data points;

[0053] Obtain the position change data of the data points corresponding to the facial action performed by the current user by sampling the position information set E of the data points multiple times;

[0054] Generate a facial action liveness judgment result M1 in combination with the corresponding data point position change data and the facial action judgment model;

[0055] If the position change data of the data points corresponding to the facial action performed by the current user obtained by multiple sampling conforms to the position change data of the data points in the facial action judgment model, M1=1;

[0056] If the position change data of the data points corresponding to the facial actions performed by the current user obtained by multiple samplings does not match the position change data of the data points in the facial action judgment model, then M1 = 0;

[0057] The lighting system is configured to provide various color lighting variations during facial recognition;

[0058] Obtain real-time illumination information for all data points in the current user's facial image;

[0059] The facial lighting liveness detection result M2 is generated based on the lighting color and corresponding real-time lighting information;

[0060] If the consistency evaluation value of real-time illumination information is greater than the preset value, then M2 = 1;

[0061] If the consistency evaluation value of real-time illumination information is less than the preset value, then M2 = 0;

[0062] Based on the facial motion liveness detection result M1 and the facial illumination liveness detection result M2, a secondary liveness recognition result H2 is generated;

[0063] When M1 = 1 and M2 = 1, H2 = 1; when M1 ≠ 1 or M2 ≠ 1, H2 = 0.

[0064] In some embodiments of the present invention, the step of generating the sign verification result by combining the current user's identity recognition result F and the liveness recognition result H3 includes:

[0065] Based on the liveness detection result H3, determine whether to perform identity verification for the current user;

[0066] Repeatedly obtain the liveness detection result H3. If all H3 values ​​are 0, then do not perform the current user's identity verification.

[0067] If H3 = 1, then perform identity verification on the current user and generate the identity verification result F for the current user;

[0068] Generate sign verification results based on the current user's identity verification result F.

[0069] The biometric-based smart signage verification and management method provided in this invention has the following advantages compared to existing technologies:

[0070] By combining liveness detection models with biometric databases, the accuracy and security of identity verification are improved.

[0071] By monitoring data such as temperature, pulse, and capacitance in the liveness detection model, as well as judging facial movements and lighting, attacks that forge biometric features are effectively prevented.

[0072] By considering the influence of ambient temperature on fingerprint temperature, a temperature range correction coefficient b is generated to adjust the temperature range, enabling the model to adapt to different environmental conditions and reducing the interference of environmental factors on the recognition results.

[0073] Accurate liveness detection is achieved by using a facial liveness recognition model and combining facial movements and lighting features, which improves the model's adaptability to various practical application scenarios.

[0074] By obtaining the light reflection intensity of facial data points under different colored lights through light experiments, and calculating the variance to generate a consistency evaluation value of light reflection intensity, it is helpful to accurately identify facial features under different lighting conditions and reduce misidentification caused by changes in lighting.

[0075] By setting facial movements during the facial recognition process, multiple samplings are used to obtain data point position changes. This data is then combined with a facial movement judgment model to generate a facial movement liveness judgment result M1. Accurate analysis of the naturalness and coherence of facial movements helps to distinguish between real and fake facial movements, thus improving the accuracy of recognition. Attached Figure Description

[0076] Figure 1 This is a flowchart of a smart signage verification and management method based on biometrics provided in an embodiment of the present invention. Detailed Implementation

[0077] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0078] In the description of this invention, it should be understood that the terms "center", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0079] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0080] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0081] Example 1:

[0082] This invention provides a biometric-based smart signage verification and management method, such as... Figure 1 As shown, it includes:

[0083] A liveness detection model is established based on historical liveness feature data;

[0084] The liveness detection result H3 for the current user is generated by combining the acquired liveness feature data of the current user with the liveness detection model;

[0085] A biometric database is established by recording the fingerprint and facial features of all users; the biometric database is retrieved based on the fingerprint and facial features of the current user, and the identity recognition result F of the current user is generated.

[0086] The current user's identity verification result F and the liveness detection result H3 are combined to generate the sign verification result;

[0087] The historical liveness feature data includes: historical first liveness feature data and historical second liveness feature data.

[0088] Example 2:

[0089] When generating the current user's liveness detection result H3, the following steps are included:

[0090] Historical first-life characteristic data includes: historical temperature data, historical pulse data, and historical capacitance data;

[0091] A fingerprint liveness recognition model is established based on the first liveness feature data, and a first-level liveness recognition result H1 is generated by combining the fingerprint liveness recognition model and the acquired real-time first liveness feature data.

[0092] Historical second-generation liveness data includes: historical facial lighting data and historical facial movement data;

[0093] A facial liveness recognition model is established based on the second liveness feature data. The established facial liveness recognition model and the acquired real-time second liveness feature data are combined to generate a second-level liveness recognition result H2.

[0094] Generate the current user's liveness detection result H3 based on the first-level liveness detection result H1 and the second-level liveness detection result H2;

[0095] When H1 = 1 and H2 = 1, H3 = 1; when H1 ≠ 1 or H2 ≠ 1, H3 = 0.

[0096] Example 3:

[0097] The process of establishing the fingerprint liveness detection model includes:

[0098] Historical temperature data, historical pulse data, historical capacitance data;

[0099] The first living body temperature range (a1, a2) is generated based on a preset temperature range using historical temperature data.

[0100] Based on the current ambient temperature, a temperature range correction coefficient b is generated. Then, the second living temperature range (a1*(1-b), a2*(1+b)) is generated by combining the temperature range correction coefficient and the first living temperature range.

[0101] The pulse monitoring interval (c1, c2) is set based on historical pulse data;

[0102] The capacitance monitoring interval (d1, d2) is set based on historical capacitance data;

[0103] Where a1 represents the first preset value of living body temperature, a2 represents the second preset value of living body temperature, c1 represents the first preset value of pulse monitoring, c2 represents the second preset value of pulse monitoring, d1 represents the first preset value of capacitance monitoring, and d2 represents the second preset value of capacitance monitoring.

[0104] A fingerprint liveness recognition model is generated by combining the second liveness temperature range (a1*(1-b), a2*(1+b)), the pulse monitoring range (c1, c2), and the capacitance monitoring range (d1, d2).

[0105] In this embodiment, a large amount of historical temperature data is collected, which should come from fingerprint collection processes of different individuals under different environmental conditions. Statistical analysis is performed on the historical temperature data, such as calculating statistical quantities like mean, variance, and median.

[0106] Considering the differences in body temperature among different groups (such as different ages, genders, and health conditions) and the influence of environmental factors (such as seasons and regions) on body temperature, a reasonable temperature range is determined. For example, normal human body temperature is generally between 36 and 37.5 degrees Celsius, but during fingerprint collection, due to factors such as skin-to-device contact, there may be some fluctuation.

[0107] Based on the above analysis, a first in vivo temperature range (a1, a2) is defined. The values ​​of a1 and a2 should cover the normal temperature range of a living organism while excluding abnormal temperature conditions. For example, a1 can be set to 35.5 degrees Celsius and a2 to 38 degrees Celsius. This range takes into account fluctuations in normal body temperature as well as possible increases in body temperature due to slight exercise or environmental factors.

[0108] Generating the temperature range correction factor b: Accurate measurement of the current ambient temperature can be achieved using a high-precision temperature sensor. The temperature range correction factor b is determined by establishing a function based on the relationship between ambient temperature and normal human body temperature.

[0109] For example, if the current ambient temperature is low, the body surface temperature may decrease accordingly. In this case, b is a positive value, used to appropriately extend the lower limit of the first living body temperature range. If the current ambient temperature is high, the body surface temperature may increase, and b is also a positive value, used to appropriately extend the upper limit of the first living body temperature range. If the ambient temperature is within the normal comfort range, b can be set to a value close to 0. The specific functional relationship can be determined through extensive experimental data and statistical analysis, such as b = k × (T - T0), where T is the current ambient temperature, T0 is the reference ambient temperature, and k is a coefficient determined experimentally.

[0110] Calculate the second living temperature range using the formula (a1*(1-b), a2*(1+b)). During the calculation, ensure accuracy, especially for decimal operations. For example, if a1 = 35.5 degrees Celsius and b = 0.05, then the lower limit of the second living temperature range is 35.5 × (1-0.05) = 33.725 degrees Celsius; if a2 = 38 degrees Celsius, the upper limit is 38 × (1+0.05) = 39.9 degrees Celsius.

[0111] Historical pulse data were collected from different individuals under conditions of rest, slight movement, and tension. This data was then categorized and processed to remove outliers (such as excessively high or low pulse values ​​due to measurement errors or specific physiological conditions).

[0112] Analyze the relationship between pulse data and factors such as age, gender, and physical condition. For example, young people's pulses are usually slightly faster than those of older people, and there may be differences in pulse rates between men and women in certain situations. Athletes' resting pulses may be lower than those of the average person.

[0113] Based on the above analysis, pulse monitoring intervals (c1, c2) are defined. For example, for a typical adult in a resting state, c1 can be set to 60 beats / minute, and c2 can be set to 100 beats / minute. This interval takes into account the fluctuation range of the pulse under normal physiological conditions, while also allowing for some margin for possible slight movements or emotional changes.

[0114] Example 4:

[0115] When generating the first-level liveness detection result H1, the following steps are included:

[0116] A temperature sensor, a light-emitting diode, a photodetector, and an alternating current field are incorporated into the fingerprint sensor.

[0117] Real-time temperature data for users is generated based on temperature sensors (ai).

[0118] Real-time pulse data (ci) of the user is generated based on light-emitting diodes and photodetectors;

[0119] Based on the AC electric field monitoring data of the user's electric field change when using the fingerprint sensor, the user's real-time capacitance data di is generated based on the electric field change data.

[0120] Generate real-time first liveness feature data, including: the user's real-time temperature data ai, the user's real-time pulse data ci, and the user's real-time capacitance data di;

[0121] A first-level liveness detection result H1 is generated based on the real-time first liveness feature data;

[0122] If ai∈(a1*(1-b), a2*(1+b))) and (ci∈(c1, c2)) and (di∈(d1, d2), then H1=1;

[0123] If ai∈(1-(a1*(1-b), a2*(1+b))) or ci∈1-(c1, c2) or di∈1-(d1, d2), then H1=0.

[0124] Example 5:

[0125] When generating the user's real-time pulse data ci based on light-emitting diodes and photodetectors, the following are included:

[0126] The light emitted by the light-emitting diode and passing through the finger is monitored using a photodetector;

[0127] The system determines the periodic changes in light intensity with pulse rate and generates the user's real-time pulse data (ci).

[0128] Example 6:

[0129] The process of establishing a facial liveness recognition model based on the second liveness feature data includes:

[0130] A facial feature dataset is extracted based on historical facial lighting data and historical facial motion data. The facial feature dataset includes state information of multiple data points.

[0131] The status information of the data points includes the location information and the illumination information of the data points;

[0132] A facial movement judgment model is generated by acquiring the location information of multiple data points corresponding to historical facial movement data.

[0133] A facial illumination judgment model is generated by acquiring monitoring data of illumination information from multiple data points corresponding to historical facial illumination data.

[0134] A facial liveness detection model is established based on a facial motion judgment model and a facial lighting judgment model.

[0135] In this embodiment, for each action (such as blinking, smiling, etc.) in the historical facial action data, the positional changes of multiple data points during the action process are analyzed in detail.

[0136] By continuously acquiring facial images or video frames, the location information of each data point at different time points is recorded to form location monitoring data. For example, for a simple blinking action, the sequence of coordinate position changes of data points at the corners of the eyes during the blinking process is recorded.

[0137] Statistical analysis is performed on the acquired positional information monitoring data from multiple actions. The range of positional change, velocity, acceleration, and other characteristics of each data point in different actions are calculated.

[0138] Based on these features, a facial movement judgment model can be constructed. For example, a normal range for the positional changes of data points in each movement can be defined. If the positional changes of data points in the monitored data fall within this range, the movement is judged as normal; otherwise, it is judged as abnormal. The model can take the form of a probabilistic model or a rule-based model, etc.

[0139] Based on historical facial lighting data, we analyzed the changes in lighting information at multiple data points under different lighting conditions (such as different colors of light, different intensities of light, etc.).

[0140] Record the monitoring data of illumination information (such as RGB values, brightness, etc.) for each data point under different lighting conditions. For example, as the intensity of red light gradually increases, record the changes in the RGB values ​​and brightness values ​​of the nose tip data point.

[0141] The acquired illumination information monitoring data from multiple data points were analyzed. The variation patterns of illumination information under different illumination conditions were calculated, including statistical characteristics such as variance, mean, and correlation.

[0142] A facial lighting assessment model can be constructed based on these statistical characteristics. For example, a range of lighting information variation for facial data points under certain lighting conditions can be defined under normal circumstances. If the lighting information variation in the monitored data falls within this range, it is judged as a normal lighting condition; otherwise, it is considered an abnormal lighting condition. The model can take the form of a threshold-based model or a machine learning model, etc.

[0143] Example 7:

[0144] The process of generating the facial lighting determination model includes:

[0145] Lighting experiments were conducted based on historical facial lighting data.

[0146] Acquire the light reflection intensity of data points on the face of the test target under different colored lighting conditions;

[0147] Calculate the variance of light reflection intensity of all facial data points of the test target under the current color illumination, and generate a consistency evaluation value of light reflection intensity based on the variance of light reflection intensity of all facial data points.

[0148] Calculate the consistency evaluation value of light reflection intensity under all colored lighting conditions, and compare it with the preset value to generate the facial lighting judgment result;

[0149] A facial lighting judgment model is generated by combining the lighting color type and the facial lighting judgment results.

[0150] In this embodiment, representative data samples are selected from a large amount of historical facial lighting data. These samples should cover facial data from different ethnic groups, genders, and age groups, as well as data under different ambient lighting conditions. For example, this includes facial lighting data under conditions such as low indoor light, high indoor light, and natural outdoor light.

[0151] The selected historical facial lighting data is preprocessed, such as by normalizing the images, to ensure that parameters such as brightness and contrast are within a suitable range, so as to accurately analyze the lighting effects later.

[0152] Based on the ambient lighting conditions from historical facial lighting data, these lighting conditions are simulated as accurately as possible in a laboratory environment. For example, using light sources with adjustable color and intensity, the same lighting angles and distances as in the historical data are set.

[0153] Under simulated lighting conditions, facial lighting data is collected from the test target (which may be a representative facial model or a real volunteer). During the collection process, it is necessary to ensure that the parameters of the collection device (such as a high-precision camera) are stable and that it can accurately capture the lighting information of each data point on the face.

[0154] First, the definition and selection criteria for facial data points are clarified. These data points can be key facial feature points, such as the corners of the eyes, the tip of the nose, and the corners of the mouth, or they can be the center coordinates of facial regions divided according to certain rules. A precise facial recognition algorithm is then used to locate these data points within the image.

[0155] For each data point, its light reflection intensity is obtained using a specialized light intensity measurement tool (either a light intensity sensor integrated with the acquisition device or indirectly calculated through image analysis algorithms) under different colored light illuminations. During the measurement process, the influence of the spectral characteristics of different colored light on the light reflection intensity measurement must be considered. For example, the skin's reflection and absorption characteristics of red light under red light are different from those under blue light.

[0156] The light reflection intensity data of each facial data point under different colored lighting conditions were recorded in detail. A data table was created, where rows represent different lighting colors, columns represent different facial data points, and the elements in the table are the corresponding light reflection intensity values. At the same time, relevant environmental parameters during data collection, such as temperature and humidity, were recorded for reference during subsequent data analysis.

[0157] The light reflection intensity data of facial data points under the current color lighting is calculated according to the variance calculation formula.

[0158] When calculating variance, it is important to pay attention to the accuracy of the data and the precision of the calculation. Since light reflectance data can be affected by various factors, such as errors in the acquisition equipment and subtle differences in facial skin, multiple measurements and statistical analyses are necessary to ensure the reliability of the variance calculation results.

[0159] A consistency evaluation value for light reflection intensity is generated based on the calculated variance. A mapping function can be established to map the variance value to a specific range of evaluation values. For example, a small variance value indicates relatively consistent light reflection intensity, and the evaluation value can be close to 1; a large variance value indicates significant differences in light reflection intensity, and the evaluation value can be close to 0. This mapping function can be determined based on a large amount of experimental data and experience, and should be adjusted according to the actual application scenario.

[0160] Following the method described above, the consistency evaluation value of light reflection intensity under all colored illuminations is calculated sequentially. For each colored illumination, the steps of variance calculation and evaluation value generation are repeated to obtain a set of consistency evaluation values ​​corresponding to different colored illuminations.

[0161] These evaluation values ​​can be organized and analyzed. For example, a bar chart can be drawn to visually display the distribution of evaluation values ​​under different colors of light, so as to better understand the light reflection characteristics of facial skin under different colors of light.

[0162] A reasonable preset value is determined, which can be obtained by analyzing a large amount of normal facial lighting data. The calculated consistency evaluation values ​​of light reflection intensity under all colored lighting are then compared with the preset value.

[0163] If all evaluation values ​​are greater than the preset value, it means that the facial lighting data is in line with the normal situation, and the facial lighting judgment result is passed, for example, it can be marked as 1; if there is at least one evaluation value that is less than or equal to the preset value, the facial lighting judgment result is failed, and it is marked as 0.

[0164] The facial lighting judgment model can be represented in a structured form, such as a matrix, where rows represent different lighting color types and columns represent the facial lighting judgment result (pass or fail) and related parameters (such as light reflection intensity variance, consistency evaluation value, etc.).

[0165] Alternatively, a rule-based representation can be used, which combines the lighting color type, facial lighting judgment results, and related calculation rules (such as variance calculation, evaluation value generation, preset value comparison, etc.) into a series of judgment rules.

[0166] After generating the facial lighting judgment model, it needs to be validated using additional validation data. This validation data should differ from the historical facial lighting data used to build the model. The accuracy and reliability of the model are evaluated using the validation data. If errors or deficiencies are found, the model should be adjusted and improved, such as adjusting preset values ​​or optimizing calculation methods, until the model can accurately judge facial lighting conditions.

[0167] Example 8:

[0168] When generating the secondary liveness detection result H2, the following steps are included:

[0169] Set facial actions during the facial recognition process and obtain the set of location information E of the data points corresponding to the facial actions performed by the current user, E = {e1, e2, ..., ei, ..., en};

[0170] Where ei represents the location information of the i-th data point, and n represents the total number of data points;

[0171] The location information set E of the data points is sampled multiple times to obtain the location change data of the data points corresponding to the facial actions performed by the current user;

[0172] The facial movement liveness detection result M1 is generated by combining the positional change data of the corresponding data points with the facial movement judgment model.

[0173] If the position change data of the data points corresponding to the facial actions performed by the current user obtained by multiple samplings match the position change data M1=1 of the data points in the facial action judgment model;

[0174] If the position change data of the data points corresponding to the facial actions performed by the current user obtained by multiple samplings does not match the position change data of the data points in the facial action judgment model, then M1 = 0;

[0175] The lighting system is configured to provide various color lighting variations during facial recognition;

[0176] Obtain real-time illumination information for all data points in the current user's facial image;

[0177] The facial lighting liveness detection result M2 is generated based on the lighting color and corresponding real-time lighting information;

[0178] If the consistency evaluation value of real-time illumination information is greater than the preset value, then M2 = 1;

[0179] If the consistency evaluation value of real-time illumination information is less than the preset value, then M2 = 0;

[0180] Based on the facial motion liveness detection result M1 and the facial illumination liveness detection result M2, a secondary liveness recognition result H2 is generated;

[0181] When M1 = 1 and M2 = 1, H2 = 1; when M1 ≠ 1 or M2 ≠ 1, H2 = 0.

[0182] In this embodiment, when setting facial actions during facial recognition, it is necessary to consider the ability to comprehensively detect the movement and coordination of facial muscles. For example, in addition to common actions such as blinking and opening the mouth, some combined actions can also be set, such as blinking before smiling. For each set facial action, a set of location information E = {e1, e2…ei…, en} of the data points corresponding to the current user performing the action is obtained through high-precision facial recognition technology (such as a deep learning-based facial landmark detection algorithm). These data points can be the coordinates of key facial feature points, such as the corners of the eyes, corners of the mouth, and the tip of the nose.

[0183] Accuracy of data point location information: To ensure the accuracy of data point location information, image preprocessing is required during acquisition, such as noise removal and image enhancement. Simultaneously, the coordinates of the data points are precisely quantized based on the resolution and accuracy of the facial recognition device.

[0184] Sampling strategy: The location information set E of the data points is sampled multiple times. The sampling frequency should be determined based on the duration and speed of the facial movements. For example, a higher sampling frequency (e.g., 10-20 times per second) can be used for rapid blinking movements, while a sampling frequency of 5-10 times per second can be used for relatively slow smiling movements.

[0185] Calculation of position change data: The position change data of the data points is calculated by comparing the positions of the data points sampled each time. For example, for the i-th data point, the difference in coordinates between adjacent sampling times is calculated as Δei = ei(t+1) - ei(t), where t represents the sampling time. The position change data of all data points are combined to form a complete set of position change data.

[0186] Construction of the Facial Movement Judgment Model: The facial movement judgment model is built based on a large amount of real facial movement data. During the construction process, data on the positional changes of data points when performing the same facial movements in different groups of people and under different environments are collected, statistically analyzed, and the normal positional change range and pattern of each facial movement are determined.

[0187] The positional change data of the data points corresponding to the facial actions performed by the current user, obtained from multiple samplings, is compared with the data in the facial action judgment model. If these data are within the normal range determined by the model, that is, they match the positional change data of the data points in the facial action judgment model, then M1 = 1; otherwise, M1 = 0.

[0188] Lighting system design: When setting up the lighting system to provide multiple color variations during facial recognition, it's crucial to select appropriate color combinations. For example, the three primary colors—red, green, and blue—and their mixtures can be chosen to comprehensively detect the reflectivity of facial skin to different colors of light. The light intensity must also be set appropriately, ensuring sufficient illumination for a clear facial image while avoiding excessive glare that could negatively impact image quality.

[0189] Acquiring real-time illumination information: For all data points in the current user's facial image, real-time illumination information is acquired via an image sensor (such as a camera). Illumination information can include the color component values ​​(such as R, G, and B values ​​in the RGB color space) and the illumination intensity value for each data point. To improve the accuracy of the illumination information, the acquired facial image also needs to be preprocessed, such as through color correction.

[0190] Consistency evaluation value calculation: Calculate the consistency evaluation value of real-time illumination information. For example, a reference model can be established based on the expected reflectivity of different facial regions to different colors of light. The acquired real-time illumination information is compared with the reference model, the deviation value of each data point is calculated, and then a weighted average is used to combine the deviation values ​​of all data points to obtain the consistency evaluation value.

[0191] Judgment result generation: If the consistency evaluation value of real-time illumination information is greater than the preset value, then M2 = 1; if the consistency evaluation value of real-time illumination information is less than or equal to the preset value, then M2 = 0.

[0192] When M1=1 and M2=1, it means that the current user's facial movements are in accordance with the normal pattern and the facial lighting information is also in accordance with expectations. At this time, H2=1, indicating that facial liveness detection is successful.

[0193] When M1≠1 or M2≠1, indicating an anomaly in facial movement or lighting information, H2=0, meaning facial liveness detection failed. In this case, it's important to record whether the problem lies in facial movement or lighting for further analysis or to prompt the user to re-perform the detection.

[0194] Example 9:

[0195] When generating the sign verification result by combining the current user's identity recognition result F and the liveness recognition result H3, the following is included:

[0196] Based on the liveness detection result H3, determine whether to perform identity verification for the current user;

[0197] Repeatedly obtain the liveness detection result H3. If all H3 values ​​are 0, then do not perform the current user's identity verification.

[0198] If H3 = 1, then perform identity verification on the current user and generate the identity verification result F for the current user;

[0199] Generate sign verification results based on the current user's identity verification result F.

[0200] In this embodiment, when the liveness detection result H3 is 0, the sign verification result fails regardless of the identity verification result F. Because the liveness detection failed, even if identity verification might be successful, it cannot be guaranteed that it was performed by a genuine user.

[0201] When H3 is 1 and the identity recognition result F indicates successful recognition, the sign verification result is successful, indicating that the current user has passed the liveness detection and the identity recognition is correct.

[0202] When H3 is 1 but the identity recognition result F indicates recognition failure, the sign verification result is failure, meaning that although the liveness detection passed, the identity does not match.

[0203] Finally, it should be noted that those skilled in the art can obviously make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims and their equivalents, this invention also intends to include these modifications and variations.

[0204] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention.

Claims

1. A smart signage verification and management method based on biometrics, characterized in that, include: A liveness detection model is established based on historical liveness feature data; The liveness detection result H3 for the current user is generated by combining the acquired liveness feature data of the current user with the liveness detection model; A biometric database is established by recording the fingerprint and facial features of all users; the biometric database is retrieved based on the fingerprint and facial features of the current user, and the identity recognition result F of the current user is generated. The current user's identity verification result F and the liveness detection result H3 are combined to generate the sign verification result; The historical liveness feature data includes: historical first liveness feature data and historical second liveness feature data; When generating the current user's liveness detection result H3, the following steps are included: Historical first-life characteristic data includes: historical temperature data, historical pulse data, and historical capacitance data; A fingerprint liveness recognition model is established based on the first liveness feature data, and a first-level liveness recognition result H1 is generated by combining the fingerprint liveness recognition model and the acquired real-time first liveness feature data. Historical second-generation liveness data includes: historical facial lighting data and historical facial movement data; A facial liveness recognition model is established based on the second liveness feature data. The established facial liveness recognition model and the acquired real-time second liveness feature data are combined to generate a second-level liveness recognition result H2. Generate the current user's liveness detection result H3 based on the first-level liveness detection result H1 and the second-level liveness detection result H2; When H1=1 and H2=1, H3=1; when H1≠1 or H2≠1, H3=0. When generating the first-level liveness detection result H1, the following steps are included: A temperature sensor, a light-emitting diode, a photodetector, and an alternating current field are incorporated into the fingerprint sensor. Real-time temperature data for users is generated based on temperature sensors (ai). Real-time pulse data (ci) of the user is generated based on light-emitting diodes and photodetectors; Based on the AC electric field monitoring data of the user's electric field change when using the fingerprint sensor, the user's real-time capacitance data di is generated based on the electric field change data. Generate real-time first liveness feature data, including: the user's real-time temperature data ai, the user's real-time pulse data ci, and the user's real-time capacitance data di; A first-level liveness detection result H1 is generated based on the real-time first liveness feature data; If ai∈(a1*(1-b), a2*(1+b)) and (ci∈(c1, c2)) and (di∈(d1, d2)), then H1=1; If ai∈(1-(a1*(1-b), a2*(1+b))) or ci∈1-(c1, c2) or di∈1-(d1, d2), then H1=0.

2. The biometric-based smart signage verification and management method as described in claim 1, characterized in that, The process of establishing the fingerprint liveness detection model includes: Historical temperature data, historical pulse data, historical capacitance data; The first living body temperature range (a1, a2) is generated based on a preset temperature range using historical temperature data. Based on the current ambient temperature, a temperature range correction coefficient b is generated. Then, the second living temperature range (a1*(1-b), a2*(1+b)) is generated by combining the temperature range correction coefficient and the first living temperature range. The pulse monitoring interval (c1, c2) is set based on historical pulse data; The capacitance monitoring interval (d1, d2) is set based on historical capacitance data; Where a1 represents the first preset value of living body temperature, a2 represents the second preset value of living body temperature, c1 represents the first preset value of pulse monitoring, c2 represents the second preset value of pulse monitoring, d1 represents the first preset value of capacitance monitoring, and d2 represents the second preset value of capacitance monitoring. A fingerprint liveness recognition model is generated by combining the second liveness temperature range (a1*(1-b), a2*(1+b)), the pulse monitoring range (c1, c2), and the capacitance monitoring range (d1, d2).

3. The biometric-based smart signage verification and management method as described in claim 1, characterized in that, When generating the user's real-time pulse data ci based on light-emitting diodes and photodetectors, the following are included: The light emitted by the light-emitting diode and passing through the finger is monitored using a photodetector; The system determines the periodic changes in light intensity with pulse rate and generates the user's real-time pulse data (ci).

4. The biometric-based smart signage verification and management method as described in claim 3, characterized in that, The process of establishing a facial liveness recognition model based on the second liveness feature data includes: A facial feature dataset is extracted based on historical facial lighting data and historical facial motion data. The facial feature dataset includes state information of multiple data points. The status information of the data points includes the location information and the illumination information of the data points; A facial movement judgment model is generated by acquiring the location information of multiple data points corresponding to historical facial movement data. A facial illumination judgment model is generated by acquiring monitoring data of illumination information from multiple data points corresponding to historical facial illumination data. A facial liveness detection model is established based on a facial motion judgment model and a facial lighting judgment model.

5. The biometric-based smart signage verification and management method as described in claim 4, characterized in that, The process of generating the facial lighting determination model includes: Lighting experiments were conducted based on historical facial lighting data. Acquire the light reflection intensity of data points on the face of the test target under different colored lighting conditions; Calculate the variance of light reflection intensity of all facial data points of the test target under the current color illumination, and generate a consistency evaluation value of light reflection intensity based on the variance of light reflection intensity of all facial data points. Calculate the consistency evaluation value of light reflection intensity under all colored lighting conditions, and compare it with the preset value to generate the facial lighting judgment result; A facial lighting judgment model is generated by combining the lighting color type and the facial lighting judgment results.

6. The biometric-based smart signage verification and management method as described in claim 5, characterized in that, When generating the secondary liveness detection result H2, the following steps are included: Set facial actions during the facial recognition process and obtain the set of location information E of the data points corresponding to the facial actions performed by the current user, E={e1, e2…ei…en}; Where ei represents the location information of the i-th data point, and n represents the total number of data points; The location information set E of the data points is sampled multiple times to obtain the location change data of the data points corresponding to the facial actions performed by the current user; By combining the positional change data of the corresponding data points, a facial motion liveness detection result M1 is generated based on the facial motion judgment model; If the position change data of the data points corresponding to the facial actions performed by the current user obtained by multiple samplings match the position change data M1=1 of the data points in the facial action judgment model; If the position change data of the data points corresponding to the facial actions performed by the current user obtained by multiple samplings does not match the position change data of the data points in the facial action judgment model, then M1=0; The lighting system is configured to provide various color lighting variations during facial recognition; Obtain real-time illumination information for all data points in the current user's facial image; The facial lighting liveness detection result M2 is generated based on the lighting color and corresponding real-time lighting information; If the consistency evaluation value of real-time illumination information is greater than the preset value, then M2=1; If the consistency evaluation value of real-time illumination information is less than the preset value, then M2=0; Based on the facial motion liveness detection result M1 and the facial illumination liveness detection result M2, a secondary liveness recognition result H2 is generated; When M1=1 and M2=1, H2=1; when M1≠1 or M2≠1, H2=0.

7. The biometric-based smart signage verification and management method as described in claim 6, characterized in that, When generating the sign verification result by combining the current user's identity recognition result F and the liveness recognition result H3, the following is included: Based on the liveness detection result H3, determine whether to perform identity verification for the current user; Repeatedly obtain the liveness detection result H3. If all H3 values ​​are 0, then do not perform the current user's identity verification. If H3=1 exists, then the current user is identified, and the identification result F of the current user is generated; Generate sign verification results based on the current user's identity verification result F.

Images