Multi-mode biological recognition safe unlocking method and system for smart phone

Abstract

The invention relates to the technical field of face recognition, in particular to a multi-mode biological recognition safe unlocking method and system for a smart phone. The method comprises the following steps: firstly, acquiring three-dimensional coordinate points of a face model; further screening to obtain non-consensus coordinate points; further obtaining the noise conformity of each non-consensus coordinate point, and obtaining a noise point based on the noise conformity; further performing movement correction on the noise points by means of distribution of non-noise points closest to the noise points, and obtaining a corrected face point cloud in combination with three-dimensional coordinates of the non-noise points; and finally, unlocking at least according to the corrected face point cloud. According to the method, the non-consensus coordinate points collected by the two cameras are screened out, the outlier features are analyzed by means of density clustering, so that the noise points are screened out, the corrected face point cloud is obtained after the noise points are moved and corrected, a more reliable basis is provided for constructing a face three-dimensional model, and the unlocking efficiency and accuracy are improved.

Description

Technical Field

[0001] The present invention relates to the field of face recognition technology, and in particular to a multi-modal biometric security unlocking method and system for a smart phone. Background Art

[0002] Multimodal biometric security unlocking refers to the integration of multiple biometric technologies, using the unique advantages of multiple biometric technologies to analyze and judge the characteristic values ​​of multiple biometric methods for identification and authentication. The existing common combination of fingerprint and face recognition can integrate multiple biometric modes, relieve the pressure of single-modal systems and provide higher accuracy.

[0003] However, the traditional method uses the ICP algorithm to minimize the least squares error between the corresponding three-dimensional coordinate points obtained by binocular vision as the final registration purpose. The final result only averages the error over the entire point cloud, which is not ideal for constructing a three-dimensional face model, resulting in reduced unlocking efficiency and accuracy. Summary of the invention

[0004] In order to solve the technical problem that the existing face recognition technology averages the error to the entire point cloud, the construction effect of the face three-dimensional model is not ideal, resulting in reduced unlocking efficiency and accuracy, the purpose of the present invention is to provide a multi-modal biometric security unlocking method and system for smart phones, and the technical solutions adopted are as follows:

[0005] A multimodal biometric security unlocking method for a smart phone, the method comprising:

[0006] Acquire the three-dimensional coordinate points of the face model through the binocular structured light system; select the three-dimensional coordinate points corresponding to any camera as the coordinate points to be analyzed;

[0007] Perform nearest neighbor matching on each coordinate point to be analyzed and the three-dimensional coordinate point corresponding to another camera, and calculate the spatial distance between each coordinate point to be analyzed and the corresponding matched three-dimensional coordinate point; filter the coordinate points to be analyzed based on the spatial distance and each distance ratio in the preset distance ratio sequence, and obtain non-consensus coordinate points according to the change in the number of the coordinate points to be analyzed filtered by the adjacent distance ratios and the change in the spatial distance;

[0008] Performing a first density clustering on the non-consensus coordinate points to obtain a first clustering cluster; selecting any of the non-consensus coordinate points as a target coordinate point; in the first clustering cluster of the target coordinate point, obtaining a radius sequence based on the maximum spatial distance between the non-consensus coordinate points, a preset radius change step and a preset number of spheres; in the first clustering cluster of the target coordinate point, taking the target coordinate point as the center of the sphere and each element in the radius sequence as the radius, obtaining the spherical range of the target coordinate point; performing a second density clustering on the non-consensus coordinate points within each spherical range of the target coordinate point to obtain a second clustering cluster and a target cluster where the target coordinate point is located; obtaining the noise conformity of the target coordinate point based on the distance characteristics of the target cluster and other second clusters within all the spherical ranges, combined with the number of elements in the target cluster; obtaining a noise point based on the noise conformity;

[0009] Obtain a preset number of non-noise points closest to each of the noise points to fit a plane; obtain a corrected movement analysis range for each of the noise points based on a vertical distance between each of the noise points and the corresponding fitting plane, combined with the overall characteristics of the noise conformity of the non-noise points obtained; within the corrected movement analysis range for each of the noise points, take the intersection of the vertical lines of the fitting plane as the starting point, move the intersection of the vertical lines with a preset moving step length to obtain a noise movement analysis point for each of the noise points; obtain a corrected face point cloud based on the noise conformity of all of the noise movement analysis points of each of the noise points, combined with the three-dimensional coordinates of the non-noise points;

[0010] Unlocking is performed at least according to the corrected face point cloud.

[0011] Furthermore, the method for obtaining the non-consensus coordinate points includes:

[0012] The ratio of the spatial distance corresponding to the coordinate point to be analyzed to the maximum spatial distance is used as the normalized distance corresponding to the coordinate point to be analyzed; when the normalized distance of the coordinate point to be analyzed is less than or equal to the distance ratio, it is determined that the screening corresponding to the distance ratio is passed;

[0013] The area formed by the coordinate points to be analyzed that are screened by each distance ratio is used as the initial consensus area of ​​each distance ratio; the product of the absolute value of the difference between the number of coordinate points to be analyzed that are screened by adjacent distance ratios and the absolute value of the difference between the average values ​​of the spatial distances of the coordinate points to be analyzed is used as the consensus accuracy of the smallest distance ratio among the adjacent distance ratios;

[0014] The initial consensus region corresponding to the maximum consensus accuracy is selected as the final consensus region; and the three-dimensional coordinate points outside the final consensus region are used as non-consensus coordinate points.

[0015] Furthermore, the radius sequence acquisition method includes:

[0016] In the first clustering cluster of the target coordinate points, the maximum spatial distance between the non-consensus coordinate points is taken as the maximum radius; based on the maximum radius, the maximum radius is iteratively reduced with a preset radius change step size, and a new radius is obtained in each iteration, and a preset number of spheres are obtained in total. The maximum radius and the iteratively obtained radii are constructed to obtain a radius sequence.

[0017] Furthermore, the method for obtaining the noise conformity includes:

[0018] Select any of the spherical ranges of the target coordinate point as the target spherical range; within the target spherical range, use the number of elements in the target cluster as the first denominator, use the average value of the distances between the center point of the target cluster and the center points of other secondary clusters as the first numerator, and use the ratio of the first denominator to the first numerator as the noise sub-parameter of the target spherical range;

[0019] The average value of the noise sub-parameters of all the spherical ranges of the target coordinate point is taken as the noise conformity of the target coordinate point.

[0020] Furthermore, the method for acquiring noise points includes:

[0021] The non-consensus coordinate points whose noise conformity is greater than a preset conformity threshold are marked as noise points.

[0022] Furthermore, the method for obtaining the modified movement analysis range includes:

[0023] The product of the normalized value of the sum of the noise conformity values ​​of the non-noise points obtained by each noise point and the vertical distance of each noise point is used as the modified analysis range radius of each noise point;

[0024] On the perpendicular line between each noise point and the fitting plane, with the intersection of the perpendicular lines as the center, two line segments with a length equal to the radius of the corrected analysis range are intercepted on the perpendicular line as the corrected moving analysis range of each noise point.

[0025] Furthermore, the method for obtaining a corrected face point cloud according to the noise conformity of the noise motion analysis point includes:

[0026] Each of the noise points is moved to the corresponding noise movement analysis point where the noise conformity is the smallest, and the three-dimensional coordinates of the non-noise points are combined to obtain a corrected face point cloud.

[0027] Furthermore, the method for unlocking at least according to the corrected face point cloud includes:

[0028] Acquire a face matching degree according to the pre-stored face point cloud in the smart phone and the corrected face point cloud; unlock the phone when the face matching degree is greater than a preset face matching threshold;

[0029] When the face matching degree is less than or equal to the preset face matching threshold, a fingerprint unlocking prompt is given to obtain the fingerprint matching degree after the prompt; the sum of the noise conformity degrees of all noise points in the corrected face point cloud and the product of the fingerprint matching degree and the face matching degree are normalized as the unlocking suitability; when the unlocking suitability is greater than the preset unlocking suitability threshold, the phone is unlocked; when the unlocking suitability is less than or equal to the preset unlocking suitability threshold, it is determined that the recognition has failed, and the fingerprint unlocking or face unlocking prompt is given again.

[0030] Furthermore, the binocular structured light system at least includes a left camera, a right camera and a projector.

[0031] The present invention also proposes a multimodal biometric security unlocking system for a smartphone, the system comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor implementing any one of the steps of the multimodal biometric security unlocking method for a smartphone when executing the computer program.

[0032] The present invention has the following beneficial effects:

[0033] The present invention firstly obtains the three-dimensional coordinate points of the face model through a binocular structured light system to provide an analysis basis; further, according to the change in the number of coordinate points to be analyzed and the change in spatial distance screened according to the adjacent distance ratio, the change in the number of coordinate points to be analyzed is observed by continuously changing the screened distance range, and the density change is analyzed, so as to screen and obtain non-consensus coordinate points that are prone to noise when collected by only a single camera; further, the first clustering cluster is obtained by performing the first density clustering on the non-consensus coordinate points, so as to facilitate the analysis of the local distribution characteristics of the non-consensus coordinate points; further, the characteristic that noise points will maintain a high degree of outlier under density clustering in various ranges is utilized to obtain multiple spherical ranges of the target coordinate points; the non-consensus coordinate points within each spherical range of the target coordinate points are clustered. The punctuation points are clustered for secondary density to obtain secondary clustering clusters, the distance characteristics between the target clustering cluster where the target coordinate point is located and other secondary clustering clusters are analyzed, and the noise conformity of the target coordinate point is obtained in combination with the number of elements in the target clustering cluster, the conformity of the target coordinate point as a noise point is characterized, and the noise point is obtained based on the noise conformity; the local geometric distribution of the non-noise point is further represented by the fitting plane, the overall characteristics of the noise conformity of the non-noise point in the local vicinity of the noise point are analyzed, the corrected movement analysis range of each noise point is obtained, the noise point is corrected for movement, and the three-dimensional coordinates of the non-noise point are combined to obtain the corrected face point cloud, which provides a more reliable basis for constructing a three-dimensional face model; finally, unlocking is performed at least according to the corrected face point cloud. The present invention screens out the non-consensus coordinate points collected by two cameras, screens out the noise points by using the density clustering analysis outlier characteristics, and obtains the corrected face point cloud after the noise point is corrected for movement, which provides a more reliable basis for constructing a three-dimensional face model and improves the unlocking efficiency and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0035] Figure 1 A flowchart of a multi-modal biometric security unlocking method for a smart phone provided by an embodiment of the present invention;

[0036] Figure 2 A schematic diagram of a binocular structured light system according to an embodiment of the present invention;

[0037] Figure 3 A schematic diagram of noise movement analysis point distribution provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0038] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of the multimodal biometric security unlocking method and system for a smart phone proposed by the present invention, its specific implementation, structure, features and effects, in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0039] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0040] The specific scheme of a multimodal biometric security unlocking method and system for a smart phone provided by the present invention is described in detail below with reference to the accompanying drawings.

[0041] See also Figure 1 , which shows a flowchart of a multi-modal biometric security unlocking method for a smart phone provided by an embodiment of the present invention, specifically comprising:

[0042] Step S1: Acquire the three-dimensional coordinate points of the face model through the binocular structured light system; select the three-dimensional coordinate points corresponding to any camera as the coordinate points to be analyzed.

[0043] The embodiment of the present invention is aimed at a smartphone equipped with a face recognition system of a binocular structured light system and a fingerprint recognition system, and performs biometric unlocking on the smartphone. First, the three-dimensional coordinate points of the face model are obtained through the binocular structured light system, and a corrected face point cloud is obtained after analysis. Face recognition judgment is performed based on the corrected face point cloud. When the face recognition judgment fails, a fingerprint unlocking prompt is given and fingerprint data is collected to obtain the fingerprint matching degree, and the face recognition system is used to jointly determine whether to unlock the smartphone.

[0044] First, the binocular structured light system obtains the three-dimensional coordinate points of the face model to obtain the basis for analysis data; it should be noted that the binocular structured light system includes at least a left camera, a right camera and a projector, and the working principles of each component are already existing technologies and will not be repeated here.

[0045] Considering that the left and right cameras have a common capture area, please refer to Figure 2 , which shows a working schematic diagram of a binocular structured light system provided by an embodiment of the present invention, including a left camera, a right camera and a projector, wherein two rays starting from the camera lens are the capture area of ​​the camera, Figure 2 The bottom middle curve is a face schematic curve, the area between AC is the capture area of ​​the left camera, the area between BD is the capture area of ​​the right camera, and the area between BC is the common capture area.

[0046] When two cameras can recognize a certain area from different perspectives, it means that the distribution of three-dimensional coordinate points in a certain area is more consistent with the actual face situation. When the three-dimensional face model is reconstructed with monocular structured light, there will often be holes or noise in the obtained face point cloud due to the blockage of light by the body position or the reflection of light by the accessories worn. Therefore, the three-dimensional coordinate points need to be screened. Therefore, the three-dimensional coordinate points corresponding to any camera are selected as the coordinate points to be analyzed, so as to perform comparative analysis in the subsequent steps and screen out non-consensus coordinate points in non-commonly recognized areas.

[0047] In one embodiment of the present invention, the three-dimensional coordinate points corresponding to the left camera are selected as the coordinate points to be analyzed, and the three-dimensional coordinate points corresponding to the right camera are used as references, and the two are compared to obtain non-consensus coordinate points. In other embodiments of the present invention, the implementer can also set the three-dimensional coordinate points of the right camera to be selected as the coordinate points to be analyzed.

[0048] Step S2: Perform nearest neighbor matching on each coordinate point to be analyzed and the three-dimensional coordinate point corresponding to another camera, and calculate the spatial distance between each coordinate point to be analyzed and the corresponding matching three-dimensional coordinate point; filter the coordinate points to be analyzed based on the spatial distance and each distance ratio in the preset distance ratio sequence, and obtain non-consensus coordinate points according to the change in the number and spatial distance of the coordinate points to be analyzed filtered by the adjacent distance ratio.

[0049] Considering that the closer the coordinate point to be analyzed is to the three-dimensional coordinate point corresponding to the other camera, the more likely it is that the coordinate point to be analyzed is in the face area that can be captured by both cameras, so each coordinate point to be analyzed is matched with the three-dimensional coordinate point corresponding to the other camera by the nearest neighbor, and the spatial distance between each coordinate point to be analyzed and the corresponding matching three-dimensional coordinate point is calculated to facilitate subsequent analysis.

[0050] It should be noted that, in one embodiment of the present invention, the Euclidean distance between three-dimensional coordinate points is obtained as the spatial distance, and each coordinate point to be analyzed collected by the left camera is matched with the nearest three-dimensional coordinate point collected by the right camera to obtain the nearest neighbor matching result. In other embodiments of the present invention, the implementer may also obtain other distances such as the Manhattan distance as the spatial distance, which are all technical means well known to those skilled in the art and will not be described in detail.

[0051] Considering that the three-dimensional coordinate points of the face area captured by two cameras are denser than those of the face area captured by a single camera, the boundary between the common capture area and the non-common identification area can be analyzed through the density change of the data points to be analyzed. Therefore, the coordinate points to be analyzed can be screened based on the spatial distance and each distance ratio in the preset distance ratio sequence. The number of coordinate points to be analyzed screened according to the adjacent distance ratios changes and the spatial distance changes, and the density changes are analyzed by continuously changing the screening distance range, observing the number changes of the coordinate points to be analyzed, and obtaining non-consensus coordinate points.

[0052] Preferably, in one embodiment of the present invention, considering that when the spatial distance of the coordinate points to be analyzed is large, the possibility of a common capture area is low, so the preset distance ratio sequence is set to [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8], and the ratio of the spatial distance corresponding to the coordinate point to be analyzed to the maximum spatial distance is used as the normalized distance of the corresponding coordinate point to be analyzed; when the normalized distance of the coordinate point to be analyzed is less than or equal to the distance ratio, it is determined that the corresponding distance ratio screening is passed;

[0053] Considering that the greater the change in the number of coordinate points to be analyzed screened by adjacent distance ratios and the greater the change in the mean value of spatial distances, the greater the change in the density of the coordinate points to be analyzed, the area formed by the coordinate points to be analyzed screened by each distance ratio is used as the initial consensus area for each distance ratio; the product of the absolute value of the difference in the number of coordinate points to be analyzed screened by adjacent distance ratios and the absolute value of the difference in the mean value of the spatial distances of the coordinate points to be analyzed is used as the consensus accuracy of the smallest distance ratio in the corresponding adjacent distance ratios;

[0054] The initial consensus area corresponding to the maximum consensus accuracy is selected as the final consensus area; the three-dimensional coordinate points outside the final consensus area are taken as non-consensus coordinate points.

[0055] As an example, the calculation formula for consensus accuracy includes:

[0056]

[0057] Where m is the sequence number of the distance ratio in the preset distance ratio sequence; Q m represents the consensus accuracy corresponding to the mth distance ratio; n m Indicates the number of coordinate points to be analyzed for the mth distance ratio screening; n m+1 Indicates the number of coordinate points to be analyzed for the m+1th distance ratio screening; || indicates taking the absolute value; l m,p Indicates the spatial distance corresponding to the pth coordinate point to be analyzed selected by the mth distance ratio; l m+1,pIndicates the spatial distance corresponding to the pth coordinate point to be analyzed selected by the m+1th distance ratio.

[0058] In the calculation formula of consensus accuracy, the absolute value of the difference is used to express the change characteristics of the quantity and the average spatial distance. By multiplying, the change in quantity and the change in spatial distance are integrated to reflect the distribution change characteristics of the coordinate points to be analyzed; |n m+1 -n m | and The larger the value is, the greater the distribution change of the coordinate points to be analyzed is, and the more likely it is that there is a transition from a common identification area to a non-common identification area. The distance ratio starts from the m+1th distance ratio, and is more likely to include non-common identification areas that cannot be observed by the left and right cameras at the same time. The greater the accuracy of the coordinate points to be analyzed selected by the mth distance ratio belonging to the common identification area, the more likely it is that the coordinate points to be analyzed belong to the common identification area. As the consensus accuracy corresponding to the smallest distance ratio among the adjacent distance ratios, the initial consensus region corresponding to the largest consensus accuracy is selected as the final consensus region.

[0059] It should be noted that adjacent distance ratios refer to distance ratios that are adjacent in a preset distance ratio sequence, such as 0.1 and 0.2, 0.4 and 0.5 in [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8]. In other embodiments of the present invention, the implementer may also set other preset distance ratio sequences.

[0060] It should be noted that in other embodiments of the present invention, the implementer may also combine |n m+1 -n m | and to obtain consensus accuracy.

[0061] Step S3: Perform the first density clustering on the non-consensus coordinate points to obtain the first clustering cluster; select any non-consensus coordinate point as the target coordinate point; in the first clustering cluster of the target coordinate point, based on the maximum spatial distance between the non-consensus coordinate points, the preset radius change step and the preset number of spheres, obtain the radius sequence; in the first clustering cluster of the target coordinate point, take the target coordinate point as the center of the sphere and each element in the radius sequence as the radius to obtain the spherical range of the target coordinate point; perform secondary density clustering on the non-consensus coordinate points within each spherical range of the target coordinate point to obtain the secondary clustering cluster and the target cluster where the target coordinate point is located; according to the distance characteristics of the target cluster and other secondary clusters in all spherical ranges, combined with the number of elements in the target cluster, obtain the noise conformity of the target coordinate point; obtain the noise point based on the noise conformity.

[0062] Due to the blockage of light by the body position of the recognizer or the reflection of the jewelry worn, noise points will appear in the non-consensus coordinate points. Because the noise points themselves do not belong to the human face, the distribution of the three-dimensional coordinate points on the human face will be quite different. When the noise points are used as the center of the sphere and the range of spheres with different radii are circled, the degree of outliers shown by the noise points is relatively large. Therefore, when the non-consensus coordinate points are used as the center of the sphere under multiple sphere ranges, the multiple outliers shown by the three-dimensional coordinate points are used to jointly judge whether the three-dimensional coordinate points are noise.

[0063] Taking into account the differences in the distribution of three-dimensional coordinate points between different areas of the face, the non-consensus coordinate points are first subjected to the first density clustering to obtain the first clustering cluster, which is convenient for analyzing the local distribution characteristics of the non-consensus coordinate points and more accurately screening out noise points.

[0064] Select any non-consensus coordinate point as the target coordinate point to facilitate analysis one by one, wherein the analysis method for each non-consensus coordinate point is consistent, and finally the noise conformity of each non-consensus coordinate point is obtained. Here, only any one is taken as an example and no repetition is given.

[0065] In order to analyze the degree of outlier of the target coordinate point in multiple spherical ranges with the target coordinate point as the center, it is necessary to obtain multiple spherical ranges first. Therefore, in the first clustering cluster of the target coordinate point, the radius sequence is obtained based on the maximum spatial distance between non-consensus coordinate points, the preset radius change step and the preset number of spheres; in the first clustering cluster of the target coordinate point, the spherical range of the target coordinate point is obtained with the target coordinate point as the center and each element in the radius sequence as the radius.

[0066] Preferably, in one embodiment of the present invention, in the first clustering cluster of target coordinate points, the maximum spatial distance between non-consensus coordinate points is taken as the maximum radius; based on the maximum radius, the maximum radius is iteratively reduced with a preset radius change step size, and a new radius is obtained each iteration, and a preset number of spheres are obtained in total. The maximum radius and the iteratively obtained radii are constructed to obtain a radius sequence.

[0067] As an example, the preset radius change step is 0.5 mm, the preset number of spheres is 10, the maximum radius is iteratively reduced 9 times with the preset radius change step, a new radius is obtained in each iteration, and a total of 10 radii are obtained.

[0068] It should be noted that, in one embodiment of the present invention, the unit distance is 0.1 mm; both the first density clustering and the secondary density clustering use the DBSCAN density clustering algorithm, the first density clustering sets the core radius to 0.5 and the minimum number of points to 5, and obtains multiple first clustering clusters; the secondary density clustering sets the core radius to 0.5 and the minimum number of points to 5, and obtains multiple secondary clustering clusters.

[0069] In other embodiments of the present invention, the implementer may also set other DBSCAN density clustering algorithm parameters, and may also set different parameters for the first density clustering and the second density clustering, set other preset radius change steps and preset numbers of spheres, and select other density clustering algorithms such as DENCLUE or density peak clustering, which are all existing technologies and will not be described in detail.

[0070] After obtaining the radius sequence, we can use the target coordinate point as the center of the sphere and each element in the radius sequence as the radius in the first clustering cluster of the target coordinate point to obtain the spherical range of the target coordinate point, so as to prepare for analyzing the degree of outliers of the target coordinate point under different ranges.

[0071] In order to analyze the outlier degree characteristics of the target coordinate points in the spherical range, secondary density clustering is performed on the non-consensus coordinate points within each spherical range of the target coordinate points to obtain secondary clustering clusters and the target clustering cluster where the target coordinate points are located; considering that the number of elements in the target clustering cluster and the distance between the target clustering cluster and other secondary clustering clusters can both indicate the degree of isolation and deviation of the target clustering cluster, the noise conformity of the target coordinate points is obtained based on the distance characteristics of the target clustering cluster and other secondary clustering clusters in all spherical ranges, combined with the number of elements in the target clustering cluster, to characterize the conformity of the target coordinate points as noise points, and to provide a basis for screening out noise points.

[0072] Preferably, in one embodiment of the present invention, considering that the fewer the number of elements in the target cluster, and the farther the center point of the target cluster is from other secondary clusters, it means that the target cluster is more likely to contain noise points, the more it deviates from other secondary clusters, and the greater the degree of outlier; when the overall degree of outlier of the target cluster within the range of all spheres of the target coordinate point is greater, it means that the target coordinate point is more likely to be a noise point, and the noise conformity is greater;

[0073] Based on this, any spherical range of the target coordinate point is selected as the target spherical range; within the target spherical range, the number of elements in the target cluster is used as the first denominator, the average value of the distance between the center point of the target cluster and the center points of other secondary clusters is used as the first numerator, and the ratio of the first denominator to the first numerator is used as the noise sub-parameter of the target spherical range;

[0074] The mean of the noise sub-parameters of all spherical ranges of the target coordinate point is taken as the noise conformity of the target coordinate point.

[0075] As an example, the calculation formula of noise compliance includes:

[0076]

[0077] Where i represents the serial number of the target coordinate point; W iIndicates the noise conformity of the i-th target coordinate point; N i represents the number of target sphere ranges of the i-th target coordinate point, also the number of sphere ranges; j represents the sequence number of the target sphere range; n i,j V represents the number of elements in the target cluster within the j-th target sphere range of the i-th target coordinate point, and is also the first denominator corresponding to the j-th target sphere range of the i-th target coordinate point; i,j represents the number of secondary clusters in the jth target sphere of the i-th target coordinate point; v represents the sequence number of other secondary clusters of the target cluster in the target sphere; V i,j -1 indicates the number of other secondary clusters of the target cluster in the jth target sphere of the i-th target coordinate point; l i,j,v Represents the distance between the center point of the target cluster and the center point of the vth other secondary cluster in the jth target sphere range of the i-th target coordinate point; Represents the first molecule corresponding to the jth target sphere range of the i-th target coordinate point.

[0078] In the calculation formula of noise conformity, the smaller the first denominator is, the fewer elements in the target cluster are, the higher the possibility that the target cluster contains noise, and the greater the noise conformity is; the larger the first numerator is, the farther the target cluster is from other secondary clusters, the more obvious the deviation characteristics are, the greater the degree of outlier of the target coordinate point is, and the greater the noise conformity is.

[0079] It should be noted that, in one embodiment of the present invention, the centroid is obtained by using a centroid calculation formula to represent the center point of the cluster, and the Euclidean distance between the center points is used as the distance between the center points. This is a prior art and will not be described in detail.

[0080] After obtaining the noise conformity that represents the conformity degree of the target coordinate point as a noise point, the noise point can be obtained based on the noise conformity.

[0081] Preferably, in one embodiment of the present invention, non-consensus coordinate points whose noise conformity is greater than a preset conformity threshold are marked as noise points.

[0082] As an example, the noise conformity is normalized by a sigmoid function, and the preset conformity threshold is 0.7. When the normalized noise conformity is greater than 0.7, the corresponding non-consensus coordinate point is marked as a noise point.

[0083] Step S4: Obtain a preset number of non-noise points nearest to each noise point to fit the plane; obtain the corrected movement analysis range of each noise point based on the vertical distance between each noise point and the corresponding fitting plane, combined with the overall characteristics of the noise conformity of the obtained non-noise point; within the corrected movement analysis range of each noise point, take the intersection of the vertical lines of the fitting plane as the starting point, and move the intersection of the vertical lines with a preset moving step to obtain the noise movement analysis point of each noise point; obtain the corrected face point cloud based on the noise conformity of all noise movement analysis points of each noise point, combined with the three-dimensional coordinates of the non-noise point.

[0084] After determining the noise points, they need to be corrected to reduce the impact of the noise points on the face point cloud and improve the accuracy of face recognition.

[0085] Taking into account the local neighboring non-noise points of the noise points, the reference plane generated by geometric fitting represents the local geometric distribution of the non-noise points. The vertical distance between the noise points and the fitting plane can measure the degree of deviation and provide a basis for subsequent corrections. Considering that the noise points may be located in the wrong position, simply deleting them may lead to the loss of local information of the point cloud. Therefore, the positions of the noise points are gradually corrected by means of motion analysis to get closer to the real point cloud surface. Considering that the overall characteristics of the noise conformity of the non-noise points participating in the fitting plane reflect the overall noise trend of the point cloud, it can be distinguished whether the noise points are located in highly discrete abnormal areas, and the vertical distance of the noise points can be supplemented with local regional characteristics. Therefore, the preset number of non-noise points closest to each noise point is obtained to fit the plane. According to the vertical distance between each noise point and the corresponding fitting plane, combined with the overall characteristics of the noise conformity of the obtained non-noise points, the corrected motion analysis range of each noise point is obtained.

[0086] Preferably, in one embodiment of the present invention, considering that the larger the vertical distance, the greater the deviation between the noise point and the fitting plane, the greater the difference in geometric distribution between the noise point and the local non-noise point, the larger the correction movement analysis range should be to obtain the best correction effect; considering that the larger the noise conformity and value of the non-noise point participating in the fitting plane, the more likely the noise point is to be located in a highly discrete abnormal area, and a larger correction movement analysis range is required to ensure sufficient correction analysis space;

[0087] Based on this, the normalized value of the sum of the noise conformity values ​​of the non-noise points obtained by each noise point is multiplied by the vertical distance of each noise point as the modified analysis range radius of each noise point;

[0088] On the perpendicular line between each noise point and the fitting plane, with the intersection of the perpendicular lines as the center, two line segments with a length equal to the radius of the correction analysis range are intercepted on the perpendicular line as the correction moving analysis range of each noise point.

[0089] As an example, the calculation formula for modifying the analysis range radius includes:

[0090]

[0091] Where r represents the serial number of the noise point; L r ′ represents the vertical distance of the rth noise point; L r represents the radius of the modified analysis range of the rth noise point; sigmoid{} represents the sigmoid function; M represents the preset number; k represents the serial number of the non-noise point corresponding to the noise point; W k,r Indicates the noise conformity of the kth non-noise point selected by the rth noise point.

[0092] In the calculation formula of the modified analysis range radius, the sum of the noise conformity of the non-noise points obtained by the noise point is normalized by the sigmoid function, and L is integrated by multiplication. r 'and Gets the corrected analysis range radius.

[0093] It should be noted that, in one embodiment of the present invention, the preset number is 100, and the distance between the noise point and the non-noise point is measured by the Euclidean distance. Obtaining the vertical distance between a point and the plane, the intersection of the vertical lines, and fitting the plane based on the data points are all existing technologies. Specifically, the plane can be fitted by the least squares method, which will not be repeated here.

[0094] After determining the correction movement analysis range of the noise points, it is necessary to perform movement correction analysis on the noise points. Therefore, within the correction movement analysis range of each noise point, take the intersection of the vertical lines of the fitting plane as the starting point, and move the vertical line intersection with a preset moving step to obtain the noise movement analysis point of each noise point. By gradually moving the noise points, the best correction position can be found within the correction range, rather than simply pulling the noise points back to the fitting plane, thereby improving the correction effect.

[0095] See also Figure 3 , which shows a schematic diagram of noise movement analysis point distribution provided by an embodiment of the present invention; Figure 3 It includes noise points represented by hollow circles, noise movement analysis points represented by solid circles, non-noise point fitting planes and annotations of the corrected analysis range radius L. The straight line between the noise point and the fitting plane is the perpendicular line between the noise point and the fitting plane, and the intersection of the perpendicular line and the fitting plane is the intersection of the perpendicular line. The preset moving step size is set to 0.2 mm.

[0096] Considering that noise conformity can be used to characterize the degree of noise conformity of three-dimensional coordinate points, after obtaining the noise movement analysis point of each noise point, the optimal correction position of the noise point can be determined according to the noise conformity of the noise movement analysis point. Therefore, according to the noise conformity of all noise movement analysis points of each noise point and the three-dimensional coordinates of the non-noise points, the corrected face point cloud is obtained, which provides a more reliable basis for constructing a three-dimensional face model and improves the unlocking efficiency and accuracy.

[0097] Preferably, in one embodiment of the present invention, each noise point is moved to a corresponding noise movement analysis point with the minimum noise conformity, and the three-dimensional coordinates of the non-noise point are combined to obtain a corrected face point cloud.

[0098] It should be noted that, when obtaining the noise conformity of the noise movement analysis point of the noise point, the noise point is replaced by the noise movement analysis point, and step S3 is repeated to obtain the noise conformity of the corresponding noise movement analysis point.

[0099] Step S5: Unlocking is performed at least according to the corrected face point cloud.

[0100] Taking into account the influence of shooting angle, occlusion and other factors, the corrected face point cloud may not be verified by the face recognition system, so at least unlocking is performed based on the corrected face point cloud.

[0101] Preferably, in one embodiment of the present invention, a face matching degree is obtained according to the pre-stored face point cloud and the corrected face point cloud in the smart phone; when the face matching degree is greater than a preset face matching threshold, the mobile phone is unlocked;

[0102] Since one embodiment of the present invention combines a face recognition system and a fingerprint recognition system for multimodal biometric recognition, when the face matching degree is less than or equal to a preset face matching threshold, a fingerprint unlocking prompt is provided to obtain the fingerprint matching degree after the prompt;

[0103] The sum of the noise conformity of all noise points in the corrected face point cloud, the product of the fingerprint matching degree and the face matching degree are normalized and used as the unlocking suitability; when the unlocking suitability is greater than the preset unlocking suitability threshold, the phone is unlocked; when the unlocking suitability is less than or equal to the preset unlocking suitability threshold, the recognition is judged to be unsuccessful, and the fingerprint unlocking or face unlocking is prompted again.

[0104] As an example, the preset face matching threshold is 0.8, the preset unlocking suitability threshold is 0.7, and the calculation formula for unlocking suitability includes:

[0105]

[0106] Where E represents the suitability of the most recent unlocking; sigmoid{} represents the sigmoid function; r represents the sequence number of the noise point; R represents the number of noise points; W r ′ represents the noise matching degree of the rth noise point in the corrected face point cloud; Fa represents the most recent face matching degree; Fi represents the most recent fingerprint matching degree, p represents the degree of the most recent fingerprint match, p max Indicates the maximum value of fingerprint matching degree of all fingerprint unlocking of the smartphone.

[0107] In the calculation formula of unlocking suitability, the sum of the noise conformity of the noise point in the corrected face point cloud is used to indicate the interference degree of the noise point on face unlocking, so as to make supplementary correction to the face matching degree; the most recent fingerprint matching degree is normalized by using the maximum value of the fingerprint matching degree as the fingerprint matching degree; and the fusion is performed by multiplication. Fa and Fi, obtain the unlocking suitability, and combine face recognition and fingerprint recognition for multimodal biometric recognition.

[0108] It should be noted that the fingerprint recognition system, the face matching degree and the method for obtaining the fingerprint matching degree are all existing technologies. In one embodiment of the present invention, a template-based method is used to obtain the face matching degree, and a Biokey algorithm is used to obtain the fingerprint matching degree. Both are existing technologies and will not be described in detail.

[0109] An embodiment of the present invention also provides a multimodal biometric security unlocking system for a smartphone, the system comprising a memory, a processor and a computer program, wherein the memory is used to store the corresponding computer program, the processor is used to run the corresponding computer program, and when the computer program runs in the processor, it can implement a multimodal biometric security unlocking method for a smartphone described in steps S1-S5.

[0110] In summary, in view of the technical problem that the existing face recognition technology averages the error to the entire point cloud, the construction effect of the three-dimensional face model is not ideal, and the unlocking efficiency and accuracy are reduced, the present invention provides a multimodal biometric security unlocking method and system for smart phones. The present invention first obtains the three-dimensional coordinate points of the face model; further screens and obtains non-consensus coordinate points; further obtains the noise conformity of each non-consensus coordinate point, and obtains the noise point based on the noise conformity; further uses the distribution of the non-noise points closest to the noise point to move and correct the noise point, and combines the three-dimensional coordinates of the non-noise point to obtain a corrected face point cloud; finally, unlocks at least according to the corrected face point cloud. The present invention screens out the non-consensus coordinate points collected by two cameras, uses density clustering analysis to analyze the outlier features to screen the noise points, and obtains the corrected face point cloud after moving the noise points, which provides a more reliable basis for building a three-dimensional face model and improves the unlocking efficiency and accuracy.

[0111] It should be noted that the sequence of the above embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0112] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

Claims

1. A multimodal biometric security unlocking method for a smart phone, characterized in that: The method comprises: Acquire the three-dimensional coordinate points of the face model through the binocular structured light system; select the three-dimensional coordinate points corresponding to any camera as the coordinate points to be analyzed; Perform nearest neighbor matching on each coordinate point to be analyzed and the three-dimensional coordinate point corresponding to another camera, and calculate the spatial distance between each coordinate point to be analyzed and the corresponding matched three-dimensional coordinate point; filter the coordinate points to be analyzed based on the spatial distance and each distance ratio in the preset distance ratio sequence, and obtain non-consensus coordinate points according to the change in the number of the coordinate points to be analyzed filtered by the adjacent distance ratios and the change in the spatial distance; Performing a first density clustering on the non-consensus coordinate points to obtain a first clustering cluster; selecting any of the non-consensus coordinate points as a target coordinate point; in the first clustering cluster of the target coordinate point, obtaining a radius sequence based on the maximum spatial distance between the non-consensus coordinate points, a preset radius change step and a preset number of spheres; in the first clustering cluster of the target coordinate point, taking the target coordinate point as the center of the sphere and each element in the radius sequence as the radius, obtaining the spherical range of the target coordinate point; performing a second density clustering on the non-consensus coordinate points within each spherical range of the target coordinate point to obtain a second clustering cluster and a target cluster where the target coordinate point is located; obtaining the noise conformity of the target coordinate point based on the distance characteristics of the target cluster and other second clusters within all the spherical ranges, combined with the number of elements in the target cluster; obtaining a noise point based on the noise conformity; Obtain a preset number of non-noise points closest to each of the noise points to fit a plane; obtain a corrected movement analysis range for each of the noise points based on a vertical distance between each of the noise points and the corresponding fitting plane, combined with the overall characteristics of the noise conformity of the non-noise points obtained; within the corrected movement analysis range for each of the noise points, take the intersection of the vertical lines of the fitting plane as the starting point, move the intersection of the vertical lines with a preset moving step length to obtain a noise movement analysis point for each of the noise points; obtain a corrected face point cloud based on the noise conformity of all of the noise movement analysis points of each of the noise points, combined with the three-dimensional coordinates of the non-noise points; Unlocking is performed at least according to the corrected face point cloud.

2. A multi-modal biometric security unlocking method for a smart phone according to claim 1, characterized in that: The method for obtaining the non-consensus coordinate points includes: The ratio of the spatial distance corresponding to the coordinate point to be analyzed to the maximum spatial distance is used as the normalized distance corresponding to the coordinate point to be analyzed; when the normalized distance of the coordinate point to be analyzed is less than or equal to the distance ratio, it is determined that the screening corresponding to the distance ratio is passed; The area formed by the coordinate points to be analyzed that are screened by each distance ratio is used as the initial consensus area of ​​each distance ratio; the product of the absolute value of the difference between the number of coordinate points to be analyzed that are screened by adjacent distance ratios and the absolute value of the difference between the average values ​​of the spatial distances of the coordinate points to be analyzed is used as the consensus accuracy of the smallest distance ratio among the adjacent distance ratios; The initial consensus region corresponding to the maximum consensus accuracy is selected as the final consensus region; and the three-dimensional coordinate points outside the final consensus region are used as non-consensus coordinate points.

3. The multi-modal biometric security unlocking method for a smart phone according to claim 1, characterized in that: The method for obtaining the radius sequence includes: In the first clustering cluster of the target coordinate points, the maximum spatial distance between the non-consensus coordinate points is taken as the maximum radius; based on the maximum radius, the maximum radius is iteratively reduced with a preset radius change step size, and a new radius is obtained in each iteration, and a preset number of spheres are obtained in total. The maximum radius and the iteratively obtained radii are constructed to obtain a radius sequence.

4. The multi-modal biometric security unlocking method for a smart phone according to claim 1, characterized in that: The method for obtaining the noise compliance includes: Select any of the spherical ranges of the target coordinate point as the target spherical range; within the target spherical range, use the number of elements in the target cluster as the first denominator, use the average value of the distances between the center point of the target cluster and the center points of other secondary clusters as the first numerator, and use the ratio of the first denominator to the first numerator as the noise sub-parameter of the target spherical range; The average value of the noise sub-parameters of all the spherical ranges of the target coordinate point is taken as the noise conformity of the target coordinate point.

5. A multi-modal biometric security unlocking method for a smart phone according to claim 4, characterized in that: The method for obtaining the noise points comprises: The non-consensus coordinate points whose noise conformity is greater than a preset conformity threshold are marked as noise points.

6. A multi-modal biometric security unlocking method for a smart phone according to claim 1, characterized in that: The method for obtaining the modified movement analysis range includes: The product of the normalized value of the sum of the noise conformity values ​​of the non-noise points obtained by each noise point and the vertical distance of each noise point is used as the modified analysis range radius of each noise point; On the perpendicular line between each noise point and the fitting plane, with the intersection of the perpendicular lines as the center, two line segments with a length equal to the radius of the corrected analysis range are intercepted on the perpendicular line as the corrected moving analysis range of each noise point.

7. A multi-modal biometric security unlocking method for a smart phone according to claim 1, characterized in that: The method for obtaining a corrected face point cloud according to the noise conformity of the noise motion analysis point comprises: Each of the noise points is moved to the corresponding noise movement analysis point where the noise conformity is the smallest, and the three-dimensional coordinates of the non-noise points are combined to obtain a corrected face point cloud.

8. The multi-modal biometric security unlocking method for a smart phone according to claim 1, characterized in that: The method for unlocking at least according to the modified face point cloud comprises: Acquire a face matching degree according to the pre-stored face point cloud in the smart phone and the corrected face point cloud; unlock the phone when the face matching degree is greater than a preset face matching threshold; When the face matching degree is less than or equal to the preset face matching threshold, a fingerprint unlocking prompt is given to obtain the fingerprint matching degree after the prompt; the sum of the noise conformity degrees of all noise points in the corrected face point cloud and the product of the fingerprint matching degree and the face matching degree are normalized as the unlocking suitability; when the unlocking suitability is greater than the preset unlocking suitability threshold, the phone is unlocked; when the unlocking suitability is less than or equal to the preset unlocking suitability threshold, it is determined that the recognition has failed, and the fingerprint unlocking or face unlocking prompt is given again.

9. The multi-modal biometric security unlocking method for a smart phone according to claim 1, characterized in that: The binocular structured light system at least includes a left camera, a right camera and a projector.

10. A multimodal biometric security unlocking system for a smart phone, the system comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of a multimodal biometric security unlocking method for a smartphone as described in any one of claims 1 to 9 are implemented.

Images