A multi-modal biometric security unlocking method and system for a smart phone

Abstract

The present invention relates to the technical field of face recognition, and specifically relates to a multi-modal biometric security unlocking method and system for a smart phone. The present invention first obtains the three-dimensional coordinate points of a face model; further screens and obtains non-consensus coordinate points; further obtains the noise compliance of each non-consensus coordinate point, and obtains noise points based on the noise compliance; further, by means of the distribution of non-noise points closest to the noise points, the noise points are corrected by moving, and combined with the three-dimensional coordinates of the non-noise points, a corrected face point cloud is obtained; finally, unlocking is performed at least according to the corrected face point cloud. By screening out non-consensus coordinate points collected by two cameras, the present invention uses density clustering analysis to identify outlier features so as to screen noise points, and after correcting the noise points by moving, a corrected face point cloud is obtained, providing a more reliable basis for constructing a three-dimensional face model and improving the unlocking efficiency and accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of face recognition, and particularly to a multi-modal biometric security unlocking method and system for a smart phone. Background Art

[0002] Multi-modal biometric security unlocking refers to the integration of multiple biometric technologies, making use of the unique advantages of its multiple biometric technologies, and performing identification and authentication by analyzing and judging the feature values of multiple biometric methods. The existing common combination of fingerprint and face recognition can integrate multiple biometric modes, relieve the pressure of a single-modal system, and provide higher accuracy at the same time.

[0003] However, the traditional method uses the ICP algorithm to minimize the least square error between the corresponding three-dimensional coordinate points obtained by binocular vision as the final registration purpose. The final result only averages the error to the entire point cloud, and the construction effect of the face three-dimensional model is not ideal, resulting in reduced unlocking efficiency and accuracy. Summary of the Invention

[0004] In order to solve the technical problem that the existing face recognition technology averages the error to the entire point cloud, and the construction effect of the face three-dimensional model is not ideal, resulting in reduced unlocking efficiency and accuracy, the purpose of the present invention is to provide a multi-modal biometric security unlocking method and system for a smart phone. The specific technical solutions adopted are as follows:

[0005] A multi-modal biometric security unlocking method for a smart phone, the method comprising:

[0006] Obtaining three-dimensional coordinate points of a face model through a binocular structured light system; selecting the three-dimensional coordinate points corresponding to any one camera as the coordinate points to be analyzed;

[0007] Performing nearest neighbor matching on each of the coordinate points to be analyzed with the three-dimensional coordinate points corresponding to the other camera, and calculating the spatial distance between each of the coordinate points to be analyzed and the corresponding matched three-dimensional coordinate points; screening the coordinate points to be analyzed based on the spatial distance and each distance ratio in a preset distance ratio sequence, and obtaining non-consensus coordinate points according to the change in the number of the coordinate points to be analyzed screened by adjacent distance ratios and the change in the spatial distance.

[0008] Perform first - density clustering on the non - consensus coordinate points to obtain first - clustering clusters; select any one of the non - consensus coordinate points as the target coordinate point; in the first - clustering cluster of the target coordinate point, based on the maximum spatial distance between the non - consensus coordinate points, a preset radius change step, and a preset number of spheres, obtain a radius sequence; in the first - clustering cluster of the target coordinate point, with the target coordinate point as the center of the sphere and each element in the radius sequence as the radius, obtain the sphere range of the target coordinate point; perform second - density clustering on the non - consensus coordinate points within each sphere range of the target coordinate point to obtain second - clustering clusters and the target - clustering cluster where the target coordinate point is located; according to the distance characteristics of all the sphere ranges, the target - clustering cluster, and other second - clustering clusters, combined with the number of elements within the target - clustering cluster, obtain the noise compliance of the target coordinate point; obtain noise points based on the noise compliance;

[0009] For each noise point, obtain the nearest preset number of non - noise points for plane fitting; according to the perpendicular distance between each noise point and the corresponding fitting plane, combined with the overall characteristics of the noise compliance of the obtained non - noise points, obtain the corrected movement analysis range of each noise point; within the corrected movement analysis range of each noise point, starting from the intersection point of the perpendicular lines of the fitting plane, move the intersection point of the perpendicular lines with a preset movement step to obtain the noise movement analysis points of each noise point; according to the noise compliance of all the noise movement analysis points of each noise point, combined with the three - dimensional coordinates of non - noise points, obtain the corrected face point cloud;

[0010] Perform unlocking at least based on the corrected face point cloud.

[0011] Further, the method for obtaining non - consensus coordinate points includes:

[0012] Take the ratio of the spatial distance corresponding to the coordinate point to be analyzed to the maximum spatial distance as the normalized distance corresponding to the coordinate point to be analyzed; when the normalized distance of the coordinate point to be analyzed is less than or equal to the distance ratio, determine that it passes the screening by the corresponding distance ratio;

[0013] Take the region composed of the coordinate points to be analyzed that pass the screening by each distance ratio as the initial consensus region for each distance ratio; take the product of the absolute value of the difference in the number of coordinate points to be analyzed screened by adjacent distance ratios and the absolute value of the difference in the average value of the spatial distances of the coordinate points to be analyzed as the consensus accuracy of the smallest distance ratio in the corresponding adjacent distance ratios;

[0014] Select the initial consensus region corresponding to the maximum of the consensus accuracies as the final consensus region; consider the three-dimensional coordinate points outside the final consensus region as non-consensus coordinate points.

[0015] Further, the method for obtaining the radius sequence includes:

[0016] In the first clustering cluster of the target coordinate point, consider the maximum spatial distance between the non-consensus coordinate points as the maximum radius; based on the maximum radius, iteratively reduce the maximum radius by a preset radius change step size, obtaining a new radius each time, and obtaining a total of the preset number of sphere radii. Construct a radius sequence from the maximum radius and the iteratively obtained radii.

[0017] Further, the method for obtaining the noise compliance includes:

[0018] Select any sphere range of the target coordinate point as the target sphere range; within the target sphere range, consider the number of elements in the target clustering cluster as the first denominator, and consider the average value of the distances between the center point of the target clustering cluster and the center points of other secondary clustering clusters as the first numerator. Consider the ratio of the first denominator to the first numerator as the noise sub-parameter of the target sphere range.

[0019] Consider the average value of the noise sub-parameters of all the sphere ranges of the target coordinate point as the noise compliance of the target coordinate point.

[0020] Further, the method for obtaining the noise points includes:

[0021] Mark the non-consensus coordinate points with a noise compliance greater than a preset compliance threshold as noise points.

[0022] Further, the method for obtaining the corrected movement analysis range includes:

[0023] Take the product of the normalized value of the sum of the noise compliances of the non-noise points obtained for each noise point and the vertical distance of each noise point as the corrected analysis range radius for each noise point.

[0024] On the perpendicular line of each noise point and the fitting plane, with the intersection point of the perpendicular line as the center, intercept two line segments of length equal to the corrected analysis range radius on the perpendicular line as the corrected movement analysis range for each noise point.

[0025] Further, the method for obtaining the corrected face point cloud based on the noise compliance of the noise movement analysis points includes:

[0026] Move each of the noise points to the corresponding noise movement analysis point with the minimum noise compliance, and combine the three-dimensional coordinates of the non-noise points to obtain a corrected face point cloud.

[0027] Further, the method for unlocking at least based on the corrected face point cloud includes:

[0028] Obtain a face matching degree according to the pre-stored face point cloud in the smart phone and the corrected face point cloud; when the face matching degree is greater than a preset face matching threshold, unlock the phone;

[0029] When the face matching degree is less than or equal to the preset face matching threshold, prompt for fingerprint unlocking and obtain the fingerprint matching degree after the prompt; normalize the sum of the noise compliances of all noise points in the corrected face point cloud, the product of the fingerprint matching degree and the face matching degree, and use it as the unlocking suitability; when the unlocking suitability is greater than the preset unlocking suitability threshold, unlock the phone; when the unlocking suitability is less than or equal to the preset unlocking suitability threshold, determine that the recognition fails, and re-prompt for fingerprint unlocking or face unlocking.

[0030] Further, the binocular structured light system at least includes a left camera, a right camera, and a projector.

[0031] The present invention also provides a multi-modal biometric security unlocking system for a smart phone. The system includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of any one of the multi-modal biometric security unlocking methods for a smart phone.

[0032] The present invention has the following beneficial effects:

[0033] The present invention first obtains the three-dimensional coordinate points of the face model through a binocular structured light system to provide an analysis basis; further, according to the change in the number of coordinate points to be analyzed and the change in spatial distance screened by the adjacent distance ratio, by continuously changing the screened distance range, observing the change in the number of coordinate points to be analyzed to analyze the density change, and screening to obtain non-consensus coordinate points that are prone to noise when collected by only a single camera; further, performing a first density clustering on the non-consensus coordinate points to obtain a first clustering cluster, which is convenient for analyzing the local distribution characteristics of the non-consensus coordinate points; further, taking advantage of the characteristic that noise points will maintain a high degree of outlier under density clustering in multiple ranges, obtaining multiple spherical ranges of the target coordinate points; performing a second density clustering on the non-consensus coordinate points within each spherical range of the target coordinate points to obtain a second clustering cluster, analyzing the distance characteristics between the target clustering cluster where the target coordinate points are located and other second clustering clusters, and combining the number of elements within the target clustering cluster, obtaining the noise compliance of the target coordinate points, which characterizes the degree of compliance of the target coordinate points as noise points, and obtaining noise points based on the noise compliance; further, using a fitting plane to represent the local geometric distribution of non-noise points, analyzing the overall characteristics of the noise compliance of non-noise points locally adjacent to the noise points, obtaining the corrected movement analysis range of each noise point, performing movement correction on the noise points, and combining the three-dimensional coordinates of the non-noise points to obtain a corrected face point cloud, providing a more reliable basis for constructing a three-dimensional face model; finally, unlocking at least according to the corrected face point cloud. By screening out non-consensus coordinate points collected by two cameras, the present invention uses density clustering analysis to screen noise points based on outlier characteristics, performs movement correction on the noise points, and then obtains a corrected face point cloud, providing a more reliable basis for constructing a three-dimensional face model, and improving the unlocking efficiency and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0035] Figure 1 It is a flowchart of a multi-modal biometric security unlocking method for a smart phone provided by an embodiment of the present invention;

[0036] Figure 2 It is a working schematic diagram of a binocular structured light system provided by an embodiment of the present invention;

[0037] Figure 3 It is a schematic diagram of the distribution of noise movement analysis points provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0038] To further elaborate on the technical means and effects adopted by the present invention to achieve the intended invention purpose, the following specifically describes, in conjunction with the accompanying drawings and preferred embodiments, a multi-modal biometric security unlocking method and system for a smart phone according to the present invention, including its specific implementation manners, structures, features, and effects. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0039] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which the present invention belongs.

[0040] The following specifically describes, in conjunction with the accompanying drawings, the specific solutions of a multi-modal biometric security unlocking method and system for a smart phone provided by the present invention.

[0041] Please refer to Figure 1 , which shows a flowchart of a multi-modal biometric security unlocking method for a smart phone provided by an embodiment of the present invention, specifically including:

[0042] Step S1: Obtain the three-dimensional coordinate points of the face model through a binocular structured light system; select the three-dimensional coordinate points corresponding to any one of the cameras as the coordinate points to be analyzed.

[0043] In the embodiment of the present invention, for a smart phone equipped with a face recognition system and a fingerprint recognition system with a binocular structured light system, biometric unlocking of the smart phone is performed. First, the three-dimensional coordinate points of the face model are obtained through the binocular structured light system, and the corrected face point cloud is obtained through analysis. Based on the corrected face point cloud, face recognition determination is performed. When the face recognition determination fails, fingerprint unlocking is prompted and fingerprint data is collected, and the fingerprint matching degree is obtained. The face recognition system is jointly used to determine whether to unlock.

[0044] First, the three-dimensional coordinate points of the face model are obtained through the binocular structured light system to obtain the basis for analysis data; it should be noted that the binocular structured light system at least includes a left camera, a right camera, and a projector, and the working principles of each component are already prior art and will not be elaborated here.

[0045] Considering that there will be a common capture area for the left and right cameras, please refer to Figure 2 , which shows a schematic diagram of the operation of a binocular structured light system provided by an embodiment of the present invention. The figure includes a left camera, a right camera, and a projector. Among them, two rays starting from the camera lens are the capture areas of the cameras. Figure 2 The bottom curve in is the face schematic curve. The area between A and C is the capture area of the left camera, the area between B and D is the capture area of the right camera, and the area between B and C is the common capture area.

[0046] If both cameras can recognize a certain area from different perspectives, it indicates that the distribution of three-dimensional coordinate points in a certain area is more in line with the actual human face situation. When reconstructing a three-dimensional human face model with monocular structured light, due to the occlusion of light by the body position itself or the reflection of light by the accessories worn, holes or noises will often appear in the obtained human face point cloud. Therefore, it is necessary to screen the three-dimensional coordinate points. So, select the three-dimensional coordinate points corresponding to any one camera as the coordinate points to be analyzed, so as to conduct comparative analysis in the subsequent steps and screen out the non-consensus coordinate points in the non-common recognition area.

[0047] In an embodiment of the present invention, select the three-dimensional coordinate points collected by the left camera as the coordinate points to be analyzed, and use the three-dimensional coordinate points collected by the right camera as a reference to compare the two to obtain non-consensus coordinate points. In other embodiments of the present invention, the implementer can also set to select the three-dimensional coordinate points of the right camera as the coordinate points to be analyzed.

[0048] Step S2: Perform nearest neighbor matching between each coordinate point to be analyzed and the three-dimensional coordinate points corresponding to the other camera, and calculate the spatial distance between each coordinate point to be analyzed and the corresponding matched three-dimensional coordinate points; screen the coordinate points to be analyzed based on the spatial distance and each distance ratio in the preset distance ratio sequence, and obtain non-consensus coordinate points according to the change in the number of coordinate points to be analyzed screened by adjacent distance ratios and the change in spatial distance.

[0049] Considering that when the coordinate point to be analyzed is closer to the three-dimensional coordinate points corresponding to the other camera, the coordinate point to be analyzed is more likely to be in the human face area that can be captured by both cameras. Therefore, perform nearest neighbor matching between each coordinate point to be analyzed and the three-dimensional coordinate points corresponding to the other camera, and calculate the spatial distance between each coordinate point to be analyzed and the corresponding matched three-dimensional coordinate points for subsequent analysis.

[0050] It should be noted that in an embodiment of the present invention, the Euclidean distance between the three-dimensional coordinate points is obtained as the spatial distance, and each coordinate point to be analyzed collected by the left camera is matched with the three-dimensional coordinate points collected by the nearest right camera to obtain the nearest neighbor matching result. In other embodiments of the present invention, the implementer can also obtain other distances such as the Manhattan distance as the spatial distance, which are all well-known technical means in the art and will not be elaborated here.

[0051] Considering that the three-dimensional coordinate points in the face regions captured by two cameras are denser than those captured by a single camera, the boundary between the common capture region and the non-common recognition region can be analyzed through the density change of the data points to be analyzed. Therefore, the coordinate points to be analyzed can be screened based on the spatial distance and each distance ratio in the preset distance ratio sequence. According to the change in the number of coordinate points to be analyzed screened by adjacent distance ratios and the change in spatial distance, by continuously changing the screening distance range, the density change is analyzed by observing the change in the number of coordinate points to be analyzed, and the non-consensus coordinate points are obtained.

[0052] Preferably, in an embodiment of the present invention, considering that when the spatial distance of the coordinate points to be analyzed is large, the possibility of being in the common capture region is low, the preset distance ratio sequence is set as [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8]. The ratio of the spatial distance corresponding to the coordinate points to be analyzed to the maximum spatial distance is used as the normalized distance of the corresponding coordinate points to be analyzed. When the normalized distance of the coordinate points to be analyzed is less than or equal to the distance ratio, it is determined that the screening is passed through the corresponding distance ratio.

[0053] Considering that the greater the change in the number of coordinate points to be analyzed screened by adjacent distance ratios and the greater the change in the average spatial distance, the greater the density change of the coordinate points to be analyzed. The region formed by the coordinate points to be analyzed screened by each distance ratio is used as the initial consensus region for each distance ratio. The product of the absolute value of the difference in the number of coordinate points to be analyzed screened by adjacent distance ratios and the absolute value of the difference in the average spatial distance of the coordinate points to be analyzed is used as the consensus accuracy of the smallest distance ratio in the corresponding adjacent distance ratios.

[0054] The initial consensus region corresponding to the maximum consensus accuracy is selected as the final consensus region. The three-dimensional coordinate points outside the final consensus region are used as non-consensus coordinate points.

[0055] As an example, the calculation formula of the consensus accuracy includes:

[0056]

[0057] where m is the serial number of the distance ratio in the preset distance ratio sequence; Q m represents the consensus accuracy corresponding to the mth distance ratio; n m represents the number of coordinate points to be analyzed screened by the mth distance ratio; n m+1 represents the number of coordinate points to be analyzed screened by the (m + 1)th distance ratio; || represents taking the absolute value; l m,p represents the spatial distance corresponding to the pth coordinate point to be analyzed screened by the mth distance ratio; l m+1,pIt represents the spatial distance corresponding to the p-th coordinate point to be analyzed in the (m + 1)-th distance ratio screening.

[0058] In the calculation formula of the consensus accuracy, the change characteristics of the quantity and the average spatial distance are represented by the absolute value of the difference, and the change of the quantity and the change of the spatial distance are fused by multiplication to reflect the change characteristics of the distribution of the coordinate points to be analyzed; |n m+1 -n m | and The larger they are, the greater the change in the distribution of the coordinate points to be analyzed, and the more likely it is a transition from the common recognition area to the non-common recognition area. Starting from the (m + 1)-th distance ratio, it is more likely to include the non-common recognition area that cannot be observed by the actual left and right cameras at the same time. The greater the accuracy of the coordinate points to be analyzed screened by the m-th distance ratio belonging to the common recognition area. Therefore, is used as the consensus accuracy of the smallest distance ratio in the corresponding adjacent distance ratios, and the initial consensus area corresponding to the largest consensus accuracy is selected as the final consensus area.

[0059] It should be noted that the adjacent distance ratios refer to those adjacent in the preset distance ratio sequence, such as 0.1 and 0.2, 0.4 and 0.5 in [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8]. In other embodiments of the present invention, the implementer can also set other preset distance ratio sequences.

[0060] It should be noted that in other embodiments of the present invention, the implementer can also fuse |n m+1 -n m | and by addition or weighted summation to obtain the consensus accuracy.

[0061] Step S3: Perform the first density clustering on the non-consensus coordinate points to obtain the first clustering clusters; select any non-consensus coordinate point as the target coordinate point; in the first clustering cluster of the target coordinate point, based on the maximum spatial distance between the non-consensus coordinate points, the preset radius change step size, and the preset number of spheres, obtain the radius sequence; in the first clustering cluster of the target coordinate point, with the target coordinate point as the center of the sphere and each element in the radius sequence as the radius, obtain the sphere range of the target coordinate point; perform the second density clustering on the non-consensus coordinate points within each sphere range of the target coordinate point to obtain the second clustering clusters and the target clustering cluster where the target coordinate point is located; according to the distance characteristics of all sphere ranges, the target clustering cluster and other second clustering clusters, and combining the number of elements within the target clustering cluster, obtain the noise compliance of the target coordinate point; obtain the noise points based on the noise compliance.

[0062] Due to the occlusion of light by the posture of the identifier or the reflection of the ornaments worn, noise points will appear in the non-consensus coordinate points. Since the noise points themselves do not belong to the human face, there will be a significant difference in the distribution of the three-dimensional coordinate points on the human face. When spheres with different radii are defined with the noise points as the centers, the degree of outliers shown by the noise points is relatively large. Therefore, when the non-consensus coordinate points are used as the centers in multiple sphere ranges, the multiple degrees of outliers shown by the three-dimensional coordinate points are jointly used to judge whether this three-dimensional coordinate point is noise.

[0063] Considering the differences in the distribution of three-dimensional coordinate points between different regions of the human face, the non-consensus coordinate points are first subjected to the first density clustering to obtain the first clustering clusters, which is convenient for analyzing the local distribution characteristics of the non-consensus coordinate points and more accurately screening out the noise points.

[0064] Select any non-consensus coordinate point as the target coordinate point for convenient individual analysis. The analysis method for each non-consensus coordinate point is the same. Finally, the noise compliance of each non-consensus coordinate point is obtained. Here, only one example is given and will not be repeated.

[0065] To analyze the degree of outliers of the target coordinate point in multiple sphere ranges with the target coordinate point as the center, it is first necessary to obtain multiple sphere ranges. Therefore, in the first clustering cluster of the target coordinate point, based on the maximum spatial distance between non-consensus coordinate points, the preset radius change step size, and the preset number of spheres, a radius sequence is obtained; in the first clustering cluster of the target coordinate point, with the target coordinate point as the center and each element in the radius sequence as the radius, the sphere ranges of the target coordinate point are obtained.

[0066] Preferably, in an embodiment of the present invention, in the first clustering cluster of the target coordinate point, the maximum spatial distance between non-consensus coordinate points is used as the maximum radius; based on the maximum radius, the maximum radius is iteratively reduced with the preset radius change step size. Each iteration obtains a new radius, and a total of the preset number of spheres of radii are obtained. The maximum radius and the radii obtained by iteration are used to construct a radius sequence.

[0067] As an example, the preset radius change step size is 0.5 mm, the preset number of spheres is 10, and the maximum radius is iteratively reduced 9 times with the preset radius change step size. Each iteration obtains a new radius, and a total of 10 radii are obtained.

[0068] It should be noted that, in an embodiment of the present invention, the unit distance is 0.1 mm; both the first density clustering and the second density clustering use the DBSCAN density clustering algorithm. The first density clustering sets the core radius to 0.5 and the minimum number of points to 5 to obtain multiple first clustering clusters; the second density clustering sets the core radius to 0.5 and the minimum number of points to 5 to obtain multiple second clustering clusters.

[0069] In other embodiments of the present invention, the implementer can also set other DBSCAN density clustering algorithm parameters, and can also set different parameters for the first density clustering and the second density clustering, set other preset radius change steps and preset number of spheres, and select other density clustering algorithms such as DENCLUE or density peak clustering, which are all prior arts and will not be elaborated herein.

[0070] After obtaining the radius sequence, in the first clustering cluster of the target coordinate point, with the target coordinate point as the center of the sphere and each element in the radius sequence as the radius, the sphere range of the target coordinate point can be obtained, preparing for analyzing the outlier degree of the target coordinate point under different ranges.

[0071] To analyze the outlier degree characteristics of the target coordinate point in the sphere range, perform secondary density clustering on the non-consensus coordinate points within each sphere range of the target coordinate point to obtain the secondary clustering clusters and the target clustering cluster where the target coordinate point is located; considering that the number of elements in the target clustering cluster and the distance between the target clustering cluster and other secondary clustering clusters can both represent the isolation degree and deviation degree of the target clustering cluster, so according to the distance characteristics between the target clustering cluster and other secondary clustering clusters in all sphere ranges, combined with the number of elements in the target clustering cluster, obtain the noise compliance of the target coordinate point, which characterizes the compliance degree of the target coordinate point as a noise point, providing a basis for screening out noise points.

[0072] Preferably, in one embodiment of the present invention, considering that the fewer the number of elements in the target clustering cluster and the farther the center point of the target clustering cluster is from other secondary clustering clusters, it indicates that the target clustering cluster is more likely to contain noise points, is more deviated from other secondary clustering clusters, and has a greater outlier degree; when the overall outlier degree of the target clustering clusters in all sphere ranges of the target coordinate point is greater, it indicates that the target coordinate point is more likely to be a noise point and the noise compliance is greater;

[0073] Based on this, select any sphere range of the target coordinate point as the target sphere range; within the target sphere range, use the number of elements in the target clustering cluster as the first denominator, use the average value of the distances between the center point of the target clustering cluster and the center points of other secondary clustering clusters as the first numerator, and use the ratio of the first denominator to the first numerator as the noise sub-parameter of the target sphere range;

[0074] Use the mean value of the noise sub-parameters of all sphere ranges of the target coordinate point as the noise compliance of the target coordinate point.

[0075] As an example, the calculation formula of the noise compliance includes:

[0076]

[0077] Among them, i represents the serial number of the target coordinate point; W iDenote the noise compliance of the $i$-th target coordinate point; $N$ i Denote the number of target sphere ranges of the $i$-th target coordinate point, which is also the number of sphere ranges; $j$ denotes the serial number of the target sphere range; $n$ i,j Denote the number of elements within the target clustering cluster in the $j$-th target sphere range of the $i$-th target coordinate point, which is also the first denominator corresponding to the $j$-th target sphere range of the $i$-th target coordinate point; $V$ i,j Denote the number of secondary clustering clusters in the $j$-th target sphere range of the $i$-th target coordinate point; $v$ denotes the serial number of other secondary clustering clusters of the target clustering cluster in the target sphere range; $V$ i,j -1 denotes the number of other secondary clustering clusters of the target clustering cluster in the $j$-th target sphere range of the $i$-th target coordinate point; $l$ i,j,v Denote the distance between the center point of the target clustering cluster and the center point of the $v$-th other secondary clustering cluster in the $j$-th target sphere range of the $i$-th target coordinate point; Denote the first numerator corresponding to the $j$-th target sphere range of the $i$-th target coordinate point.

[0078] In the calculation formula of the noise compliance, the smaller the first denominator is, the fewer the number of elements within the target clustering cluster is, and the higher the possibility that the target clustering cluster contains noise is, and the greater the noise compliance is; the larger the first numerator is, the farther the target clustering cluster is from other secondary clustering clusters, the more obvious the deviation feature is, the greater the degree of outlier of the target coordinate point is, and the greater the noise compliance is.

[0079] It should be noted that in an embodiment of the present invention, obtaining the center point representing the clustering cluster through the centroid calculation formula and taking the Euclidean distance between the center points as the distance between the center points is already a prior art and will not be elaborated herein.

[0080] After obtaining the noise compliance that characterizes the degree of compliance of the target coordinate point as a noise point, the noise points can be obtained based on the noise compliance.

[0081] Preferably, in an embodiment of the present invention, the non-consensus coordinate points with a noise compliance greater than a preset compliance threshold are marked as noise points.

[0082] As an example, the noise compliance is normalized through the sigmoid function, the preset compliance threshold is 0.7, and when the normalized noise compliance is greater than 0.7, the corresponding non-consensus coordinate points are marked as noise points.

[0083] Step S4: Obtain a preset number of non-noise points closest to each noise point for fitting a plane; according to the perpendicular distance between each noise point and the corresponding fitted plane, combined with the overall characteristics of the noise compliance of the obtained non-noise points, obtain the corrected movement analysis range for each noise point; within the corrected movement analysis range of each noise point, starting from the intersection point of the perpendicular line of the fitted plane, move the intersection point of the perpendicular line by a preset movement step to obtain the noise movement analysis points for each noise point; according to the noise compliance of all the noise movement analysis points of each noise point, combined with the three-dimensional coordinates of the non-noise points, obtain the corrected face point cloud.

[0084] After determining the noise points, it is necessary to correct the noise points to reduce the influence of the noise points on the face point cloud and improve the accuracy of face recognition.

[0085] Considering the non-noise points locally adjacent to the noise points, the reference plane generated by geometric fitting represents the local geometric distribution of the non-noise points. The perpendicular distance between the noise point and the fitted plane can measure its deviation degree, providing a basis for subsequent correction; considering that the noise points may be in the wrong positions, simply deleting them may cause loss of local information of the point cloud. Therefore, by means of movement analysis, the positions of the noise points are gradually corrected to be closer to the real point cloud surface; considering that the overall characteristics of the noise compliance of the non-noise points participating in the fitted plane reflect the overall noise trend of the point cloud, which can distinguish whether the noise points are located in highly discrete abnormal regions, and can supplement the local region characteristics to the vertical distance of the noise points. Therefore, obtain a preset number of non-noise points closest to each noise point for fitting a plane; according to the perpendicular distance between each noise point and the corresponding fitted plane, combined with the overall characteristics of the noise compliance of the obtained non-noise points, obtain the corrected movement analysis range for each noise point.

[0086] Preferably, in an embodiment of the present invention, considering that the greater the perpendicular distance, the greater the deviation degree of the noise point from the fitted plane, and the greater the geometric distribution difference between the noise point and the local non-noise points, the corrected movement analysis range should also be larger to obtain the best correction effect; considering that the greater the sum value of the noise compliance of the non-noise points participating in the fitted plane, the more likely the noise points are located in highly discrete abnormal regions, and a larger corrected movement analysis range is required to ensure sufficient correction analysis space;

[0087] Based on this, the product of the normalized value of the sum value of the noise compliance of the non-noise points obtained for each noise point and the perpendicular distance of each noise point is used as the radius of the corrected analysis range for each noise point;

[0088] On the perpendicular line between each noise point and the fitted plane, with the intersection point of the perpendicular line as the center, intercept two line segments with a length equal to the radius of the corrected analysis range on the perpendicular line as the corrected movement analysis range for each noise point.

[0089] As an example, the formula for correcting the analysis range radius includes:

[0090]

[0091] where r represents the serial number of the noise point; L r ′ represents the vertical distance of the r-th noise point; L r represents the corrected analysis range radius of the r-th noise point; sigmoid{} represents the sigmoid function; M represents the preset quantity; k represents the serial number of the non-noise point corresponding to the noise point; W k,r represents the noise compliance of the k-th non-noise point selected by the r-th noise point.

[0092] In the formula for calculating the corrected analysis range radius, the sum value of the noise compliance of the non-noise points obtained by the noise points is normalized through the sigmoid function, and L r ′ and are fused by multiplication to obtain the corrected analysis range radius.

[0093] It should be noted that in an embodiment of the present invention, the preset quantity is 100. The Euclidean distance is used to measure the distance between the noise point and the non-noise point, and obtaining the vertical distance between a point and a plane, the perpendicular intersection point, and fitting a plane based on data points are all prior arts. Specifically, the plane can be fitted by the least squares method, which will not be elaborated here.

[0094] After determining the corrected moving analysis range of the noise point, it is necessary to perform a moving correction analysis on the noise point. Therefore, within the corrected moving analysis range of each noise point, starting from the perpendicular intersection point of the fitting plane, the perpendicular intersection point is moved by the preset moving step length to obtain the noise moving analysis point of each noise point. By gradually moving the noise point, the best correction position can be found within the correction range, rather than simply pulling the noise point back to the fitting plane, which improves the correction effect.

[0095] Please refer to Figure 3 , which shows a schematic diagram of the distribution of noise moving analysis points provided by an embodiment of the present invention; Figure 3 includes the noise points represented by hollow circles, the noise moving analysis points represented by solid circles, the non-noise point fitting plane, and the annotation of the corrected analysis range radius L. The straight line between the noise point and the fitting plane is the perpendicular line of the noise point and the fitting plane, and the intersection point of the perpendicular line and the fitting plane is the perpendicular intersection point; where the preset moving step length is set to 0.2 mm.

[0096] Considering that the noise compliance can be used to characterize the noise compliance degree of three-dimensional coordinate points, after obtaining the noise movement analysis points of each noise point, the optimal correction position of the noise point can be determined according to the noise compliance of the noise movement analysis points. Therefore, according to the noise compliance of all the noise movement analysis points of each noise point, combined with the three-dimensional coordinates of the non-noise points, a corrected face point cloud is obtained, providing a more reliable basis for constructing a three-dimensional face model and improving the unlocking efficiency and accuracy.

[0097] Preferably, in an embodiment of the present invention, each noise point is moved to the noise movement analysis point with the minimum corresponding noise compliance, and combined with the three-dimensional coordinates of the non-noise points, a corrected face point cloud is obtained.

[0098] It should be noted that when obtaining the noise compliance of the noise movement analysis points of the noise points, the noise movement analysis points are used to replace the noise points, and step S3 is repeated to obtain the noise compliance of the corresponding noise movement analysis points.

[0099] Step S5: Unlock at least according to the corrected face point cloud.

[0100] Considering that due to factors such as shooting angle and occlusion, the corrected face point cloud may not pass the verification of the face recognition system, so unlock at least according to the corrected face point cloud.

[0101] Preferably, in an embodiment of the present invention, according to the pre-stored face point cloud and the corrected face point cloud in the smart phone, a face matching degree is obtained; when the face matching degree is greater than a preset face matching threshold, the phone is unlocked;

[0102] Since an embodiment of the present invention combines a face recognition system and a fingerprint recognition system for multi-modal biometric recognition, when the face matching degree is less than or equal to the preset face matching threshold, a fingerprint unlock is prompted, and the fingerprint matching degree after the prompt is obtained;

[0103] After normalizing the product of the sum value of the noise compliance of all noise points in the corrected face point cloud, the fingerprint matching degree and the face matching degree, it is used as the unlocking suitability; when the unlocking suitability is greater than the preset unlocking suitability threshold, the phone is unlocked; when the unlocking suitability is less than or equal to the preset unlocking suitability threshold, it is determined that the recognition fails, and a fingerprint unlock or a face unlock is prompted again.

[0104] As an example, the preset face matching threshold is 0.8, the preset unlocking suitability threshold is 0.7, and the calculation formula of the unlocking suitability includes:

[0105]

[0106] Among them, E represents the most recent unlocking suitability; sigmoid{} represents the sigmoid function; r represents the serial number of the noise point; R represents the number of noise points; W r ′ represents the noise matching degree of the r-th noise point in the corrected face point cloud; Fa represents the most recent face matching degree; Fi represents the most recent fingerprint matching degree. p represents the most recent fingerprint matching degree, p max represents the maximum value of the fingerprint matching degrees of all fingerprint unlockings of the smart phone.

[0107] In the calculation formula of the unlocking suitability, the sum of the noise conformities of the noise points in the corrected face point cloud represents the interference degree of the noise points on the face unlocking, so as to supplement and correct the face matching degree; the most recent fingerprint matching degree is normalized by means of the maximum value of the fingerprint matching degree to serve as the fingerprint matching degree; through the multiplication method, Fa and Fi are fused to obtain the unlocking suitability, and multi-modal biometric recognition is performed by combining face recognition and fingerprint recognition.

[0108] It should be noted that the methods for obtaining the fingerprint recognition system, the face matching degree, and the fingerprint matching degree are all prior arts. In an embodiment of the present invention, a template-based method is specifically used to obtain the face matching degree, and the Biokey algorithm is used to obtain the fingerprint matching degree, which are all prior arts and will not be elaborated here.

[0109] An embodiment of the present invention further provides a multi-modal biometric security unlocking system for a smart phone. The system includes a memory, a processor, and a computer program. The memory is used to store the corresponding computer program, the processor is used to run the corresponding computer program, and when the computer program runs in the processor, it can implement a multi-modal biometric security unlocking method for a smart phone described in steps S1-S5.

[0110] In summary, in view of the technical problem that the existing face recognition technology averages errors to the entire point cloud, resulting in unsatisfactory construction effect of the 3D face model and reduced unlocking efficiency and accuracy, the present invention provides a multi-modal biometric security unlocking method and system for a smart phone. The present invention first obtains the three-dimensional coordinate points of the face model; further screens to obtain non-consensus coordinate points; further obtains the noise compliance of each non-consensus coordinate point, and obtains noise points based on the noise compliance; further moves and corrects the noise points by means of the distribution of the non-noise points closest to the noise points, and combines the three-dimensional coordinates of the non-noise points to obtain a corrected face point cloud; finally, unlocks at least according to the corrected face point cloud. By screening out the non-consensus coordinate points collected by two cameras, the present invention uses density clustering analysis to identify outlier features to screen noise points, and moves and corrects the noise points to obtain a corrected face point cloud, providing a more reliable basis for constructing a 3D face model and improving unlocking efficiency and accuracy.

[0111] It should be noted that the above sequence of embodiments of the present invention is only for description and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the particular order or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0112] Each embodiment in this specification is described in a progressive manner, and the same or similar parts among the embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments.

Claims

1. A multi-modal biometric security unlocking method for a smart phone, characterized in that, The method includes: Obtaining three-dimensional coordinate points of a face model through a binocular structured light system; selecting any of the three-dimensional coordinate points corresponding to a camera as the coordinate points to be analyzed; Performing nearest neighbor matching on each of the coordinate points to be analyzed with the three-dimensional coordinate points corresponding to the other camera, and calculating the spatial distance between each of the coordinate points to be analyzed and the correspondingly matched three-dimensional coordinate points; screening the coordinate points to be analyzed based on the spatial distance and each distance ratio in a preset distance ratio sequence, and obtaining non-consensus coordinate points according to the change in the number of the coordinate points to be analyzed screened by adjacent distance ratios and the change in the spatial distance; Performing first density clustering on the non-consensus coordinate points to obtain first clustering clusters; selecting any of the non-consensus coordinate points as the target coordinate point; in the first clustering cluster of the target coordinate point, obtaining a radius sequence based on the maximum spatial distance between the non-consensus coordinate points, a preset radius change step size, and a preset number of spheres; in the first clustering cluster of the target coordinate point, taking the target coordinate point as the center of the sphere and each element in the radius sequence as the radius, obtaining the sphere range of the target coordinate point; performing second density clustering on the non-consensus coordinate points within each sphere range of the target coordinate point to obtain second clustering clusters and the target clustering cluster where the target coordinate point is located; obtaining the noise compliance of the target coordinate point according to the distance characteristics of all the sphere ranges, the target clustering cluster, and the other second clustering clusters, and combining the number of elements within the target clustering cluster; obtaining noise points based on the noise compliance; Obtaining a preset number of non-noise points closest to each of the noise points for fitting a plane; obtaining the corrected movement analysis range of each of the noise points according to the perpendicular distance between each of the noise points and the corresponding fitting plane, and combining the overall characteristics of the noise compliance of the obtained non-noise points; within the corrected movement analysis range of each of the noise points, starting from the intersection point of the perpendicular lines of the fitting plane, moving the intersection point of the perpendicular lines by a preset movement step size to obtain the noise movement analysis points of each of the noise points; obtaining a corrected face point cloud according to the noise compliance of all the noise movement analysis points of each of the noise points and combining the three-dimensional coordinates of the non-noise points; Unlocking at least based on the corrected face point cloud.

2. The multimodal biometric security unlocking method for a smart phone according to claim 1, characterized in that, The method for obtaining the non-consensus coordinate points includes: Taking the ratio of the spatial distance corresponding to the coordinate point to be analyzed to the maximum spatial distance as the normalized distance corresponding to the coordinate point to be analyzed; when the normalized distance of the coordinate point to be analyzed is less than or equal to the distance ratio, it is determined that the screening is passed through the corresponding distance ratio; Taking the region composed of the coordinate points to be analyzed screened by each distance ratio as the initial consensus region for each distance ratio; taking the product of the absolute value of the difference in the number of the coordinate points to be analyzed screened by adjacent distance ratios and the absolute value of the difference in the average value of the spatial distances of the coordinate points to be analyzed as the consensus accuracy of the smallest distance ratio in the corresponding adjacent distance ratios; Select the initial consensus region corresponding to the maximum consensus accuracy as the final consensus region; take the three-dimensional coordinate points outside the final consensus region as non-consensus coordinate points.

3. A multimodal biometric security unlocking method for a smartphone according to claim 1, characterized in that, The method for obtaining the radius sequence includes: In the first clustering cluster of the target coordinate point, take the maximum spatial distance between the non-consensus coordinate points as the maximum radius; based on the maximum radius, iteratively reduce the maximum radius by a preset radius change step size, obtain a new radius each time, and obtain a total of a preset number of sphere radii. Construct a radius sequence from the maximum radius and the iteratively obtained radii.

4. A multimodal biometric security unlocking method for a smart phone according to claim 1, characterized in that, The method for obtaining the noise compliance includes: Select any sphere range of the target coordinate point as the target sphere range; within the target sphere range, take the number of elements in the target clustering cluster as the first denominator, take the average value of the distances between the center point of the target clustering cluster and the center points of other secondary clustering clusters as the first numerator, and take the ratio of the first denominator to the first numerator as the noise sub-parameter of the target sphere range. Take the mean value of the noise sub-parameters of all the sphere ranges of the target coordinate point as the noise compliance of the target coordinate point.

5. A multimodal biometric security unlocking method for a smart phone according to claim 4, characterized in that, The method for obtaining the noise points includes: Mark the non-consensus coordinate points with a noise compliance greater than a preset compliance threshold as noise points.

6. A multimodal biometric security unlocking method for a smart phone according to claim 1, characterized in that, The method for obtaining the corrected movement analysis range includes: Take the product of the normalized value of the sum of the noise compliances of the non-noise points obtained for each noise point and the vertical distance of each noise point as the corrected analysis range radius for each noise point. On the perpendicular line of each noise point and the fitting plane, with the intersection point of the perpendicular line as the center, intercept two line segments with a length equal to the corrected analysis range radius on the perpendicular line as the corrected movement analysis range for each noise point.

7. A multimodal biometric security unlocking method for a smart phone according to claim 1, characterized in that, The method for obtaining the corrected face point cloud according to the noise compliance of the noise movement analysis points includes: Move each noise point to the noise movement analysis point with the minimum noise compliance corresponding to it, and combine the three-dimensional coordinates of the non-noise points to obtain the corrected face point cloud.

8. A multimodal biometric security unlocking method for a smart phone according to claim 1, characterized in that, The method for unlocking at least based on the corrected face point cloud includes: Obtain a face matching degree based on the pre-stored face point cloud in the smart phone and the corrected face point cloud; when the face matching degree is greater than a preset face matching threshold, unlock the phone. When the face matching degree is less than or equal to the preset face matching threshold, prompt for fingerprint unlocking and obtain the fingerprint matching degree after the prompt; normalize the product of the sum of the noise compliances of all noise points in the corrected face point cloud, the fingerprint matching degree, and the face matching degree as the unlocking suitability; when the unlocking suitability is greater than a preset unlocking suitability threshold, unlock the phone; when the unlocking suitability is less than or equal to the preset unlocking suitability threshold, determine that the recognition fails, and re-prompt for fingerprint unlocking or face unlocking.

9. A multi-modal biometric security unlocking method for a smart phone according to claim 1, characterized in that, The binocular structured light system includes at least a left camera, a right camera, and a projector.

10. A multimodal biometric security unlocking system for a smart phone, the system comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of a multi-modal biometric security unlocking method for a smart phone according to any one of claims 1 to 9.

Images