Intelligent lock fingerprint identification method and system based on multi-source feature coupling

Abstract

The application belongs to the technical field of intelligent lock identification, and discloses an intelligent lock fingerprint identification method and system based on multi-source feature coupling, which comprises collecting a fingerprint pressing signal, determining a unified triggering moment according to the force change feature in the fingerprint pressing process, collecting a face image signal in a corresponding time window, and establishing an identity triggering sequence; fingerprint detail feature points and face key feature points are extracted respectively, a cross-modal mapping relationship between the fingerprint detail feature points and the face key feature points is constructed according to the identity triggering sequence and the spatial position distribution of each feature point; the fingerprint detail feature points and the face key feature points are taken as nodes, and the cross-modal mapping relationship is taken as a connecting edge, a coupling feature graph is constructed, and the coupling feature graph is stored as a registration template; the normal unlocking and forced unlocking states are distinguished, and the security protection capability of the intelligent lock is improved.

Description

Technical Field

[0001] This invention relates to the field of smart lock recognition technology, and more specifically, to a smart lock fingerprint recognition method and system based on multi-source feature coupling. Background Technology

[0002] With the widespread application of smart locks in residential, office, and smart security scenarios, biometric identification technologies such as fingerprint and facial recognition have become the main methods of identity authentication for smart locks. Existing smart locks typically use a method of separately matching and comprehensively judging fingerprint and facial features to complete identity verification. They mainly focus on whether each biometric feature matches the registered template, and directly execute the unlocking operation after successful identity verification.

[0003] However, most existing technologies employ static template comparison methods, performing independent matching on a single biometric feature. They lack analysis of the correlation between fingerprint and facial features, and cannot verify the consistency of cross-modal identity information at the overall structural level. When local features change or the correspondence between different modalities becomes abnormal, it is difficult to accurately identify changes in the identity association structure, which can easily affect the reliability of identity verification.

[0004] Furthermore, existing smart locks typically only verify identity, lacking the ability to assess the user's current operational state. Even if authentication is successful, they cannot identify whether the user is in a state of coercion or other involuntary operation. While some solutions incorporate auxiliary information such as pressure changes or facial expressions, they often employ independent analysis or simple combinations of these features, failing to establish a coupled analysis mechanism between different biometric characteristics. This results in insufficient comprehensive utilization of abnormal shaking characteristics during pressing and facial tension, making it difficult to effectively distinguish between normal unlocking and coerced unlocking scenarios, thus impacting the overall security capabilities of the smart lock.

[0005] In view of this, the present invention proposes a smart lock fingerprint recognition method based on multi-source feature coupling to solve the above problems. Summary of the Invention

[0006] To overcome the aforementioned shortcomings of the prior art and to achieve the above objectives, the present invention provides the following technical solution: a smart lock fingerprint recognition method based on multi-source feature coupling, comprising: S1. Collect fingerprint pressing signals, determine a unified trigger time based on the force change characteristics during the fingerprint pressing process, and collect facial image signals within the corresponding time window to establish an identity trigger sequence; S2. Extract fingerprint detail points and facial key feature points respectively. Based on the identity trigger sequence and the spatial distribution of each feature point, construct the cross-modal mapping relationship between fingerprint detail points and facial key feature points. S3. Construct a coupled feature map using fingerprint detail points and facial key feature points as nodes and cross-modal mapping relationships as connecting edges, and store the coupled feature map as a registration template. S4. Call the registration template according to the coupling feature map, propagate the coupling state layer by layer along the connection edge, and perform state flip detection. Obtain the identity verification result based on the proportion of nodes covered by the propagation after state flip. S5. Call the authentication result, obtain the high-frequency jitter energy of the fingerprint pressing signal and the coordinate offset of the key facial feature points, and perform coupling calculation to obtain the coercion score; when the authentication is passed and the coercion score reaches the preset score threshold, it is determined to be coerced unlocking and a silent alarm is triggered; otherwise, the lock is unlocked normally.

[0007] Preferably, the method for determining the unified trigger time includes: The system continuously collects the pressure signal during the fingerprint pressing process and generates a pressure change sequence according to the collection time order. It calculates the pressure change between adjacent sampling times and determines the pressing stability state based on the cumulative pressure change value within a preset time window. When the pressure reaches the preset pressure and the cumulative pressure change value is less than or equal to the preset cumulative change value threshold, the corresponding sampling time is determined as the unified trigger time, and the unified trigger time is used as the time reference for face image acquisition.

[0008] Preferably, the method for establishing the identity trigger sequence includes: A time window is established using a unified trigger time as the time reference. Face image signals are continuously acquired within the time window, and the signals are sorted by time according to the acquisition time corresponding to each face image signal to generate a face image sequence. The fingerprint pressing signal corresponding to the unified trigger time is temporally associated with the face image sequence and assigned the same identity identifier to establish an identity trigger sequence.

[0009] Preferably, the method for constructing the cross-modal mapping relationship includes: Based on the identity trigger sequence, obtain the corresponding fingerprint pressing signal and face image signal, extract fingerprint detail points from the fingerprint pressing signal and face key feature points from the face image signal respectively; obtain the spatial location parameters corresponding to each fingerprint detail point and each face key feature point, determine the positional association between different feature points based on the spatial location parameters, and combine the temporal correspondence in the identity trigger sequence to map and connect fingerprint detail points and face key feature points that meet the association conditions to generate cross-modal mapping relationships.

[0010] Preferably, the method for constructing the coupled feature map includes: Fingerprint detail points and facial key feature points are used as graph nodes, and a node set is generated based on the feature attribute information of each node. The cross-modal mapping relationship is converted into connection edges between fingerprint detail point nodes and facial key feature point nodes, and a set of connection edges is generated. A coupled feature graph is constructed based on the node set and the set of connection edges, and the node information, connection edge information and corresponding coupling state information in the coupled feature graph are associated and stored to generate a registration template.

[0011] Preferably, the method for performing state flip detection includes: The system acquires real-time coupled feature maps corresponding to the collected fingerprint pressure signals and face image signals, and calls the node information, connection edge information and coupling state information in the registration template to associate the real-time coupled feature maps with the registration template. Using the coupling state in the registration template as the initial propagation state, the node coupling state is propagated layer by layer along the direction of the connection edge, and the corresponding connection edge state is updated according to the node state after propagation. The connection edge state in the real-time coupling feature map is compared with the corresponding connection edge state in the registration template, and state reversal detection is performed on the connection edge to obtain the state reversal result of the corresponding connection edge.

[0012] Preferably, the method for obtaining the authentication result includes: The set of nodes affected by the state reversal is determined based on the state reversal results of the connecting edges, and the node coverage range of the node set in the coupled feature map is counted. The proportion of nodes covered by the state reversal propagation is calculated and compared with the preset propagation coverage node proportion threshold. When the proportion of nodes covered by the state reversal propagation is less than the preset propagation coverage node proportion threshold, the authentication is deemed successful; when the proportion of nodes covered by the state reversal propagation is greater than or equal to the preset propagation coverage node proportion threshold, the authentication is deemed unsuccessful.

[0013] Preferably, the method for obtaining the stress score through coupling operations includes: The system retrieves fingerprint and facial image signals collected during the identity verification process. It extracts high-frequency jitter energy within a preset frequency range based on the force variation characteristics in the fingerprint signal. It obtains key facial feature points associated with the state of tension based on the facial image signal and calculates the coordinate offset of the key facial feature points relative to the corresponding key facial feature points in the registration template. It normalizes the high-frequency jitter energy and coordinate offset and performs a weighted coupling operation to generate the corresponding stress score.

[0014] Preferably, the method for determining coerced unlocking and triggering a silent alarm includes: Obtain the authentication result and the corresponding duress score, and compare the duress score with the preset score threshold; when the authentication result is successful and the duress score reaches the preset score threshold, determine that the current operation is a duress unlocking state, control the smart lock to perform the unlocking action and trigger a silent alarm simultaneously; When the authentication result is successful and the coercion score does not reach the preset score threshold, the current operation is determined to be a normal unlocking state, and the smart lock is controlled to perform normal unlocking; when the authentication result is unsuccessful, the smart lock remains locked.

[0015] A smart lock fingerprint recognition system based on multi-source feature coupling includes: The feature acquisition module is used to acquire fingerprint pressing signals, determine a unified trigger time based on the force change characteristics during the fingerprint pressing process, and acquire face image signals within the corresponding time window to establish an identity trigger sequence. The topology mapping module is used to extract fingerprint detail points and facial key feature points respectively, and construct a cross-modal mapping relationship between fingerprint detail points and facial key feature points based on the identity trigger sequence and the spatial distribution of each feature point. The graph construction module is used to construct a coupled feature graph using fingerprint detail points and facial key feature points as nodes and cross-modal mapping relationships as connecting edges, and to store the coupled feature graph as a registration template. The propagation verification module is used to call the registration template based on the coupling feature map, propagate the coupling state layer by layer along the connection edge, perform state flip detection, and obtain the identity verification result based on the proportion of nodes covered by the propagation after the state flip. The security linkage module is used to call the authentication result, obtain the high-frequency jitter energy of the fingerprint pressing signal and the coordinate offset of the key facial feature points, and perform coupling calculation to obtain the coercion score; when the authentication is successful and the coercion score reaches the preset score threshold, it is determined to be coerced unlocking and a silent alarm is triggered; otherwise, the lock is unlocked normally.

[0016] Compared with the prior art, the present invention has the following beneficial effects: This invention constructs a coupled feature map, using fingerprint detail points and facial key feature points as nodes, cross-modal mapping relationships as connecting edges, and starting from the coupling state of the registration template, propagating node states layer by layer along the connecting edges. This transforms identity verification from a single-point comparison to a consistency verification of the entire cross-modal feature association structure. Through the connecting edge state update mechanism, node states can cover the entire map as they propagate, reflecting the impact of local feature changes on the overall structure. When cross-modal feature correspondences change, association anomalies are amplified, providing a more comprehensive verification of structural consistency and improving recognition capabilities in complex scenarios. XOR state flip detection determines whether cross-modal association relationships have changed, converting changes in the association structure into state flip results. This verifies both the consistency of biometric features and the consistency of association relationships between biometric features, achieving quantitative detection of cross-modal association consistency, improving the ability to identify association anomalies, and enhancing the security and reliability of smart lock identity verification.

[0017] After identity verification, this invention introduces a joint analysis mechanism combining fingerprint pressing behavior features and facial state features. It couples the high-frequency jitter features of user pressing with the spatial changes of key facial feature points, determining unlocking not only based on identity matching results but also establishing a comprehensive evaluation mechanism for user operation status. This achieves coordinated judgment of identity authenticity and operational security, allowing the smart lock to identify the user's actual operational state. By extracting high-frequency jitter features of pressing intensity changes to screen for abnormal stress behaviors, and combining this with the spatial offset of key facial feature points relative to the registered state, the current user state is described from both behavioral and facial expression dimensions. This allows the two types of information to form a complementary verification coupling relationship, avoiding judgment bias caused by single features being affected by environmental or individual differences, and improving the stability and reliability of coercion state recognition. This invention constructs a unified coercion score through weighted coupling, converting two independent types of abnormal information into a comprehensive evaluation result reflecting the user's current operational state. This allows different anomalies to be measured within the same system, providing a basis for lock control strategies. Compared with existing unlocking methods that rely solely on identity verification results, this invention can effectively distinguish between legitimate users unlocking normally and those unlocking under duress, enhancing the smart lock's ability to identify abnormal unlocking scenarios and improving the overall security level. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of a fingerprint recognition method for smart locks based on multi-source feature coupling according to the present invention. Figure 2 This is a schematic diagram of the structure of a smart lock fingerprint recognition system based on multi-source feature coupling according to the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. Example

[0020] Please see Figure 1 As shown in the figure, this embodiment provides a smart lock fingerprint recognition method based on multi-source feature coupling, which specifically includes the following steps: S1. Collect fingerprint pressing signals, determine a unified trigger time based on the force change characteristics during the fingerprint pressing process, and collect facial image signals within the corresponding time window to establish an identity trigger sequence; S2. Extract fingerprint detail points and facial key feature points respectively. Based on the identity trigger sequence and the spatial distribution of each feature point, construct the cross-modal mapping relationship between fingerprint detail points and facial key feature points. S3. Construct a coupled feature map using fingerprint detail points and facial key feature points as nodes and cross-modal mapping relationships as connecting edges, and store the coupled feature map as a registration template. S4. Call the registration template according to the coupling feature map, propagate the coupling state layer by layer along the connection edge, and perform state flip detection. Obtain the identity verification result based on the proportion of nodes covered by the propagation after state flip. S5. Call the authentication result, obtain the high-frequency jitter energy of the fingerprint pressing signal and the coordinate offset of the key facial feature points, and perform coupling calculation to obtain the coercion score; when the authentication is passed and the coercion score reaches the preset score threshold, it is determined to be coerced unlocking and a silent alarm is triggered; otherwise, the lock is unlocked normally.

[0021] Methods for determining a unified trigger time include: The system continuously collects the pressure signal during the fingerprint pressing process and generates a pressure change sequence according to the collection time order; it calculates the pressure change between adjacent sampling times and determines the pressing stability state based on the cumulative pressure change value within a preset time window. Change in pressure: ;in, This indicates the change in pressure intensity between adjacent sampling times; Indicates the first Each sampling time The pressure value; Indicates the first Each sampling time The pressure value; Indicates the first Each sampling time; An index indicating the sampling time; Cumulative change in pressure intensity: ;in, Indicates the first Each sampling moment represents the cumulative change in pressure within a preset time window at the current moment. Indicates the sampling offset index within the preset time window; Indicates the number of sampling points included within the preset time window; Indicates the first The change in pressure at each sampling time; This represents the absolute value operation; When the pressure reaches the preset pressure and the cumulative change in pressure is less than or equal to the preset cumulative change threshold, the corresponding sampling time is determined as the unified trigger time, and the unified trigger time is used as the time reference for face image acquisition.

[0022] In this embodiment, it should be noted that when the pressure reaches the preset pressure and the cumulative change value of the pressure meets the preset cumulative change value threshold, it is determined that the user's finger has completed the pressing adjustment and entered a stable contact state. The current sampling time is determined as the unified trigger time, and the unified trigger time is used as the time reference for face image acquisition. The camera is started to acquire face images within the preset time window, so that the fingerprint pressing signal and the face image correspond to the same unlocking action, avoiding the inconsistency in the acquisition time of the two types of biometrics due to the continuous adjustment of the finger posture or the continuous movement of the face during the user's pressing process.

[0023] The preset pressure level is used to determine whether the finger has made effective contact with the fingerprint sensor. It is determined based on the minimum effective recognition pressure of the fingerprint sensor, so that when the pressure is reached, the fingerprint ridges can completely cover the effective collection area of ​​the sensor. The preset cumulative change value threshold is used to determine whether the pressure level has entered a stable state. It is obtained by statistically analyzing the cumulative change values ​​of the pressure level corresponding to multiple consecutive press samples during normal unlocking. The cumulative change value covering most stable press samples is selected as the preset cumulative change value threshold. When the cumulative change value of the pressure level does not exceed the threshold, the pressure level is considered to be basically stable and there is no longer any obvious pressing adjustment action.

[0024] For example, when the minimum effective recognition pressure of the fingerprint sensor is 0.8N, the preset pressure can be set to 0.8N. If the cumulative change in pressure corresponding to 5 consecutive sampling points is 0.06N, and the preset cumulative change threshold is set to 0.08N, since the current pressure has reached 0.8N and the cumulative change value of 0.06N is less than 0.08N, it is determined that the user's finger has been pressed stably. The current sampling time is determined as the unified trigger time, and face image acquisition is immediately started to ensure that the currently acquired face image and the fingerprint pressing signal come from the same identity verification process.

[0025] Methods for establishing identity trigger sequences include: A time window is established using a unified trigger time as the time reference. Face image signals are continuously acquired within the time window, and the signals are sorted by time according to the acquisition time corresponding to each face image signal to generate a face image sequence. The fingerprint pressing signal corresponding to the unified trigger time is temporally associated with the face image sequence and assigned the same identity identifier to establish an identity trigger sequence.

[0026] Specifically, using a unified trigger time as the time base, corresponding time windows are established by extending the time window forward and backward by preset time lengths. Within each time window, facial image signals are continuously acquired according to a preset sampling frame rate. The facial images are then sorted chronologically according to their corresponding acquisition times to form a facial image sequence. Using time windows for continuous acquisition, rather than acquiring only single-frame facial images, covers the entire process from the user completing the press to maintaining a stable posture, avoiding inaccurate facial images that reflect the true identity verification status due to blinking, slight head movements, or camera exposure delays.

[0027] Subsequently, the fingerprint press signal corresponding to the unified trigger time is used as the time anchor point for this authentication. The face image sequence is traversed, and according to the temporal correspondence between the acquisition time of each face image and the unified trigger time, face images belonging to the same time window are associated with their corresponding fingerprint press signals and assigned the same identity identifier, thus establishing an identity trigger sequence. The identity identifier is used to uniquely identify the fingerprint press signal and face image sequence in the same unlocking process, ensuring that the subsequently extracted fingerprint detail features and facial key features all originate from the same authentication process, providing a unified temporal basis for constructing cross-modal mapping relationships.

[0028] For example, when the unified trigger time is T1, a time window from T1−50ms to T1+150ms is established centered on T1. Within this time window, six frames of face images (G1, G2, G3, G4, G5, G6) are continuously acquired, and a face image sequence {G1, G2, G3, G4, G5, G6} is generated sequentially according to the acquisition time. Subsequently, the fingerprint pressing signal corresponding to the unified trigger time T1 and the face image sequence are jointly assigned an identity identifier ID001, establishing an identity trigger sequence {ID001, fingerprint pressing signal, G1, G2, G3, G4, G5, G6}. During subsequent identity verification, cross-modal feature mapping is only performed on data with the same identity identifier, thereby ensuring that fingerprint detail features and facial key features originate from the same unlocking action, improving the consistency and reliability of cross-modal feature coupling.

[0029] Methods for constructing cross-modal mapping relationships include: Based on the identity trigger sequence, obtain the corresponding fingerprint pressing signal and face image signal, extract fingerprint detail points from the fingerprint pressing signal and face key feature points from the face image signal respectively; obtain the spatial location parameters corresponding to each fingerprint detail point and each face key feature point, determine the positional association between different feature points based on the spatial location parameters, and combine the temporal correspondence in the identity trigger sequence to map and connect fingerprint detail points and face key feature points that meet the association conditions to generate cross-modal mapping relationships.

[0030] In this embodiment, it should be noted that the fingerprint pressing signal and the face image signal belong to the same unlocking process. Texture analysis is performed on the fingerprint image corresponding to the fingerprint pressing signal to identify changes in the fingerprint ridge structure, extract detailed feature points such as fingerprint ridge endpoints and bifurcation points, and record the position parameters of each fingerprint detail feature point in the fingerprint image coordinate system. Facial structure analysis is performed on the face image signal to extract key facial feature points corresponding to areas such as the eyes, eyebrows, nose, and mouth, and record the position parameters of each key facial feature point in the face image coordinate system.

[0031] Based on the positional parameters of fingerprint minutiae and facial key features, the positional relationships between different feature points are calculated. Since fingerprint minutiae and facial key features belong to different spatial domains, their coordinate distances are not directly compared. Instead, a relationship description is constructed based on the spatial distribution of feature points in their respective modalities. When a fingerprint minutiae and a facial key feature meet preset positional association conditions, a mapping connection is established between them. Simultaneously, based on the temporal correspondence in the identity trigger sequence, feature point combinations belonging to the same authentication process are selected to avoid erroneous associations between feature points generated by different unlocking actions. Finally, a cross-modal mapping relationship between fingerprint minutiae and facial key features is generated.

[0032] For example, during an identity verification process, the corresponding fingerprint press signal and facial image signal are obtained based on the identity trigger sequence, and a set of fingerprint detail feature points is extracted from the fingerprint image. ;in, These represent different fingerprint endpoints or bifurcation points; a set of key facial feature points is extracted from the face image. Here, represent key points for the eyes, nose, or mouth, respectively. Based on the positional distribution of each feature point and the temporal correspondence in the identity trigger sequence, the following is determined: and , and If the association conditions are met, then establish... As a cross-modal mapping relationship.

[0033] Methods for constructing coupled feature maps include: Fingerprint detail points and facial key feature points are used as graph nodes, and a node set is generated based on the feature attribute information of each node. The cross-modal mapping relationship is converted into connection edges between fingerprint detail point nodes and facial key feature point nodes, and a set of connection edges is generated. A coupled feature graph is constructed based on the node set and the set of connection edges, and the node information, connection edge information and corresponding coupling state information in the coupled feature graph are associated and stored to generate a registration template.

[0034] Specifically, based on the generated cross-modal mapping relationship, fingerprint minutiae and facial key feature points are defined as nodes in the coupled feature map, and a node set is generated based on the feature attribute information corresponding to each node. The feature attribute information of the fingerprint minutiae node includes the position parameters, orientation information, and texture type of the fingerprint minutiae; the feature attribute information of the facial key feature point node includes the position parameters and facial region information of the facial key feature point. By performing node-based processing on different types of feature points, fingerprint and facial identity features are associated and represented in the same graph structure.

[0035] Furthermore, the correspondence between fingerprint minutiae nodes and facial key feature nodes is determined based on cross-modal mapping relationships, and connection edges are established between two nodes that satisfy the mapping relationship, generating a set of connection edges. These connection edges represent the association between feature points of different modalities and record the corresponding coupling state information, which describes the association state between fingerprint minutiae nodes and facial key feature points. A coupled feature map is constructed based on the node set and the connection edge set, and the node information, connection edge information, and corresponding coupling state information in the coupled feature map are associated and stored to form a registration template. The registration template is used to save the structured association between fingerprint features and facial features formed during user registration, enabling subsequent identity verification to propagate and verify multi-source features collected in real time based on the registration template.

[0036] For example, determining fingerprint detail points. Key facial features If a mapping relationship exists, then establish a connecting edge. , used to represent and The correlation status between them; determining fingerprint minutiae. Key facial features If a mapping relationship exists, then establish a connecting edge. The aforementioned nodes and connecting edges together constitute a coupled feature graph. The feature attribute information corresponding to each node, the mapping relationship information corresponding to the connecting edges, and the coupling state information are associated and stored to form the registration template corresponding to the user.

[0037] Methods for performing state flip detection include: The system acquires real-time coupled feature maps corresponding to the collected fingerprint pressure signals and face image signals, and calls the node information, connection edge information and coupling state information in the registration template to associate the real-time coupled feature maps with the registration template. Using the coupling state in the registration template as the initial propagation state, the node coupling state is propagated layer by layer along the direction of the connection edge, and the corresponding connection edge state is updated according to the node state after propagation. Initial propagation state: ;in, Indicates the first The initial propagation state of each node; Indicates the first in the registration template The coupling state corresponding to each node; Subsequently, according to the connection relationships in the coupled feature map, the node state is propagated layer by layer along the connection edge directions. In the... During layer propagation, the state of the next layer node is updated based on the current node state and the state of the edges connected to that node: ;in, Indicates the first The first layer after propagation The status of each node; This indicates the node state after propagation from the previous layer; Represents a node With nodes The state of the connecting edges between them; This represents the computational process of associating and transmitting the coupling states of adjacent nodes and updating the current node state based on the connection edges between nodes; and Indicates the index of the node; The states of the connecting edges in the real-time coupled feature map are compared with the corresponding states of the connecting edges in the registered template. State flip detection is performed on the connecting edges to obtain the state flip results of the corresponding connecting edges.

[0038] Result of state reversal of connected edges: ;in, This indicates the result of the connection edge state reversal; This indicates the state of the connected edges under real-time verification. Indicates the status of the connection edges in the registration template; This represents the XOR operation; It should be noted that when This indicates that the state of the corresponding connection edge has not changed during the real-time verification process, meaning that the coupling relationship between the current fingerprint feature and the facial feature remains consistent; when This indicates that the state of the corresponding connection edge has been reversed, meaning that there is a difference between the association state and the registration state between the multi-source identity features collected in real time.

[0039] For example, in a registration template, fingerprint minutiae... Key facial features They are connected by edges The association has been established, and its registration status is: In the current identity verification process, the following is calculated based on the fingerprint and facial information collected in real time: ,but , indicating connecting edges A state flip has occurred. The system continues to monitor the state changes of other connected edges and records the corresponding node and propagation level of the connected edge that experienced the flip.

[0040] Methods for obtaining authentication results include: The set of nodes affected by the state reversal is determined based on the state reversal results of the connecting edges, and the node coverage range of the node set in the coupled feature map is counted. The proportion of nodes covered by the state reversal propagation is calculated and compared with the preset propagation coverage node proportion threshold. State reversal propagation coverage of node ratio: ;in, This indicates the proportion of nodes affected by state reversal propagation, that is, the proportion of the total number of nodes in the graph affected by state reversal. This represents the set of nodes affected by a state reversal, specifically the set of nodes affected by the propagation of the connecting edges that have experienced a state reversal. Represents the set of all nodes in the coupled feature map; This indicates the number of nodes in the node set affected by the state reversal; This represents the total number of nodes in the coupled feature map.

[0041] When the proportion of nodes covered by the state reversal propagation is less than the preset propagation coverage node proportion threshold, the authentication is deemed successful; when the proportion of nodes covered by the state reversal propagation is greater than or equal to the preset propagation coverage node proportion threshold, the authentication is deemed unsuccessful.

[0042] It should be noted that the preset propagation coverage node ratio threshold is used to distinguish between local state changes and overall changes in the identity association structure during normal identity verification. Its setting is determined based on the propagation coverage of normal identity samples during the registration phase. After a user completes identity registration, state propagation and state inversion detection are performed using the registration template and the real-time coupled feature map generated during multiple normal verification processes. The propagation coverage node ratio for each state inversion corresponding to each normal verification is calculated, and the distribution range of each propagation coverage node ratio under normal verification conditions is statistically analyzed.

[0043] The maximum percentage of nodes covered by state flip propagation during normal verification is used as a reference value, with a certain propagation margin reserved, to generate a preset propagation coverage node percentage threshold. When the percentage of nodes covered by state flip propagation calculated in subsequent identity verification processes is less than the preset propagation coverage node percentage threshold, it indicates that the state flip only occurs in local connectivity relationships, which is a change in association caused by normal acquisition errors, posture changes, or local feature fluctuations, and does not affect the overall identity association structure. When the percentage of nodes covered by state flip propagation reaches or exceeds the preset propagation coverage node percentage threshold, it indicates that the state flip has propagated to a large range of nodes, and the overall association relationship between the real-time coupled feature map and the registration template has changed significantly, therefore, identity verification is determined to have failed.

[0044] For example, after user registration, multiple successful verification data are collected, and the percentage of nodes covered by each state flip propagation is calculated. The highest propagation coverage percentage is 0.18. Therefore, a certain propagation margin can be reserved, and the preset propagation coverage percentage threshold can be set to 0.25. In subsequent verification, if the calculated state flip propagation coverage percentage is 0.12, it indicates that only a small number of nodes are affected by the state flip, and the overall coupling relationship remains consistent, thus the identity verification is considered successful. If the calculated state flip propagation coverage percentage is 0.31, it indicates that the state flip has spread to a large number of nodes, and the overall identity association structure differs significantly from the registration template, thus the identity verification is considered unsuccessful. This setting balances the tolerance for local fluctuations during normal operation with the ability to identify overall association changes in the event of identity anomalies.

[0045] For example, in an identity verification process, the registration template contains 100 fingerprint detail feature point nodes and facial key feature point nodes. During real-time verification, it is found through edge state reversal detection that 10 of these nodes are affected by state changes. Therefore: If the preset propagation coverage threshold is 0.3, then 0.1 is less than 0.3, indicating that the range of state changes is small, and the coupling relationship between the current fingerprint feature and facial feature has not changed significantly, thus the identity verification is deemed successful. If 40 nodes are affected by state flip propagation in another verification process, then: Since 0.4 is greater than 0.3, it indicates that the cross-modal coupling relationship has changed significantly, and the authentication is deemed unsuccessful.

[0046] Methods for obtaining stress scores through coupling operations include: The fingerprint pressing signal and facial image signal collected during the identity verification process are invoked, and high-frequency jitter energy within a preset frequency range is extracted based on the force change characteristics in the fingerprint pressing signal. High-frequency jitter energy: ;in, It represents the high-frequency jitter energy in the pressure signal, used to characterize the degree of abnormal force fluctuation caused by tension during the user's pressing process; This indicates that the change in pressure is squared, which is used to amplify larger changes in force. This represents the corresponding frequency component obtained after performing frequency analysis on the change in pressure. This indicates the preset range of frequencies for nervous hand tremors; This represents a frequency filtering function. The value is 1 when the frequency corresponding to the change is within the preset range of tense hand tremors, and 0 otherwise. Indicates that it belongs to the symbol; This indicates the lower limit of the preset jitter frequency range; This indicates the upper limit of the preset jitter frequency range; Based on the facial image signal, key facial feature points associated with the state of tension are obtained, and the coordinate offset of the key facial feature points relative to the corresponding key facial feature points in the registered template is calculated. Coordinate offset of key facial feature points: ;in, Indicates the first Position offset vector of key facial feature points; Indicates the index of key facial feature points; This indicates that among the facial images collected during the current identity verification process, the first... Location coordinates of key facial feature points; This indicates the number saved in the registration template. Location coordinates of key facial feature points; Coordinate offset of key facial feature points: ;in, Indicates the degree of change in key facial feature points; Indicates the number of key feature points related to tension; The L2 norm of the keypoint offset vector; The high-frequency jitter energy and coordinate offset are normalized and weighted coupled to generate a corresponding stress score. Stress score: ;in, Indicates the overall coercion score; This represents the weighting coefficient corresponding to the energy of fingerprint pressing and shaking; The weighting coefficients representing the degree of offset of facial key points; In this embodiment, it should be noted that since the high-frequency vibration energy of fingerprint pressing and the degree of facial key feature point offset have different numerical sources and dimensions, the two indicators are first normalized before weighted coupling to ensure they are on a unified evaluation scale. Subsequently, corresponding weight coefficients are set according to the sensitivity of the two types of features to the stress state. The weight coefficients can be pre-configured during system initialization or adjusted according to the actual application scenario, and must satisfy the following: In the specific setup, multiple sets of fingerprint pressing signals and facial image signals during normal unlocking processes are first collected. The corresponding high-frequency jitter energy and the degree of deviation of key facial feature points are calculated respectively, and the fluctuation of each indicator under normal conditions is statistically analyzed. When a certain type of feature has small fluctuations and good stability under normal conditions, it indicates that this type of feature has a high ability to distinguish abnormal states, and its corresponding weight coefficient can be appropriately increased. When a certain type of feature is easily affected by environmental changes, collection posture, or individual habits, resulting in normal fluctuations, its corresponding weight coefficient can be appropriately reduced, so that the final stress score reflects more stable features, thereby improving the reliability of stress state determination.

[0047] For example, in application scenarios with stable indoor lighting and fixed smart lock installation locations, the positioning of facial key feature points is relatively stable. However, different users may experience variations in pressure during normal pressing. In such cases, the weighting coefficient corresponding to the degree of facial key point offset can be appropriately increased, while the weighting coefficient corresponding to the fingerprint pressing jitter energy can be decreased. Conversely, in application scenarios at night or with significant lighting variations, facial images are easily affected by lighting conditions. Therefore, the weighting coefficient corresponding to the fingerprint pressing jitter energy can be appropriately increased, while the weighting coefficient corresponding to the degree of facial key point offset can be decreased. This makes the coercion scoring more consistent with the actual unlocking environment and improves the stability and adaptability of coercion state determination in different application scenarios.

[0048] Methods for determining coerced unlocking and triggering a silent alarm include: Obtain the authentication result and the corresponding duress score, and compare the duress score with the preset score threshold; when the authentication result is successful and the duress score reaches the preset score threshold, determine that the current operation is a duress unlocking state, control the smart lock to perform the unlocking action and trigger a silent alarm simultaneously; When the authentication result is successful and the coercion score does not reach the preset score threshold, the current operation is determined to be a normal unlocking state, and the smart lock is controlled to perform normal unlocking; when the authentication result is unsuccessful, the smart lock remains locked.

[0049] In this embodiment, it should be noted that the preset scoring threshold is used to determine whether the current unlocking operation is under duress. Its setting process is based on the range of duress scores generated under normal user unlocking conditions. During the smart lock initialization and registration phase, fingerprint pressure signals and facial image signals are collected during multiple normal unlocking processes. The high-frequency jitter energy of the fingerprint pressure and the offset of key facial feature points are calculated for each normal unlocking operation. Based on the coupled calculation results of the high-frequency jitter energy of the fingerprint pressure and the offset of key facial feature points, duress scores for multiple normal operating conditions are obtained. Furthermore, the duress scores corresponding to multiple normal operating conditions are statistically analyzed, and the maximum score value is used as a reference value for the normal state. A preset safety margin is then added to generate the preset scoring threshold, thereby avoiding misjudgments caused by slight finger tremors, facial posture changes, or other factors during normal user operation.

[0050] For example, during the smart lock initialization process, 20 normal unlocking attempts by the user are collected, and a coercion score is calculated for each attempt. The highest score is 0.35, and a preset safety margin is set to 0.10, resulting in a preset score threshold of 0.45. In subsequent unlocking processes, if user authentication is successful and the calculated coercion score is 0.30 (below 0.45), it indicates the current operation is within the normal range, and the smart lock performs normal unlocking. However, if user authentication is successful but the calculated coercion score is 0.60 (exceeding 0.45), it indicates that while the user's identity is genuine, there are abnormal changes in fingerprint pressing behavior and facial recognition. The system determines this as a coercive unlocking state, executes the unlocking action, and triggers a silent alarm. Through this method, the preset score threshold can be determined based on the user's own operational characteristics, improving the accuracy of coercion state judgment. Example

[0051] Please see Figure 2 As shown, parts not described in detail in this embodiment are described in Embodiment 1. A smart lock fingerprint recognition system based on multi-source feature coupling is provided, including: The feature acquisition module is used to acquire fingerprint pressing signals, determine a unified trigger time based on the force change characteristics during the fingerprint pressing process, and acquire face image signals within the corresponding time window to establish an identity trigger sequence. The topology mapping module is used to extract fingerprint detail points and facial key feature points respectively, and construct a cross-modal mapping relationship between fingerprint detail points and facial key feature points based on the identity trigger sequence and the spatial distribution of each feature point. The graph construction module is used to construct a coupled feature graph using fingerprint detail points and facial key feature points as nodes and cross-modal mapping relationships as connecting edges, and to store the coupled feature graph as a registration template. The propagation verification module is used to call the registration template based on the coupling feature map, propagate the coupling state layer by layer along the connection edge, perform state flip detection, and obtain the identity verification result based on the proportion of nodes covered by the propagation after the state flip. The security linkage module is used to retrieve the authentication result, obtain the high-frequency jitter energy of the fingerprint press signal and the coordinate offset of key facial feature points, and perform coupled calculations to obtain a coercion score. When authentication is successful and the coercion score reaches a preset threshold, it is determined to be coerced unlocking and a silent alarm is triggered; otherwise, unlocking is performed normally. The various modules are connected via wired and / or wireless means to achieve data transmission between them. Example

[0052] This embodiment discloses an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the operation mode of the smart lock fingerprint recognition method based on multi-source feature coupling provided above.

[0053] Since the electronic device described in this embodiment is the one used in implementing the smart lock fingerprint recognition method and system based on multi-source feature coupling described in this application, those skilled in the art can understand the specific implementation methods and various variations of the electronic device in this embodiment based on the smart lock fingerprint recognition method and system based on multi-source feature coupling described in this application. Therefore, how the electronic device implements the method in this application will not be described in detail here. Any electronic device used by those skilled in the art in implementing the smart lock fingerprint recognition method and system based on multi-source feature coupling described in this application falls within the scope of protection of this application.

[0054] It should be noted that all formulas in this manual are calculated by removing dimensions and taking their numerical values. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters and thresholds in the formulas are set by those skilled in the art according to the actual situation.

[0055] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A fingerprint recognition method for smart locks based on multi-source feature coupling, characterized in that, include: S1. Collect fingerprint pressing signals, determine a unified trigger time based on the force change characteristics during the fingerprint pressing process, and collect facial image signals within the corresponding time window to establish an identity trigger sequence; S2. Extract fingerprint detail points and facial key feature points respectively. Based on the identity trigger sequence and the spatial distribution of each feature point, construct the cross-modal mapping relationship between fingerprint detail points and facial key feature points. S3. Construct a coupled feature map using fingerprint detail points and facial key feature points as nodes and cross-modal mapping relationships as connecting edges, and store the coupled feature map as a registration template. S4. Call the registration template according to the coupling feature map, propagate the coupling state layer by layer along the connection edge, and perform state flip detection. Obtain the identity verification result based on the proportion of nodes covered by the propagation after state flip. S5. Call the authentication result, obtain the high-frequency jitter energy of the fingerprint pressing signal and the coordinate offset of the key facial feature points, and perform coupling calculation to obtain the coercion score; when the authentication is passed and the coercion score reaches the preset score threshold, it is determined to be coerced unlocking and a silent alarm is triggered; otherwise, the lock is unlocked normally.

2. The fingerprint recognition method for smart locks based on multi-source feature coupling according to claim 1, characterized in that, The method for determining the unified trigger time includes: The system continuously collects the pressure signal during the fingerprint pressing process and generates a pressure change sequence according to the collection time order. It calculates the pressure change between adjacent sampling times and determines the pressing stability state based on the cumulative pressure change value within a preset time window. When the pressure reaches the preset pressure and the cumulative pressure change value is less than or equal to the preset cumulative change value threshold, the corresponding sampling time is determined as the unified trigger time, and the unified trigger time is used as the time reference for face image acquisition.

3. The fingerprint recognition method for smart locks based on multi-source feature coupling according to claim 2, characterized in that, The method for establishing the identity trigger sequence includes: A time window is established using a unified trigger time as the time reference. Face image signals are continuously acquired within the time window, and the signals are sorted by time according to the acquisition time corresponding to each face image signal to generate a face image sequence. The fingerprint pressing signal corresponding to the unified trigger time is temporally associated with the face image sequence and assigned the same identity identifier to establish an identity trigger sequence.

4. The fingerprint recognition method for smart locks based on multi-source feature coupling according to claim 3, characterized in that, The method for constructing the cross-modal mapping relationship includes: Based on the identity trigger sequence, obtain the corresponding fingerprint pressing signal and face image signal, extract fingerprint detail points from the fingerprint pressing signal and face key feature points from the face image signal respectively; obtain the spatial location parameters corresponding to each fingerprint detail point and each face key feature point, determine the positional association between different feature points based on the spatial location parameters, and combine the temporal correspondence in the identity trigger sequence to map and connect fingerprint detail points and face key feature points that meet the association conditions to generate cross-modal mapping relationships.

5. The fingerprint recognition method for smart locks based on multi-source feature coupling according to claim 4, characterized in that, The method for constructing the coupled feature map includes: Fingerprint detail points and facial key feature points are used as graph nodes, and a node set is generated based on the feature attribute information of each node. The cross-modal mapping relationship is converted into connection edges between fingerprint detail point nodes and facial key feature point nodes, and a set of connection edges is generated. A coupled feature graph is constructed based on the node set and the set of connection edges, and the node information, connection edge information and corresponding coupling state information in the coupled feature graph are associated and stored to generate a registration template.

6. The fingerprint recognition method for smart locks based on multi-source feature coupling according to claim 5, characterized in that, The method for detecting state reversal includes: The system acquires real-time coupled feature maps corresponding to the fingerprint pressure signal and face image signal collected in real time, calls the node information, connection edge information and coupling state information in the registration template, and associates the real-time coupled feature maps with the registration template. Using the coupling state in the registration template as the initial propagation state, the node coupling state is propagated layer by layer along the direction of the connection edge, and the corresponding connection edge state is updated according to the node state after propagation. The connection edge state in the real-time coupling feature map is compared with the corresponding connection edge state in the registration template, and state reversal detection is performed on the connection edge to obtain the state reversal result of the corresponding connection edge.

7. The fingerprint recognition method for smart locks based on multi-source feature coupling according to claim 6, characterized in that, The methods for obtaining authentication results include: Based on the state reversal results of the connecting edges, determine the set of nodes affected by the state reversal, and count the node coverage range of the node set in the coupled feature map; calculate the proportion of nodes covered by the state reversal propagation, and compare the proportion of nodes covered by the state reversal propagation with the preset propagation coverage node proportion threshold. When the proportion of nodes covered by the state reversal propagation is less than the preset propagation coverage node proportion threshold, the authentication is deemed successful; when the proportion of nodes covered by the state reversal propagation is greater than or equal to the preset propagation coverage node proportion threshold, the authentication is deemed unsuccessful.

8. The fingerprint recognition method for smart locks based on multi-source feature coupling according to claim 7, characterized in that, The method for obtaining the stress score through coupling calculation includes: The system retrieves fingerprint and facial image signals collected during the identity verification process. It extracts high-frequency jitter energy within a preset frequency range based on the force variation characteristics in the fingerprint signal. It obtains key facial feature points associated with the state of tension based on the facial image signal and calculates the coordinate offset of the key facial feature points relative to the corresponding key facial feature points in the registration template. It normalizes the high-frequency jitter energy and coordinate offset and performs a weighted coupling operation to generate the corresponding stress score.

9. The fingerprint recognition method for smart locks based on multi-source feature coupling according to claim 8, characterized in that, The methods for determining coerced unlocking and triggering a silent alarm include: Obtain the authentication result and the corresponding duress score, and compare the duress score with the preset score threshold; when the authentication result is successful and the duress score reaches the preset score threshold, determine that the current operation is a duress unlocking state, control the smart lock to perform the unlocking action and trigger a silent alarm simultaneously; When the authentication result is successful and the coercion score does not reach the preset score threshold, the current operation is determined to be a normal unlocking state, and the smart lock is controlled to perform normal unlocking; when the authentication result is unsuccessful, the smart lock remains locked.

10. A smart lock fingerprint recognition system based on multi-source feature coupling, used to implement the smart lock fingerprint recognition method based on multi-source feature coupling as described in any one of claims 1 to 9, characterized in that, include: The feature acquisition module is used to acquire fingerprint pressing signals, determine a unified trigger time based on the force change characteristics during the fingerprint pressing process, and acquire face image signals within the corresponding time window to establish an identity trigger sequence. The topology mapping module is used to extract fingerprint detail points and facial key feature points respectively, and construct a cross-modal mapping relationship between fingerprint detail points and facial key feature points based on the identity trigger sequence and the spatial distribution of each feature point. The graph construction module is used to construct a coupled feature graph using fingerprint detail feature points and facial key feature points as nodes and cross-modal mapping relationships as connecting edges, and to store the coupled feature graph as a registration template. The propagation verification module is used to call the registration template based on the coupling feature map, propagate the coupling state layer by layer along the connection edge, perform state flip detection, and obtain the identity verification result based on the proportion of nodes covered by the propagation after the state flip. The security linkage module is used to call the authentication result, obtain the high-frequency jitter energy of the fingerprint pressing signal and the coordinate offset of the key facial feature points, and perform coupling calculation to obtain the coercion score; when the authentication is successful and the coercion score reaches the preset score threshold, it is determined to be coerced unlocking and a silent alarm is triggered; otherwise, the lock is unlocked normally.

Images