Smart lock control system based on a motion data and method thereof
Patent Information
- Application Number
- TW114104872
- Authority / Receiving Office
- TW · TW
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2025-02-10
- Publication Date
- 2026-08-16
- Estimated Expiration
- 2045-02-09
AI Technical Summary
Current smart locks rely on passwords, fingerprints, or card swipes for security, which are inflexible and have hygiene and lifespan issues due to frequent touch-based interactions.
A smart lock control system that uses motion data, capturing and recognizing specific user gestures through a camera module, verification module, and motion capture module to securely unlock the lock.
Enhances security and hygiene by allowing touch-free operation and extending the lifespan of smart locks through gesture recognition.
Smart Images
Figure TWG2TA001072173_001 
Figure TWG2TA001072173_002 
Figure TWG2TA001072173_003
Abstract
Description
[Technical Field]
[0001] This invention relates to the technical field of smart lock control, and more particularly to a smart lock control system and method based on motion data. [Previous Technology]
[0002] Smart locks are commonly used items in modern life to ensure the security of homes or stored items. However, current smart locks mainly rely on passwords, fingerprints, or card swipes to open. While they provide a certain level of security, their flexibility is relatively insufficient. In addition, because current smart locks are frequently opened by touch, they also have hygiene and short lifespan issues.
[0003] Therefore, there is an urgent need in the present technology for a control system and method for opening smart locks based on motion data (such as gestures) to improve the problems existing in the prior art. [Summary of the Invention]
[0004] The purpose of this invention is to provide a smart lock control system based on motion data, so as to realize the technology of securely opening smart locks by capturing and recognizing specific control actions of users.
[0005] To achieve the above-mentioned objectives, the present invention provides a smart lock control system based on motion data, comprising: a camera module configured to capture user verification data and motion data; a verification module connected to the camera module to receive the verification data, the verification module executing a verification procedure based on the verification data to generate verification information; and a motion capture module connected to the verification module and the camera module, wherein when the motion capture module receives the verification information indicating that the verification result is correct, the motion capture module performs the action of receiving the motion data from the camera module, causing the motion capture module to execute a feature analysis procedure based on the motion data. The system acquires a plurality of user key points and sets action data based on each user key point to generate an action feature. When the verification module executes the verification procedure and generates verification information indicating a match, the verification module sends the verification information to the motion capture module. A comparison module, connected to the motion capture module, receives the action feature. The comparison module compares the action feature based on at least one control action feature, and when the comparison module determines that the action feature matches the control action feature, the comparison module generates unlocking information. A smart lock, connected to the comparison module, receives the unlocking information and executes an unlocking procedure.
[0006] Preferably, the motion data includes a preset motion data and a comparison motion data. When the motion capture module executes the feature analysis program, the motion capture module detects the skeletal points of a basic user image in the preset motion data based on deep learning to generate a keypoint model. Then, based on the multiple keypoint positions in the keypoint model, it selects multiple endpoint positions of the user as multiple first keypoints. The motion capture module then obtains at least one dynamic user image from the preset motion data, and based on the motion capture module comparing the image change position of the basic user image and the dynamic user image and the movement relationship between at least two of the first keypoints in the image change position, it confirms the other multiple keypoint positions in the keypoint model as multiple second keypoints. The motion capture module generates multiple user keypoints based on the multiple first keypoints and the multiple second keypoints. The motion capture module sets the comparison motion data based on each user keypoint and generates the motion feature.
[0007] Preferably, the motion data includes a preset motion data and a comparison motion data. When the motion capture module executes the feature analysis program, the motion capture module obtains a basic user image from the preset motion data. The motion capture module detects the skeletal points of the basic user image in the preset motion data based on deep learning to generate a keypoint model. The basic user image is set as a plurality of first keypoints according to the keypoint model. The motion capture module then obtains at least one dynamic user image from the preset motion data. The dynamic user image is compared with an image change position according to a preset motion model of the keypoint model and a correction data is generated. The motion capture module corrects at least one of the first keypoints in the image change position according to the correction data to serve as a second keypoint. The motion capture module generates a plurality of user keypoints based on the plurality of first keypoints and the plurality of second keypoints. The motion capture module sets the comparison motion data according to each user keypoint and generates the motion feature. The preset motion model corresponds to the motion in the dynamic user image.
[0008] Preferably, when the comparison module compares the action feature based on at least one control action feature, the comparison module determines a plurality of reference key points in the control action feature that correspond to each of the user key points, then compares the movement amount of each user key point in the comparison action data with the movement amount of each reference key point, and determines an error value between the movement amount of each user key point and the movement amount of the corresponding reference key points based on an allowance table as a comparison result, and determines whether the action feature conforms to the control action feature based on the comparison result.
[0009] Preferably, when the comparison module determines an error value between the movement amount of each user key point and the movement amount of the corresponding reference key points based on a tolerance table as a comparison result, the comparison module determines whether the error value of each user key point falls within a tolerance range according to the tolerance table. If the error value of each user key point falls within the tolerance range, the comparison module outputs the comparison result that the action feature conforms to the control action feature, and generates the unlocking information based on the comparison result. If the error value of any user key point is not within the tolerance range, the comparison module outputs the comparison result that does not conform. The tolerance range is set according to a part classification.
[0010] Preferably, when the verification module performs a verification procedure based on the verification data, the verification module performs a liveness detection procedure, and when it determines that the verification data conforms to a liveness benchmark, the verification module performs an identification procedure to judge the verification data based on at least one recorded user information, and when the user information matches the verification data, the verification module generates verification information indicating that the verification result is compliant, and when the user information does not match the verification data, the verification module generates verification information indicating that the verification result is non-compliant; wherein, when the verification data does not conform to the liveness benchmark, the verification module stops the verification procedure.
[0011] Preferably, the verification data includes an infrared image and a visible light image. When the verification module executes the liveness detection procedure, the verification module determines a plurality of liveness detection positions in the infrared image and compares these liveness detection positions with the corresponding image positions in the visible light image. When the comparison result meets the liveness benchmark, the verification module executes the recognition procedure.
[0012] Preferably, the verification data includes a depth image. When the verification module executes the liveness detection procedure, the verification module analyzes the depth information of the depth image based on geometric features. When the analysis result meets the liveness benchmark, the verification module executes the recognition procedure.
[0013] Preferably, the smart lock control system includes: a custom module connected to the comparison module, the custom module generating or storing the control action feature and sending it to the comparison module; wherein the control action feature includes an action template or an action sequence.
[0014] To achieve the above-mentioned objectives, the present invention also provides a smart lock control method applied to the smart lock control system described above, comprising: A camera module captures user verification data and motion data; a verification module executes a verification procedure based on the verification data to generate verification information; when a motion capture module receives the verification information indicating a successful verification, the motion capture module performs the action of receiving the motion data from the camera module, causing the motion capture module to execute a feature analysis procedure based on the motion data to obtain a plurality of user key points, and sets the motion data according to each user key point to generate a motion feature; when the verification module executes the verification procedure and generates the verification information indicating a successful verification, the verification module sends the verification information to the motion capture module; a comparison module compares the motion feature based on at least one control motion feature, and when the comparison module determines that the motion feature matches the control motion feature, the comparison module generates unlocking information; and when a smart lock receives the unlocking information, the smart lock executes an unlocking procedure.
[0015] In order to make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments listed below with reference to the figures are described in detail below.
Implementation Method
[0016] The advantages, features and technical methods of the present invention will be more readily understood by referring to the exemplary embodiments and the accompanying drawings. The present invention may be implemented in different forms and should not be construed as being limited to the embodiments set forth herein. Rather, the embodiments provided will make this disclosure more thorough, complete and fully convey the scope of the invention to those skilled in the art. The present invention will be defined only as provided in the appended claims.
[0017] In addition, the terms "comprising" and / or "including" refer to the presence of the said features, areas, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, areas, wholes, steps, operations, elements, components and / or combinations thereof.
[0018] To facilitate your understanding of the content of this invention and the effects it can achieve, the following detailed description is provided in conjunction with the accompanying drawings of various specific embodiments:
[0019] Please refer to Figures 1 to 5, which are schematic diagrams of the configuration relationship of each module of the smart lock control system of the present invention, a schematic diagram of selecting the user's endpoint position as the first key point in one embodiment of the basic user image, a schematic diagram of other multiple key point positions as the second key point in one embodiment, a schematic diagram of setting the basic user image as the first key point in another embodiment of the key point model, and a schematic diagram of image change position correction and the first key point corresponding to the image change position as the second key point in another embodiment. As shown in the figures, in order to realize the technology of securely opening the smart lock by capturing and recognizing the user's specific control actions, the smart lock control system based on motion data of the present invention includes a camera module 100, a verification module 200, a motion capture module 300, a comparison module 400, and a smart lock 500.
[0020] The camera module 100 is a camera device used to capture user images. In another embodiment, the camera module 100 may also be a camera device supporting infrared sensing technology (e.g., an infrared camera) or depth sensing technology (e.g., a Time of Flight (TOF) camera or a binocular camera). The camera module 100 is configured to capture a verification data 10 and a motion data 11 of the user. The verification data 10 refers to image data that at least includes information used by the verification module 200 to verify and confirm the user's identity (e.g., a user whose verification features (e.g., facial features) have been registered with the verification module 200). The motion data 11 refers to data that at least includes an image of the user (e.g., a hand image). Thus, when the smart lock control system is activated, the user is within the capture range of the camera module 100, allowing the camera module 100 to capture the verification data 10 and the motion data 11.
[0021] The verification module 200 is used to determine whether the currently acquired image is of a living person and whether it is the user himself. Therefore, when the verification module 200 is connected to the camera module 100 to receive the verification data 10, the verification module 200 can execute a verification procedure based on the verification data 10 to generate verification information 20.
[0022] However, when the verification module 200 executes the verification procedure, the verification module 200 can execute a liveness detection procedure, and when it is determined that the verification data 10 meets a liveness benchmark, the verification module 200 executes an identification procedure to judge the verification data 10 based on at least one user information 21 recorded (e.g., user information that has recorded verification features into the verification module 200). When the user information 21 matches the verification data 10, the verification module 200 generates verification information 20 with a verification result of compliance, and the verification module 200 can send the verification information 20 to the motion capture module 300. When the user information 21 does not match the verification data 10, the verification module 200 generates verification information 20 with a verification result of non-compliance, and the verification module 200 does not perform any subsequent actions. For example, when the verification data 10 is a face image, the verification module 200 uses the liveness detection program to determine that the face image meets the liveness criteria (i.e., determines that the face image is live). Then, the recognition program is executed to determine the face image based on the recorded user information (i.e., face recognition related information). If it matches, a matching result is generated; otherwise, a non-matching result is generated.
[0023] Furthermore, if the verification module 200 executes the liveness detection procedure and determines that the verification data 10 does not meet the liveness benchmark, the verification module 200 may stop the verification procedure and no longer perform the subsequent identification procedure.
[0024] The motion capture module 300 is a processor used to analyze user key points and generate corresponding motion features. The motion capture module 300 is connected to the verification module 200 and the camera module 100. When the motion capture module 300 receives verification information from the verification module 200 indicating that the verification result is correct, the motion capture module 300 can perform a feature analysis program based on the motion data 11 sent by the camera module 100 to obtain a plurality of user key points 30, and set the motion data 11 according to each user key point 30 to generate a motion feature 31.
[0025] The motion data 11 includes a preset motion data 111 and a comparison motion data 112. The preset motion data 111 indicates that the user presents a corresponding motion posture (such as a gesture with all five fingers spread) according to a preset posture template, so that the camera module 100 can acquire the preset motion data 111 of the user's motion posture corresponding to the preset posture template. The comparison motion data 112 indicates that the user presents a corresponding motion posture (i.e., a gesture that matches the control action feature) according to at least one set control action.
[0026] When the motion capture module 300 performs a feature analysis program based on the motion data 11, please refer to Figures 2 and 3. In one embodiment, the motion capture module 300 obtains a basic user image 1111 from the preset motion data 111. The basic user image 1111 is an image of the user presenting a corresponding action posture according to a preset posture template. The motion capture module 300 can detect the skeletal points of the basic user image 1111 in the preset motion data 111 based on deep learning (e.g., PoseNet, OpenPose, or MediaPipe) to generate a key point model 32. Then, based on the key point model 32, a plurality of endpoint positions of the user are selected as a plurality of first key points 321. It should be noted that when the key point model 32 is a key point model of the hand, the endpoint positions include fingertip positions, finger-to-palm connection positions, and palm-to-arm connection positions, etc.
[0027] Subsequently, the motion capture module 300 acquires a plurality of dynamic user images 1112 from the preset motion data 111, and determines the positions of other plurality of key points in the key point model 32 as a plurality of second key points 322 based on an image change position 11121 of the dynamic user image 1112 and the movement relationship between at least two of the first key points 321 in the image change position 11121. The dynamic user image 1112 represents an action different from that in the basic user image 1111, such as a clenched fist, a tiger claw, or a small heart gesture, so that the motion capture module 300 can compare the image change position 11121 between the basic user image 1111 and the dynamic user image 1112. When the dynamic user image 1112 represents a tiger claw action, the movement relationship between each of the first key points 321 in the image change position 11121 can be represented as the movement relationship between the first key point 321 at the fingertip position and the first key point 321 at the finger-palm connection position. Since the dynamic user image 1112 is a preset posture template with an action different from that in the basic user image 1111, when the first key point 321 at the fingertip position moves relative to the first key point 321 at the finger-palm connection position... When the finger movement is confirmed, the preset movement relationship can be confirmed (for example, the movement trajectory of the fingertip position and the connection position between the finger and the palm present a triangle RT movement relationship). Thus, based on the movement relationship between the first key point 321 of the fingertip position and the first key point 321 of the connection position between the finger and the palm in the image change position 11121, the finger joint key point position in the key point model 32 can be confirmed as the second key point 322. When confirming the finger joint key point position in the key point model 32, it means that the preset finger joint key point position in the key point model 32 is determined or modified (when there is a deviation between the preset finger joint key point position in the key point model 32 and the confirmed finger joint key point position in the key point model 32, the preset finger joint key point position can be modified). The finger joint key point position in the key point model 32 is then used as the second key point 322.
[0028] Thus, the motion capture module 300 can generate a plurality of user key points 30 based on a plurality of the first key points 321 and a plurality of the second key points 322, and set the comparison motion data 112 based on each user key point 30, and generate the motion feature 31. That is to say, after setting the comparison motion data 112 based on the plurality of user key points 30, the subsequent input comparison motion data 112 can determine the position of each key point of the user based on the plurality of user key points 30, and generate the motion feature 31 accordingly.
[0029] When the motion capture module 300 performs a feature analysis program based on the motion data 11, please refer to Figures 4 and 5. In another embodiment, after obtaining the key point model 32 as described above, the motion capture module 300 can set the basic user image 1111 as the plurality of first key points 321 according to the position of each key point in the key point model 32. Then, it can obtain the plurality of dynamic user images 1112 from the preset motion data 111, and compare the image change position 11121 of the dynamic user image 1112 with a preset motion model of the key point model 32 and generate a correction data 33. The motion capture module 300 corrects at least one of the first key points 321 corresponding to the image change position 11121 according to the correction data 33 as a second key point 322. The preset motion model corresponds to the motion in the dynamic user image 1112. That is, the preset motion model is a preset motion pattern (e.g., the fingertip position, finger-to-palm connection position, and finger joint position corresponding to a tiger claw gesture). When acquiring the dynamic user image 1112 corresponding to the user's tiger claw gesture, the motion capture module 300 compares the preset motion model with the image change position 11121 of the dynamic user image 1112 (i.e., the fingertip position, finger-to-palm connection position, and finger joint position in the dynamic user image 1112) and generates correction data 33. If the correction data 33 indicates a deviation between any key point in the preset motion model and the first key point 321 (e.g., the first key point 321 of the finger joint position) at the corresponding image change position 11121, the motion capture module 300 can correct at least one of the first key points 321 corresponding to the image change position 11121 based on the correction data 33. (i.e., the first key point 321 that corrects the position of the finger joint) is used as the second key point 322.
[0030] Subsequently, the motion capture module 300 can generate the plurality of user key points 30 based on the plurality of the first key points 321 and the plurality of the second key points 322. The motion capture module 300 sets the comparison motion data 112 based on each of the user key points 30 and generates the motion feature 31.
[0031] The comparison module 400 is a processor used to compare data and determine control action features that match the action features based on the comparison results. The comparison module 400 is connected to the motion capture module 300 to receive the action feature 31, so that the comparison module 400 can compare the action feature 31 based on at least one control action feature 40, and when the comparison module 400 determines that the action feature 31 matches the control action feature 40, the comparison module 400 generates unlocking information 41.
[0032] The motion capture module 300 and the comparison module 400 may be application-specific integrated circuits (ASICs) or graphics processing units (GPUs) to achieve low latency and high performance real-time feedback.
[0033] Furthermore, the motion capture module 300 can also be connected to a cloud device to output the user key point model of each user key point 30 to the cloud device, so that the user key point model can be stored and updated through the cloud device, and processed by the edge computing device to realize the functions of real-time motion processing feedback and long-term data analysis.
[0034] The smart lock 500 is a locking device used for locking and can wirelessly receive unlocking information. When the smart lock 500 is connected to the comparison module 400 and receives the unlocking information, the smart lock 500 can execute an unlocking procedure to open the locked object (such as a door or a safe).
[0035] Each time the smart lock 500 executes the unlocking procedure, a recording module (not shown) can be connected to the smart lock 500 to record the user, date, specific time, etc. of each unlocking procedure, so as to provide tracking and query of each unlocking behavior.
[0036] Please refer to Figure 6 again, which is a schematic diagram of the present invention comparing the movement amount of the user's key point with the movement amount of the reference key point to determine the error value of the user's key point based on the tolerance table. As shown in the figure, the comparison result described above can be determined by the comparison module 400 based on the multiple reference key points 401 corresponding to each user key point 30 in the control action feature 40. The movement amount of each user key point 30 in the comparison action data 112 is compared with the movement amount of each corresponding reference key point 401. An error value E between the movement amount of each user key point 30 and the movement amount of each corresponding reference key point 401 is determined based on an allowance table and used as the comparison result. Since the user can make a gesture similar to the control action feature 40 (e.g., five fingers spread or tiger claw) when making the corresponding control gesture, the specific gesture posture may still be different from the gesture posture recorded by the control action feature 40. However, if the user is required to make a gesture posture that is completely consistent with the gesture posture recorded by the control action feature 40, it may cause the user to be unable to effectively generate the unlocking information 41.
[0037] Thus, the tolerance table can be used to determine whether the error value E is within a tolerance range AR. The tolerance range AR can be given different tolerance range values depending on the key point. For example, the key point located at the fingertip is given a tolerance range AR of ±2 cm, and the key point located at the connection between the finger and the palm is given a tolerance range AR of ±1 cm, etc. If the user key point 30 is within the allowable range AR, then the user key point 30 is a valid user key point 30. That is, when the comparison module 400 compares the movement amount of each user key point 30 in the comparison action data 112 with the movement amount of each reference key point 401, after obtaining the movement amount of each user key point 30 (e.g., from five fingers fully open to a tiger claw shape), the comparison module 400 compares the movement amount of the plurality of reference key points 401 in each control action feature 40 (e.g., from five fingers fully open to different shapes of each control action feature 40, one of which can be from five fingers fully open to a tiger claw shape) to determine whether the action feature 31 conforms to the control action feature 40.
[0038] The comparison module 400 can use Convolutional Neural Networks (CNN), Long Short-Term Memory (LSTM) and / or Dynamic Time Warping (DTW) to compare the temporal and image features of each user keypoint 30 and each reference keypoint 401 to determine the movement amount of each user keypoint 30 and each reference keypoint 401 in the comparison action data 112. When the movement amount of any user keypoint 30 has an error with the movement amount of its corresponding reference keypoint 401, the error value E can be generated accordingly, and the error value E can be determined based on the tolerance table and used as the comparison result.
[0039] Accordingly, when the comparison module 400 determines whether the error value E of each user key point 30 falls within the allowable range AR according to the allowable table, if the error value E of each user key point 30 falls within the allowable range AR, the comparison module 400 outputs the comparison result that conforms to the control action feature 40, so as to determine that the action feature 31 conforms to the control action feature 40 based on the comparison result. If the error value E of any user key point 30 is not within the allowable range AR, the comparison module 400 outputs the comparison result that the action feature 31 does not conform to the control action feature 40.
[0040] The allowable range AR can be set according to a part classification. For example, when the user's key point 30 is the key point of the second joint of the middle finger, since the swing amplitude of the middle finger is large, the allowable range AR can be given a larger allowable range AR. Or when the user's key point 30 is the key point of the first joint of the thumb, since the swing amplitude of the thumb is small, the allowable range AR can be given a smaller allowable range AR, etc.
[0041] Please refer to Figures 7 and 8 again, which are schematic diagrams of the configuration relationship of the verification module containing infrared images and visible light images, and the configuration relationship of the verification module containing depth images, respectively. As shown in the figures, when the camera module 100 is a binocular camera (including an infrared camera and a visible light camera), such that the captured verification data 10 includes an infrared image 101 and a visible light image 102, when the verification module 200 executes the liveness detection program, the verification module 200 determines a plurality of liveness judgment positions in the infrared image 101, and compares these liveness judgment positions with the image positions in the visible light image 102 that correspond to these liveness judgment positions. When the comparison result meets the liveness benchmark, the verification module 200 executes the recognition program.
[0042] Among them, the liveness detection positions include the nose tip position, cheek position, and eye position of the face. When the verification module 200 determines the plurality of liveness detection positions in the infrared image 101, the face will have obvious point light source traces under the illumination of infrared light. For example, the nose tip will reflect light because it is closer to the screen, the cheek position will be darker, and the real human eye will have pupil reflection and grayish white of the eye. The visible light image 102 will not have these features. Thus, the verification module 200 can compare the liveness detection positions in the infrared image 101 with the corresponding positions in the visible light image 102 to determine whether they meet the liveness benchmark. The liveness benchmark also meets the above-mentioned features such as nose tip reflection, cheek position darker, pupil reflection, and grayish white of the eye. If they meet the benchmark, the face image is determined to be live, and the subsequent recognition procedure is executed.
[0043] When the camera module 100 includes a depth camera (e.g., a Time-of-Flight (TOF) camera), such that the captured verification data 10 includes a depth image 103, when the verification module 200 executes the liveness detection procedure, the verification module 200 analyzes the depth information 1031 of the depth image 103 based on geometric features. When the analysis result matches the liveness benchmark, the verification module 200 executes the recognition procedure. In one embodiment, when the verification module 200 analyzes the depth information 1031 of the depth image 103 based on geometric features, the verification module 200 can use the geometric structure information of the human face (e.g., the relative positions between the eyes, nose, and mouth of the face, as well as their distances and angles) to analyze the facial key points of the depth information 1031, and use it to determine whether it is a real human face (i.e., whether it matches the liveness benchmark). If it matches, it is determined to be a live person, and then the subsequent recognition procedure is executed.
[0044] Please refer to Figure 9 again, which is a schematic diagram of the configuration relationship of the custom module of the present invention. As shown in the figure, the smart lock control system disclosed in the present invention may include a custom module 600, which is connected to the comparison module 400 and is used to provide the user with custom unlocking actions as the control action feature 40, so that the custom module 600 generates or stores the control action feature 40 and sends it to the comparison module 400. The control action feature 40 may include an action template or an action sequence. The action template may be represented as a static posture image, such as a fist, a tiger claw, or a heart gesture. The action sequence may be represented as a dynamic image within a time period (e.g., within three seconds), such as a dynamic gesture from fully open fingers to a fist, from a fist to a tiger claw, or from a thumbs-up to a heart gesture.
[0045] Please refer to Figure 10 again, which is a flowchart of the steps of the smart lock control method of the present invention. As shown in the figure, in order to realize the technology of securely opening the smart lock by capturing and recognizing the user's specific control actions, the present invention also provides a smart lock control method applied to the smart lock control system described above, which includes:
[0046] S101: Capture user verification data and motion data using a camera module;
[0047] S102: Use the verification module to execute a verification procedure based on the verification data to generate verification information;
[0048] S103: When the motion capture module receives verification information indicating that the verification result is compliant, the motion capture module performs the action of receiving motion data from the camera module, so that the motion capture module performs a feature analysis program based on the motion data to obtain a plurality of user key points, and sets motion data according to each user key point to generate motion features. When the verification module executes the verification program and generates verification information indicating that the verification result is compliant, the verification module sends the verification information to the motion capture module.
[0049] S104: The comparison module compares the action features based on at least one control action feature, and when the comparison module determines that the action feature matches the control action feature, the comparison module generates unlocking information; and
[0050] S105: When the smart lock receives unlocking information, the smart lock executes the unlocking procedure.
[0051] The present invention discloses a preferred embodiment. Any partial changes or modifications that are derived from the technical concept of the present invention and can be easily deduced by those skilled in the art are not outside the scope of the patent rights of the present invention.
[0052] In summary, the present invention, in terms of purpose, means and effects, demonstrates technical features that are distinct from those of the conventional, and its invention is practical and meets all the requirements for a patent. We respectfully request that your review committee examine the invention and grant a patent as soon as possible so that it may benefit society. We would be truly grateful for your assistance. [Simplified Explanation of the Diagram]
[0053] Figure 1 is a schematic diagram of the configuration relationship of each module of the smart lock control system of the present invention; Figure 2 is a schematic diagram of selecting the endpoint position of the user as the first key point in the basic user image in one embodiment of the present invention; Figure 3 is a schematic diagram of other multiple key point positions as the second key point in one embodiment of the present invention; Figure 4 is a schematic diagram of setting the basic user image as the first key point in the key point model in another embodiment of the present invention; Figure 5 is a schematic diagram of image change position correction and the first key point corresponding to the image change position as the second key point in another embodiment of the present invention; Figure 6 is a schematic diagram of the present invention comparing the movement amount of the user key point with the movement amount of the reference key point to determine the error value of the user key point based on the tolerance table; Figure 7 is a schematic diagram of the configuration relationship of the verification module in the present invention, where the verification data includes infrared images and visible light images; Figure 8 is a schematic diagram of the configuration relationship of the verification module in the present invention, where the verification data includes depth images; Figure 9 is a schematic diagram of the configuration relationship of the custom module in the present invention; Figure 10 is a flowchart of the steps of the smart lock control method of the present invention.
Claims
1. A smart lock control system based on motion data, comprising: a camera module configured to capture user verification data and motion data; a verification module connected to the camera module to receive the verification data, the verification module executing a verification procedure based on the verification data to generate verification information; and a motion capture module connected to the verification module and the camera module, wherein when the motion capture module receives the verification information indicating a valid verification result, the motion capture module executes the action of receiving the motion data from the camera module, causing the motion capture module to execute a feature analysis procedure based on the motion data to obtain a plurality of user key points, and sets the motion data according to each user key point to generate a motion feature, wherein... When the verification module executes the verification procedure and generates verification information indicating compliance, the verification module sends the verification information to the motion capture module; a comparison module, connected to the motion capture module to receive the motion feature, compares the motion feature based on at least one control motion feature, and when the comparison module determines that the motion feature matches the control motion feature, the comparison module generates unlocking information; and a smart lock, connected to the comparison module to receive the unlocking information and execute an unlocking procedure. The motion data includes a preset motion data and a comparison motion data. When the motion capture module executes the feature analysis program, it detects the skeletal points of a basic user image in the preset motion data based on deep learning to generate a keypoint model. Then, it selects a plurality of endpoint positions of the user as a plurality of first keypoints based on the positions of a plurality of keypoints in the keypoint model. The motion capture module then obtains at least one dynamic user image from the preset motion data and, based on the comparison of the image change position between the basic user image and the dynamic user image and the movement relationship between at least two of the first keypoints in the image change position, confirms the positions of other plurality of keypoints in the keypoint model as a plurality of second keypoints. The motion capture module generates a plurality of user keypoints based on the plurality of first keypoints and the plurality of second keypoints. The motion capture module sets the comparison motion data based on each user keypoint and generates the motion feature.
2. A smart lock control system based on motion data, comprising: a camera module configured to capture user verification data and motion data; a verification module connected to the camera module to receive the verification data, the verification module executing a verification procedure based on the verification data to generate verification information; and a motion capture module connected to the verification module and the camera module, wherein when the motion capture module receives the verification information indicating a valid verification result, the motion capture module executes the action of receiving the motion data from the camera module, causing the motion capture module to execute a feature analysis procedure based on the motion data to obtain a plurality of user key points, and sets the motion data according to each user key point to generate a motion feature, wherein... When the verification module executes the verification procedure and generates verification information indicating compliance, the verification module sends the verification information to the motion capture module; a comparison module, connected to the motion capture module to receive the motion feature, compares the motion feature based on at least one control motion feature, and when the comparison module determines that the motion feature matches the control motion feature, the comparison module generates unlocking information; and a smart lock, connected to the comparison module to receive the unlocking information and execute an unlocking procedure. The motion data includes a preset motion data and a comparison motion data. When the motion capture module executes the feature analysis program, it acquires a basic user image from the preset motion data. Based on deep learning, the motion capture module detects the skeletal points of the basic user image in the preset motion data to generate a keypoint model. The basic user image is then set as a plurality of first keypoints according to the keypoint model. The motion capture module then acquires at least one dynamic user image from the preset motion data. Based on a preset motion model of the keypoint model, it compares the dynamic user image with an image change position and generates correction data. The motion capture module corrects at least one of the first keypoints in the image change position according to the correction data to serve as a second keypoint. The motion capture module generates a plurality of user keypoints based on the plurality of first keypoints and the plurality of second keypoints. The motion capture module sets the comparison motion data based on each user keypoint and generates the motion feature. The preset motion model corresponds to the motion in the dynamic user image.
3. The smart lock control system as described in claim 1 or 2, wherein, When the comparison module compares the action feature based on at least one control action feature, the comparison module determines a plurality of reference key points in the control action feature that correspond to each user key point, and then compares the movement amount of each user key point in the comparison action data with the movement amount of each reference key point. Based on an allowance table, the module determines an error value between the movement amount of each user key point and the movement amount of the corresponding reference key points as a comparison result, and determines whether the action feature conforms to the control action feature based on the comparison result.
4. The smart lock control system as described in claim 3, wherein, When the comparison module determines the error value between the movement amount of each user key point and the movement amount of the corresponding reference key points based on an allowance table as a comparison result, the comparison module determines whether the error value of each user key point falls within an allowable range according to the allowance table. If the error value of each user key point falls within the allowable range, the comparison module outputs the comparison result that the action feature matches the control action feature, and generates the unlocking information based on the comparison result. If the error value of any user key point is not within the allowable range, the comparison module outputs the comparison result that does not match. The allowable range is set according to a part classification.
5. The smart lock control system as described in claim 1 or 2, wherein, When the verification module performs a verification procedure based on the verification data, it performs a liveness detection procedure. Upon determining that the verification data conforms to a liveness benchmark, the verification module performs an identification procedure to judge the verification data based on at least one recorded user information. When the user information matches the verification data, the verification module generates verification information indicating compliance. When the user information does not match the verification data, the verification module generates verification information indicating non-compliance. When the verification data does not conform to the liveness benchmark, the verification module stops the verification procedure.
6. The smart lock control system as described in claim 5, wherein, The verification data includes an infrared image and a visible light image. When the verification module executes the liveness detection procedure, the verification module determines a plurality of liveness detection positions in the infrared image and compares these liveness detection positions with the corresponding image positions in the visible light image. When the comparison result meets the liveness benchmark, the verification module executes the recognition procedure.
7. The smart lock control system as described in claim 5, wherein, The verification data includes a depth image. When the verification module executes the liveness detection procedure, the verification module analyzes the depth information of the depth image based on geometric features. When the analysis result matches the liveness benchmark, the verification module executes the recognition procedure.
8. The smart lock control system as described in claim 1 or 2, comprising: A custom module is connected to the comparison module. The custom module generates or stores the control action feature and sends it to the comparison module. The control action feature includes an action template or an action sequence.
9. A smart lock control method applied to a smart lock control system as described in any one of claims 1 to 8, comprising: capturing user verification data and motion data using a camera module; executing a verification procedure based on the verification data using a verification module to generate verification information; when a motion capture module receives the verification information indicating that the verification result is compliant, the motion capture module executes the action of receiving the motion data from the camera module, causing the motion capture module to execute a feature analysis procedure based on the motion data to obtain a plurality of user key points, and setting the motion data according to each user key point to generate a motion feature, wherein... When the verification module executes the verification procedure and generates verification information indicating compliance, the verification module sends the verification information to the motion capture module; a comparison module compares the motion feature based on at least one control motion feature, and when the comparison module determines that the motion feature matches the control motion feature, the comparison module generates unlocking information; and when a smart lock receives the unlocking information, the smart lock executes an unlocking procedure.