Method and system for video communication collaborative unlocking

Abstract

The application discloses a kind of video communication cooperative unlocking method and system, method includes: according to the event that visitor arrives in front of owner's door lock, by owner, visitor APP is sent with the authorization message including the message allowing visitor to enter door and the face number ID of owner by door lock APP, establishes visitor temporary authorization relationship;According to the requirement of on-site verification initiated by door lock, owner and visitor establish video call connection through respective APP, visitor APP extracts the face feature vector of both sides from video picture and sends to door lock by near field communication mode;According to the decryption and association query result of face feature vector and face number ID received by door lock in video picture, respectively with the face feature vector of owner stored locally and the face feature vector of visitor collected in real time are doubly compared, and identity verification result is obtained;According to the result that both double comparisons are successful, door lock executes unlocking operation.Using the embodiment of the application, the convenience of unlocking can be improved, the risk of misoperation can be eliminated, and the safety of home can be ensured.

Description

Technical Field

[0001] This invention belongs to the field of smart door lock technology, and in particular to a method and system for video communication-assisted unlocking. Background Technology

[0002] With the widespread adoption of smart locks, biometric unlocking technology, especially facial recognition, has become mainstream. Currently, homeowners typically need to pre-register their facial information with the lock to unlock it upon returning home. However, when visitors (such as relatives, friends, or delivery personnel) arrive while the homeowner is not home, traditional remote unlocking methods, such as generating temporary passwords via an app or remote control by the homeowner, suffer from drawbacks such as unintuitive identity verification and insufficient security. Relying solely on the homeowner's judgment cannot effectively confirm the visitor's identity before unlocking, potentially leading to password leaks or accidental unlocking. Furthermore, while some locks support remote video intercom, allowing homeowners to see visitors, they lack the means to deeply integrate video information with the lock's core verification mechanism, failing to achieve automated, highly reliable biometric verification. Therefore, existing solutions struggle to balance convenience and security when dealing with temporary visitor scenarios, resulting in a need for improved user experience. Summary of the Invention

[0003] The purpose of this invention is to provide a method and system for video communication-based collaborative unlocking, which addresses the shortcomings of existing technologies, improves the convenience of unlocking, effectively eliminates the risk of accidental unlocking, and ensures home security.

[0004] One embodiment of this application provides a video communication collaborative unlocking method, the method comprising:

[0005] Visitor authorization relationship establishment: Based on the event of a visitor arriving at the homeowner's door lock, the homeowner sends an authorization message containing permission for the visitor to enter and the homeowner's facial recognition ID through the door lock APP to the visitor APP, thus establishing a temporary authorization relationship with the visitor;

[0006] Video call and facial feature extraction: Based on the on-site verification requirements initiated by the door lock, the owner and the visitor establish a video call connection through their respective apps. The visitor's app extracts the facial feature vectors of both parties from the video screen and sends them to the door lock via near-field communication.

[0007] Dual facial feature comparison verification: Based on the facial feature vector and facial ID decryption and association query results of the video image received by the door lock, the facial feature vector of the owner stored locally and the facial feature vector of the visitor collected in real time are compared to obtain the identity verification result.

[0008] Door lock opening and status notification: If both comparisons are successful, the door lock will perform the unlocking operation and send an unlocking status notification to the visitor's APP and the owner's APP.

[0009] Optionally, the establishment of the visitor authorization relationship includes:

[0010] Visitor registration: Based on the visitor's download and installation of the door lock APP, the APP role is set as visitor, and a communication connection is established with the owner's APP through the owner's mobile phone number;

[0011] Initial facial feature registration and ID management: Based on the owner's first facial registration operation on the door lock, the door lock generates a facial ID number and uses a key agreed upon with the door lock APP to encrypt and store the owner's facial feature vector and the facial ID number. At the same time, the encrypted data is sent to the owner's APP for synchronous storage.

[0012] Authorization message generation: Based on the homeowner's permission to enter command triggered on the APP interface, an authorization message containing the encrypted homeowner's facial feature vector and facial ID is generated;

[0013] Near Field Authorization Transmission: Based on the authorization message received by the visitor's APP, the authorization message is transmitted to the homeowner's door lock via the NFC protocol;

[0014] Initial authentication: The door lock receives and decrypts the owner's face ID to obtain the owner's facial feature vector, which is then compared with the locally stored feature template to obtain the initial authorization verification result.

[0015] Optionally, the video call and facial feature extraction include:

[0016] On-site verification triggered: Based on the verification prompts broadcast by the door lock, the visitor's APP receives an instruction to conduct a video call verification;

[0017] Video call setup: Based on the visitor's confirmation of the video call operation in the APP interface, a video communication connection is established with the owner's APP;

[0018] Capture of the same frame: Based on the state in which the homeowner and the visitor's headshots are in the same frame during the video call, the visitor's APP automatically captures the communication screen containing the faces of both parties;

[0019] Dual feature vector extraction: Based on the captured video footage, simultaneously extract the real-time facial feature vectors of the homeowner and the visitor.

[0020] Feature data transmission: Based on the extracted dual feature vector data, it is encapsulated and transmitted to the homeowner's door lock via the NFC communication protocol.

[0021] Optionally, the dual facial feature comparison verification includes:

[0022] The first layer of comparison: owner identity verification, which involves calculating the similarity between the owner's facial feature vector extracted from the video footage by the door lock and the owner's facial feature vector stored locally, to obtain the owner's identity verification result;

[0023] The second comparison is visitor identity verification. This involves extracting the visitor's facial feature vector from the real-time captured image of the visitor's face by the front-facing camera of the door lock, and calculating the similarity between the vector and the visitor's facial feature vector extracted from the video footage to obtain the visitor identity verification result.

[0024] On-site verification determination: If both the owner's identity verification result and the visitor's identity verification result are successful, the on-site verification is deemed valid.

[0025] Optionally, the door lock opening and status notification includes:

[0026] Unlock command execution: Based on the valid on-site verification result, the door lock control mechanism executes the unlocking operation;

[0027] Status message generation: Generate an unlock status notification message based on the event indicating that the unlocking operation has been completed;

[0028] Status notification transmission: Based on the generated unlock status message, notification messages are sent to both the visitor's app and the owner's app;

[0029] Entry Confirmation Complete: Based on the unlock status notification received by the homeowner's APP, it is confirmed that the visitor has entered the home.

[0030] Optionally, the method further includes a network fault handling mechanism:

[0031] Network status detection: Based on the event of communication failure between the owner's APP and the door lock, the door lock is detected to be in a network failure state;

[0032] Backup communication activation: Based on the network failure status, activate the backup communication channel based on NFC communication between the visitor APP and the door lock;

[0033] Offline verification mode: Based on the establishment of a backup communication channel, a dual face feature comparison verification process based on near-field communication is executed;

[0034] Fault Status Notification: Depending on the duration of the network failure, an offline status notification for the door lock will be displayed on the homeowner's app interface.

[0035] Another embodiment of this application provides a video communication collaborative unlocking system, the system comprising:

[0036] Establish a module for establishing visitor authorization relationships: Based on the event of a visitor arriving at the homeowner's door lock, the homeowner sends an authorization message containing permission for the visitor to enter and the homeowner's facial recognition ID to the visitor's app through the door lock app, thus establishing a temporary authorization relationship for the visitor;

[0037] The extraction module is used for video calls and facial feature extraction: based on the on-site verification requirements initiated by the door lock, the owner and the visitor establish a video call connection through their respective APPs. The visitor's APP extracts the facial feature vectors of both parties from the video screen and sends them to the door lock through near-field communication.

[0038] The verification module is used for dual facial feature comparison verification: based on the facial feature vector and facial ID decryption and association query results of the video image received by the door lock, it performs dual comparison with the locally stored owner facial feature vector and the real-time collected visitor facial feature vector to obtain the identity verification result;

[0039] The unlocking module is used for door lock unlocking and status notification: based on the successful results of both comparisons, the door lock performs the unlocking operation and sends the unlocking status notification to the visitor's APP and the owner's APP.

[0040] Another embodiment of this application provides a storage medium storing a computer program, wherein the computer program is configured to execute the method described in any of the preceding claims when running.

[0041] Another embodiment of this application provides an electronic device including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the method described in any of the preceding claims.

[0042] Compared with existing technologies, this invention provides a video communication collaborative unlocking method. Based on the event of a visitor arriving at the homeowner's door lock, the homeowner sends an authorization message containing permission for the visitor to enter and the homeowner's facial recognition ID via the door lock's app, establishing a temporary authorization relationship. Upon the door lock's on-site verification request, the homeowner and visitor establish a video call connection via their respective apps. The visitor's app extracts facial feature vectors from the video feed and sends them to the door lock via near-field communication. Based on the decryption and association query results of the facial feature vectors and facial recognition IDs received by the door lock, a dual comparison is performed with the homeowner's locally stored facial feature vector and the real-time collected visitor's facial feature vector to obtain an identity verification result. If both comparisons are successful, the door lock performs the unlocking operation and sends an unlocking status notification to both the visitor's app and the homeowner's app. This improves unlocking convenience while effectively preventing accidental unlocking and ensuring home security. Attached Figure Description

[0043] Figure 1 A hardware structure block diagram of a computer terminal for a video communication collaborative unlocking method provided in an embodiment of the present invention;

[0044] Figure 2 A flowchart illustrating a video communication collaborative unlocking method provided in an embodiment of the present invention;

[0045] Figure 3 This is a schematic diagram of a video communication collaborative unlocking system provided in an embodiment of the present invention. Detailed Implementation

[0046] The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0047] When homeowners are not home and visiting relatives or friends need entry, or when landlords of rental properties have booked in-home housekeeping services but are unable to come to their homes during work hours, relatives, friends, and housekeeping service personnel cannot enter the home when no one is home. Existing technology, such as generating temporary passwords for visitors in advance through door lock applications, has security vulnerabilities. These passwords could be misused, or, in the event of a lost or stolen phone, malicious third parties could use them to gain entry and cause damage or financial loss. Alternatively, if the door lock's network malfunctions, homeowners cannot remotely unlock the door, inconveniencing relatives, friends, and service personnel.

[0048] This invention first provides a video communication collaborative unlocking method, which can be applied to electronic devices, such as computer terminals, specifically ordinary computers.

[0049] The following detailed explanation uses a computer terminal as an example. Figure 1 This is a hardware structure block diagram of a computer terminal for a video communication collaborative unlocking method provided in an embodiment of the present invention. (See diagram below.) Figure 1 As shown, the computer device includes a processor, memory, and network interface connected via a system bus, wherein the memory may include non-volatile storage media and internal memory.

[0050] Non-volatile storage media can store operating systems and computer programs. These computer programs include program instructions that, when executed, cause the processor to perform any video communication-based collaborative unlocking method.

[0051] The processor provides computing and control capabilities, supporting the operation of the entire computer device.

[0052] Internal memory provides an environment for the execution of computer programs in non-volatile storage media. When executed by a processor, the computer program enables the processor to perform any video communication collaborative unlocking method.

[0053] This network interface is used for network communication, such as sending assigned tasks. Those skilled in the art will understand that... Figure 1 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0054] It should be understood that the processor can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among these, a general-purpose processor can be a microprocessor or any conventional processor.

[0055] See Figure 2 The present invention provides a video communication collaborative unlocking method, which may include the following steps:

[0056] S201, Visitor Authorization Relationship Establishment: Based on the event of a visitor arriving at the homeowner's door lock, the homeowner sends an authorization message containing permission for the visitor to enter and the homeowner's facial recognition ID to the visitor's app via the door lock app, establishing a temporary visitor authorization relationship; specifically, the establishment of the visitor authorization relationship includes:

[0057] Visitor registration: Based on the visitor's download and installation of the door lock APP, the APP role is set as visitor, and a communication connection is established with the owner's APP through the owner's mobile phone number;

[0058] Initial facial feature registration and ID management: Based on the owner's first facial registration operation on the door lock, the door lock generates a facial ID number and uses a key agreed upon with the door lock APP to encrypt and store the owner's facial feature vector and the facial ID number. At the same time, the encrypted data is sent to the owner's APP for synchronous storage.

[0059] Authorization message generation: Based on the homeowner's permission to enter command triggered on the APP interface, an authorization message containing the encrypted homeowner's facial feature vector and facial ID is generated;

[0060] Near Field Authorization Transmission: Based on the authorization message received by the visitor's APP, the authorization message is transmitted to the homeowner's door lock via the NFC protocol;

[0061] Initial authentication: The door lock receives and decrypts the owner's face ID to obtain the owner's facial feature vector, which is then compared with the locally stored feature template to obtain the initial authorization verification result.

[0062] S202, Video Call and Facial Feature Extraction: Based on the on-site verification requirements initiated by the door lock, the homeowner and visitor establish a video call connection through their respective apps. The visitor's app extracts the facial feature vectors of both parties from the video feed and sends them to the door lock via near-field communication. Specifically, the video call and facial feature extraction includes:

[0063] On-site verification triggered: Based on the verification prompts broadcast by the door lock, the visitor's APP receives an instruction to conduct a video call verification;

[0064] Video call setup: Based on the visitor's confirmation of the video call operation in the APP interface, a video communication connection is established with the owner's APP;

[0065] Capture of the same frame: Based on the state in which the homeowner and the visitor's headshots are in the same frame during the video call, the visitor's APP automatically captures the communication screen containing the faces of both parties;

[0066] Dual feature vector extraction: Based on the captured video footage, simultaneously extract the real-time facial feature vectors of the homeowner and the visitor.

[0067] Feature data transmission: Based on the extracted dual feature vector data, it is encapsulated and transmitted to the homeowner's door lock via the NFC communication protocol.

[0068] S203, Dual facial feature comparison verification: Based on the decryption and association query results of the facial feature vector and face ID received from the video image by the door lock, a dual comparison is performed with the locally stored owner facial feature vector and the real-time collected visitor facial feature vector to obtain the identity verification result; specifically, the dual facial feature comparison verification includes:

[0069] The first layer of comparison: owner identity verification, which involves calculating the similarity between the owner's facial feature vector extracted from the video footage by the door lock and the owner's facial feature vector stored locally, to obtain the owner's identity verification result;

[0070] The second comparison is visitor identity verification. This involves extracting the visitor's facial feature vector from the real-time captured image of the visitor's face by the front-facing camera of the door lock, and calculating the similarity between the vector and the visitor's facial feature vector extracted from the video footage to obtain the visitor identity verification result.

[0071] On-site verification determination: If both the owner's identity verification result and the visitor's identity verification result are successful, the on-site verification is deemed valid.

[0072] S204, Door Lock Opening and Status Notification: Based on the successful results of both verifications, the door lock performs an unlocking operation and sends an unlocking status notification to both the visitor's app and the homeowner's app. Specifically, the door lock opening and status notification includes:

[0073] Unlock command execution: Based on the valid on-site verification result, the door lock control mechanism executes the unlocking operation;

[0074] Status message generation: Generate an unlock status notification message based on the event indicating that the unlocking operation has been completed;

[0075] Status notification transmission: Based on the generated unlock status message, notification messages are sent to both the visitor's app and the owner's app;

[0076] Entry Confirmation Complete: Based on the unlock status notification received by the homeowner's APP, it is confirmed that the visitor has entered the home.

[0077] Furthermore, the method also includes a network fault handling mechanism:

[0078] Network status detection: Based on the event of communication failure between the owner's APP and the door lock, the door lock is detected to be in a network failure state;

[0079] Backup communication activation: Based on the network failure status, activate the backup communication channel based on NFC communication between the visitor APP and the door lock;

[0080] Offline verification mode: Based on the establishment of a backup communication channel, a dual face feature comparison verification process based on near-field communication is executed;

[0081] Fault Status Notification: Depending on the duration of the network failure, an offline status notification for the door lock will be displayed on the homeowner's app interface.

[0082] When a visitor arrives at a homeowner's property but the homeowner is not home and the visitor needs to enter beforehand, the homeowner's door lock network malfunctions, preventing remote door opening. The homeowner and tenant communicate via video call through the door lock's app. The visitor captures a screenshot of the video call (containing both the homeowner's and visitor's faces) and sends it to the door lock via the app. The door lock then compares the facial features of the homeowner and visitor with the facial features in the video call screenshot. If a match is found, temporary authorization is granted by the homeowner's app, allowing the visitor to enter the homeowner's property. Solution:

[0083] I. Prerequisites:

[0084] The homeowner installed a smart door lock, which is equipped with a camera that can extract facial feature vectors to unlock the door for the homeowner. The smart door lock also has a mobile APP, through which the homeowner can configure the door lock. After the homeowner's family or friends install the same APP, they can use the APP to conduct video communication, view log information, photos and videos taken by the door lock, and communicate with the door lock.

[0085] II. Solution Steps:

[0086] 1: Upon initial connection, the homeowner enters their facial information.

[0087] (1) When the owner registers their face on the door lock, the door lock checks that it is the first time registering and triggers a judgment on whether the door lock APP is connected to the door lock. When connected, the door lock recognizes the owner's facial feature vector and saves it locally. At the same time, it generates a face ID for the facial feature vector and encrypts it using the key agreed with the door lock APP. After encryption, the facial feature vector and face ID are stored in ciphertext and the ciphertext facial feature vector and face ID are sent to the owner's door lock APP. The door lock APP receives and stores it locally.

[0088] 2: When a visitor arrives at the homeowner's door, the homeowner and the visitor communicate through the mobile app connected to the door lock, and the homeowner sends a message allowing the visitor to enter.

[0089] (1) A visitor arrives at the homeowner's house at the agreed time. After communicating with the homeowner, the homeowner wants to open the door for the visitor remotely. However, the homeowner finds that the door lock APP cannot connect to the homeowner's door lock remotely. After investigation, it is found that the door lock has a network failure. The homeowner and the visitor communicate by phone. The visitor downloads and installs the door lock APP on his mobile phone as suggested by the homeowner. The visitor enters his face, saves the visitor's face image, and communicates remotely with the homeowner's door lock APP through the homeowner's mobile phone number to establish a pairing relationship between the homeowner and the visitor, specifically "homeowner APP-ID and visitor APP-ID".

[0090] (2) When the visitor's mobile phone is brought close to the door lock, the door lock performs near-field communication. At this time, the door lock also discovers through its self-test program that the network cannot be connected, but there is a mobile APP nearby communicating with it via NFC. Therefore, the door lock actively sends a "Do you want to open the door?" message to the visitor's mobile APP. After receiving the message, the visitor's APP selects the desired "Owner APP-ID and Visitor APP-ID" entries according to the owner-visitor relationship table and sends a "Do you want to open the door?" message to the owner's mobile APP via the mobile Internet. This message adds the visitor's face image to the message sent by the door lock. After receiving the "Do you want to open the door?" message, the owner's APP checks whether the visitor's face in the message is indeed the friend or other person who was supposed to come today. After the owner confirms that it is correct, the owner selects "Open the door". The owner's mobile APP sends the "Allow visitor to enter" message and its own face ID (encrypted) to the visitor's door lock APP. The visitor's door lock APP sends the "Allow visitor to enter" message and the encrypted owner's face ID to the door lock via the NFC protocol.

[0091] 3: The homeowner and the visitor make a video call through the door lock APP. The door lock verifies the visitor's identity on-site and then opens the door.

[0092] (1) To prevent the misuse of the "Allow Visitors to Enter" information (such as theft, borrowing, or impersonation), the door lock also needs to further verify the visitor's identity through on-site verification. After receiving the "Allow Visitors to Enter" message and the encrypted owner's face ID, the door lock decrypts the face ID and queries the locally stored face feature vector of owner A corresponding to the face ID. Then, the door lock triggers the "On-site Verification" mode, notifies the visitor's door lock APP, and at the same time, the door lock broadcasts a voice message: "Dear guest, we now need to further verify your identity. Please follow the instructions in the mobile APP to make a video call with the owner."

[0093] (2) The visitor's door lock APP prompts the user to open a video call for on-site verification. After the visitor confirms, the visitor's APP starts calling the owner's APP for online video chat. The visitor's and owner's avatars are on the same video screen in the chat interface. After the visitor's mobile APP recognizes that there are two faces in the screen, it automatically takes a screenshot and saves it. The visitor's APP sends the screenshot to the door lock via the NFC protocol. The door lock detects two face images from the screenshot and calculates the feature vectors of the two faces through a deep network model. Then, it determines whether there is a feature vector of the two faces with a similarity better than a threshold to the face feature vector of owner A that is stored locally. If not, the subsequent actions stop. If there is, it means that one of the faces is owner A and the other is the face feature vector Z to be verified, thus triggering on-site face recognition for the visitor.

[0094] (3) The door lock announces to the visitor, "Please face the camera for facial verification." The visitor faces the door lock camera for facial recognition. The door lock recognizes and extracts the visitor's facial feature vector and compares it with the aforementioned facial feature vector Z. If the similarity is higher than the threshold, the door lock confirms that the on-site verification is valid. (The method of extracting the facial features of the owner and visitor from the chat screenshot and performing on-site verification can significantly enhance the confidence judgment of the current person to the visitor).

[0095] (4) The door lock sends a message to the visitor's door lock APP after the on-site verification is successful, and opens the door lock. After receiving the message, the visitor's door lock APP forwards the message to the owner's APP and notifies the owner that the door lock has been opened. The owner confirms that the visitor has entered the house.

[0096] Beneficial effects: When visitors need to enter the homeowner's home in advance, the system avoids the potential loss and misuse of temporary passwords by employing on-site verification through dual matching of the visitor's and homeowner's faces on the same screen, effectively enhancing the reliability of the verification. The system also solves the problem of remote authorization acquisition in the event of network failure by using near-field communication protocol interaction between the visitor lock app and the lock itself, and further strengthens the security of the solution by encrypting and verifying the homeowner's facial recognition data.

[0097] Protection Points: 1. When the door lock cannot connect to the network, it obtains remote door-opening authorization from the homeowner through the door lock app on the visitor's mobile phone. This is achieved through two stages of on-site verification (Stage 1: video call via the app to verify that both the homeowner and visitor are present in the video feed; Stage 2: facial recognition verification by the visitor at the door lock). This significantly enhances security in this scenario and solves the problem of homeowners being unable to remotely authorize access when the door lock has no network connection. 2. The door lock assigns a unique facial feature vector number to the homeowner, which is encrypted and stored both locally and in the homeowner's app. When the visitor's door lock app obtains the homeowner's authorization and the encrypted facial feature vector number, it interacts with the door lock. Once the homeowner's facial feature vector number is decrypted and verified, on-site verification is triggered. The visitor's app automatically takes a screenshot from the video for further facial recognition verification of both the homeowner and visitor.

[0098] As can be seen, based on the event of a visitor arriving at the homeowner's door lock, the homeowner sends an authorization message containing permission for the visitor to enter and the homeowner's facial recognition ID via the door lock's app, establishing a temporary authorization relationship. Upon the door lock's on-site verification request, the homeowner and visitor establish a video call connection via their respective apps. The visitor's app extracts facial feature vectors from the video feed and sends them to the door lock via near-field communication. Based on the decryption and association query results of the facial feature vectors and facial recognition IDs received by the door lock from the video feed, a dual comparison is performed with the homeowner's locally stored facial feature vector and the visitor's real-time collected facial feature vector to obtain the identity verification result. If both comparisons are successful, the door lock performs the unlocking operation and sends an unlocking status notification to both the visitor's app and the homeowner's app. This improves unlocking convenience while effectively preventing accidental unlocking and ensuring home security.

[0099] Another embodiment of the present invention provides a video communication collaborative unlocking system, see [link to relevant documentation]. Figure 3 The system may include:

[0100] Establish module 301 for establishing visitor authorization relationship: Based on the event of a visitor arriving at the owner's door lock, the owner sends an authorization message containing permission for the visitor to enter and the owner's facial recognition ID to the visitor's APP through the door lock APP, thus establishing a temporary authorization relationship for the visitor;

[0101] Extraction module 302 is used for video calls and facial feature extraction: According to the on-site verification requirements initiated by the door lock, the owner and the visitor establish a video call connection through their respective APPs. The visitor's APP extracts the facial feature vectors of both parties from the video screen and sends them to the door lock through near-field communication.

[0102] The verification module 303 is used for dual face feature comparison verification: based on the decryption and association query results of the face feature vector and face number ID of the video image received by the door lock, it is compared with the owner's face feature vector stored locally and the visitor's face feature vector collected in real time to obtain the identity verification result.

[0103] The opening module 304 is used for door lock opening and status notification: based on the successful results of both comparisons, the door lock performs the unlocking operation and sends the unlocking status notification to the visitor APP and the owner APP.

[0104] This invention also provides a storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above method embodiments when running.

[0105] This invention also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0106] Specifically, the aforementioned electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the aforementioned processor, and the input / output device is connected to the aforementioned processor.

[0107] The above description, based on the embodiments shown in the figures, details the structure, features, and effects of the present invention. The above description is only a preferred embodiment of the present invention, but the present invention is not limited to the scope of implementation shown in the figures. Any changes made in accordance with the concept of the present invention, or equivalent embodiments modified to have equivalent changes, that do not exceed the spirit covered by the specification and figures, should be within the protection scope of the present invention.

Claims

1. A method for video communication-based collaborative unlocking, characterized in that, The method includes: Visitor authorization relationship establishment: Based on the event of a visitor arriving at the homeowner's door lock, the homeowner sends an authorization message containing permission for the visitor to enter and the homeowner's facial recognition ID through the door lock APP to the visitor APP, thus establishing a temporary authorization relationship with the visitor; Video call and facial feature extraction: Based on the on-site verification requirements initiated by the door lock, the owner and the visitor establish a video call connection through their respective apps. The visitor's app extracts the facial feature vectors of both parties from the video screen and sends them to the door lock through near-field communication. Dual facial feature comparison verification: Based on the facial feature vector and facial ID decryption and association query results of the video image received by the door lock, a dual comparison is performed with the locally stored owner facial feature vector and the real-time collected visitor facial feature vector to obtain the identity verification result; Door lock opening and status notification: If both comparisons are successful, the door lock will perform the unlocking operation and send an unlocking status notification to the visitor's APP and the owner's APP.

2. The method according to claim 1, characterized in that, The establishment of the visitor authorization relationship includes: Visitor registration: Based on the visitor's download and installation of the door lock APP, the APP role is set as visitor, and a communication connection is established with the owner's APP through the owner's mobile phone number; Initial facial feature registration and ID management: Based on the owner's first facial registration operation on the door lock, the door lock generates a facial ID number and uses a key agreed upon with the door lock APP to encrypt and store the owner's facial feature vector and the facial ID number. At the same time, the encrypted data is sent to the owner's APP for synchronous storage. Authorization message generation: Based on the homeowner's permission to enter command triggered on the APP interface, an authorization message containing the encrypted homeowner's facial feature vector and facial ID is generated; Near Field Authorization Transmission: Based on the authorization message received by the visitor's APP, the authorization message is transmitted to the homeowner's door lock via the NFC protocol; Initial authentication: The door lock receives and decrypts the owner's face ID to obtain the owner's facial feature vector, which is then compared with the locally stored feature template to obtain the initial authorization verification result.

3. The method according to claim 2, characterized in that, The video call and facial feature extraction include: On-site verification triggered: Based on the verification prompts broadcast by the door lock, the visitor's APP receives an instruction to conduct a video call verification; Video call setup: Based on the visitor's confirmation of the video call operation in the APP interface, a video communication connection is established with the owner's APP; Capture of the same frame: Based on the state in which the homeowner and the visitor's headshots are in the same frame during the video call, the visitor's APP automatically captures the communication screen containing the faces of both parties; Dual feature vector extraction: Based on the captured video footage, simultaneously extract the real-time facial feature vectors of the homeowner and the visitor. Feature data transmission: Based on the extracted dual feature vector data, it is encapsulated and transmitted to the homeowner's door lock via the NFC communication protocol.

4. The method according to claim 3, characterized in that, The dual facial feature comparison verification includes: The first layer of comparison: owner identity verification, which involves calculating the similarity between the owner's facial feature vector extracted from the video footage by the door lock and the owner's facial feature vector stored locally, to obtain the owner's identity verification result; The second layer of comparison: visitor identity verification, which involves extracting the visitor's facial feature vector from the real-time captured facial image of the visitor by the front-facing camera of the door lock, and calculating the similarity between the vector and the visitor's facial feature vector extracted from the video footage to obtain the visitor identity verification result. On-site verification determination: If both the owner's identity verification result and the visitor's identity verification result are successful, the on-site verification is deemed valid.

5. The method according to claim 4, characterized in that, The door lock opening and status notification includes: Unlock command execution: Based on the valid on-site verification result, the door lock control mechanism executes the unlocking operation; Status message generation: Generate an unlock status notification message based on the event indicating that the unlocking operation has been completed; Status notification transmission: Based on the generated unlock status message, notification messages are sent to both the visitor's app and the owner's app; Entry Confirmation Complete: Based on the unlock status notification received by the homeowner's APP, it is confirmed that the visitor has entered the home.

6. The method according to claim 5, characterized in that, The method also includes a network fault handling mechanism: Network status detection: Based on the event of communication failure between the owner's APP and the door lock, the door lock is detected to be in a network failure state; Backup communication activation: Based on the network failure status, activate the backup communication channel based on NFC communication between the visitor APP and the door lock; Offline verification mode: Based on the establishment of a backup communication channel, a dual face feature comparison verification process based on near-field communication is executed; Fault Status Notification: Depending on the duration of the network failure, an offline status notification for the door lock will be displayed on the homeowner's app interface.

7. A video communication collaborative unlocking system, characterized in that, The system includes: Establish a module for establishing visitor authorization relationships: Based on the event of a visitor arriving at the homeowner's door lock, the homeowner sends an authorization message containing permission for the visitor to enter and the homeowner's facial recognition ID to the visitor's app through the door lock app, thus establishing a temporary authorization relationship for the visitor; The extraction module is used for video calls and facial feature extraction: based on the on-site verification requirements initiated by the door lock, the owner and the visitor establish a video call connection through their respective APPs. The visitor's APP extracts the facial feature vectors of both parties from the video screen and sends them to the door lock through near-field communication. The verification module is used for dual facial feature comparison verification: based on the facial feature vector and facial ID decryption and association query results of the video image received by the door lock, it performs dual comparison with the locally stored owner facial feature vector and the real-time collected visitor facial feature vector to obtain the identity verification result; The unlocking module is used for door lock unlocking and status notification: based on the successful results of both comparisons, the door lock performs the unlocking operation and sends the unlocking status notification to the visitor's APP and the owner's APP.

8. The system according to claim 7, characterized in that, The establishment module is specifically used for: Visitor registration: Based on the visitor's download and installation of the door lock APP, the APP role is set as visitor, and a communication connection is established with the owner's APP through the owner's mobile phone number; Initial facial feature registration and ID management: Based on the owner's first facial registration operation on the door lock, the door lock generates a facial ID number and uses a key agreed upon with the door lock APP to encrypt and store the owner's facial feature vector and the facial ID number. At the same time, the encrypted data is sent to the owner's APP for synchronous storage. Authorization message generation: Based on the homeowner's permission to enter command triggered on the APP interface, an authorization message containing the encrypted homeowner's facial feature vector and facial ID is generated; Near Field Authorization Transmission: Based on the authorization message received by the visitor's APP, the authorization message is transmitted to the homeowner's door lock via the NFC protocol; Initial authentication: The door lock receives and decrypts the owner's face ID to obtain the owner's facial feature vector, which is then compared with the locally stored feature template to obtain the initial authorization verification result.

9. A storage medium, characterized in that, The storage medium stores a computer program, wherein the computer program is configured to execute the method of any one of claims 1-6 when it is run.

10. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to perform the method of any one of claims 1-6.

Images