A door opening method and system in cooperation with a door lock

Abstract

The application discloses a door opening method and system cooperating with a door lock, and the method comprises the following steps: according to the detection result of a multimedia authorization device on an indoor authorized person, reporting the authorized person identifier to a gateway device through an enhanced multicast protocol message, and establishing a binding relationship between the authorized person identifier and an interface in a multicast forwarding table by the gateway device; according to the collection result of a door lock device on a visitor biological feature, sending a multicast media stream containing the visitor biological feature to a multicast network; according to the reception of the multicast media stream by the gateway device, extracting the visitor biological feature and querying the authorized association relationship, and executing a differentiated forwarding strategy based on the query result; and according to the presentation of the forwarding content and the recognition result of the authorized action of the multimedia authorization device, completing authorization confirmation and triggering a door lock opening operation. By using the embodiment of the application, the reliability, instantaneity and flexibility of the door opening authorization process can be improved, and the safety of user biological feature data is ensured.

Description

Technical Field

[0001] This invention belongs to the field of smart door lock technology, and in particular to a door lock collaborative opening method and system. Background Technology

[0002] Existing smart lock systems typically rely on centralized cloud servers or a single mobile application for remote authorization to unlock doors. When a visitor arrives, the lock device uploads images and other information to the cloud, which then pushes the data to the homeowner's mobile app. The homeowner confirms the request and sends an unlock command. This approach has several significant drawbacks: First, it heavily depends on internet connectivity and cloud service stability; the process completely fails when the network is poor or the cloud service is interrupted. Second, the authorization path is singular and fixed, lacking flexibility; for example, it cannot intelligently select the most suitable authorization device based on the real-time location and status of family members (the authorizer) (e.g., in front of the living room TV). Finally, the entire process from information collection and cloud transmission to mobile confirmation has high latency, affecting the immediacy of the unlocking experience. Summary of the Invention

[0003] The purpose of this invention is to provide a door unlocking method and system that coordinates door locks to overcome the shortcomings of the prior art, improve the reliability, immediacy and flexibility of the door unlocking authorization process, and at the same time ensure the security of user biometric data.

[0004] One embodiment of this application provides a door unlocking method with lock coordination, the method comprising:

[0005] Authorized Person Location Awareness and Registration: Based on the detection results of the authorized persons indoors by the multimedia authorization device, the authorized person's identifier is reported to the gateway device through an enhanced multicast protocol message. The gateway device then establishes a binding relationship between the authorized person's identifier and the interface in the multicast forwarding table.

[0006] Visitor information collection and distribution: Based on the results of the door lock device collecting the visitor's biometrics, a multicast media stream containing the visitor's biometrics is sent to the multicast network;

[0007] Intelligent routing and authorization decision-making: Based on the gateway device's reception of the multicast media stream, extract visitor biometrics and query authorization associations, and execute differentiated forwarding strategies based on the query results;

[0008] Authorization confirmation and door opening execution: Based on the presentation of forwarded content and the recognition results of authorization actions by the multimedia authorization device, authorization confirmation is completed and the door lock opening operation is triggered.

[0009] Optionally, the authorized person's location awareness and registration includes:

[0010] Device registration: Based on the multimedia authorized device startup event, send a device registration message to the preset multicast address to establish a multicast forwarding path;

[0011] Authorized Person Detection: Based on the continuous monitoring of indoor personnel by multimedia authorization devices, capture the best facial images and extract biometric features;

[0012] Identity verification: Based on the comparison results between the extracted biometric features and the locally pre-stored set of authorized person features, the authorized person identifier is determined;

[0013] Location reporting: Based on the identified authorized person identifier, report the presence information of the authorized person to the gateway device via enhanced multicast protocol messages;

[0014] Binding relationship establishment: Based on the gateway device's parsing of the received packets, the authorized person identifier is registered on the corresponding interface of the multicast forwarding table to form an interface-authorized person binding relationship.

[0015] Optionally, the visitor information collection and distribution includes:

[0016] Door lock registration: Based on the door lock device startup event, send a door lock device registration message to the preset multicast address to establish a multicast path from the door lock to the aggregation point;

[0017] Visitor feature acquisition: Based on the trigger event of a visitor arriving at the door lock, the optimal facial image of the visitor is acquired and converted into a biometric feature vector;

[0018] Media stream encapsulation: Based on the extracted visitor biometric vector, it is embedded into the first message of the multicast media stream and sent.

[0019] Network distribution: Based on the established multicast path, the multicast media stream containing visitor characteristics is distributed to each gateway device.

[0020] Optionally, the intelligent routing and authorization decision includes:

[0021] Feature extraction and query: Extract the embedded visitor biometric vectors from the multicast media stream received by the gateway device;

[0022] Relationship determination: Based on the extracted feature vector, query the local authorization relationship table to determine whether a corresponding authorization relationship exists;

[0023] Initial forwarding process: Based on the determination that there is no authorization association, the multicast media stream is forwarded to all connected multimedia authorization devices;

[0024] Targeted forwarding processing: Based on the judgment result that there is an authorization relationship and the authorizer is online, the multicast media stream is forwarded only to the interface where the corresponding authorizer is located;

[0025] Waiting for forwarding: Based on the judgment result that there is an authorized association relationship but the authorizer is offline, a time-limited waiting mechanism is started. If the association relationship synchronization is received from other gateways during this period, the media stream is discarded; otherwise, it is processed as the initial forwarding after the timeout.

[0026] Optionally, the authorization confirmation and door opening execution include:

[0027] Media presentation and motion capture: Based on the multicast media stream received by the multimedia licensing device, decode it and present it on the display device, while capturing the viewer's biometrics and preset authorized actions through the associated camera;

[0028] Local authorization verification: Based on the matching results of the captured biometric features and the local authorized person features, as well as the recognition results of the preset authorization actions, authorization confirmation is completed;

[0029] Door opening command transmission: Based on the authorization confirmation result, the multimedia authorization device directly sends the door opening command to the door lock device;

[0030] Association Reporting: Based on the authorization confirmation result, the multimedia authorization device reports the visitor-authorizer association to the gateway device through the enhanced multicast protocol;

[0031] Relationship synchronization: Based on the association relationships received by the gateway device, it is synchronized to other gateway devices via a dedicated multicast address.

[0032] Optionally, the method further includes anomaly recovery processing:

[0033] Status query trigger: Based on network reset or management needs, the gateway device sends a coordination information query request to all ports;

[0034] Device status re-reporting: Based on the received query request, the door lock device re-reports the device registration information, and the multimedia authorization device re-reports the device registration information, the information of the authorized person, and the established authorization associations;

[0035] Table Reconstruction: Based on the various types of newly received reported information, the gateway device reconstructs the local multicast forwarding table and authorization association table;

[0036] Relationship resynchronization: Based on the rebuilt authorized association table, the gateway device resynchronizes to other gateway devices via a dedicated multicast address.

[0037] Another embodiment of this application provides a door lock-coordinated door opening system, the system comprising:

[0038] The sensing module is used for authorizing the location and registration of authorized persons: based on the detection results of authorized persons indoors by the multimedia authorization device, the authorized person's identifier is reported to the gateway device through enhanced multicast protocol messages, and the gateway device establishes a binding relationship between the authorized person's identifier and the interface in the multicast forwarding table;

[0039] The acquisition module is used for visitor information collection and distribution: based on the biometric data collected by the door lock device, the multicast media stream containing the visitor's biometric data is sent to the multicast network;

[0040] The decision module is used for intelligent routing and authorization decisions: based on the gateway device's reception of the multicast media stream, it extracts visitor biometrics and queries authorization relationships, and executes differentiated forwarding strategies based on the query results;

[0041] The determination module is used for authorization confirmation and door opening execution: based on the multimedia authorization device's presentation of forwarded content and recognition of authorization actions, it completes authorization confirmation and triggers the door lock opening operation.

[0042] Another embodiment of this application provides a storage medium storing a computer program, wherein the computer program is configured to execute the method described in any of the preceding claims when running.

[0043] Another embodiment of this application provides an electronic device including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the method described in any of the preceding claims.

[0044] Compared with existing technologies, the present invention provides a door-lock collaborative unlocking method. Based on the detection results of the authorized person indoors by the multimedia authorization device, the authorized person's identifier is reported to the gateway device via an enhanced multicast protocol message. The gateway device then establishes a binding relationship between the authorized person's identifier and the interface in the multicast forwarding table. Based on the door lock device's collection of visitor biometric features, a multicast media stream containing the visitor's biometric features is sent to the multicast network. Based on the gateway device's reception of the multicast media stream, visitor biometric features are extracted, and authorization associations are queried. A differentiated forwarding strategy is executed based on the query results. Based on the multimedia authorization device's presentation of the forwarded content and the recognition results of the authorization action, authorization confirmation is completed and the door lock unlocking operation is triggered. This improves the reliability, immediacy, and flexibility of the door-opening authorization process while ensuring the security of user biometric data. Attached Figure Description

[0045] Figure 1 A hardware structure block diagram of a computer terminal for a door lock-coordinated door opening method provided in an embodiment of the present invention;

[0046] Figure 2A schematic flowchart illustrating a door-lock coordinated door-opening method provided in an embodiment of the present invention;

[0047] Figure 3 This is an example diagram of a door lock collaboration relationship provided in an embodiment of the present invention;

[0048] Figure 4 This is a schematic diagram of a door lock-coordinated door opening system provided in an embodiment of the present invention. Detailed Implementation

[0049] The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0050] Duplex rooms in hotels or apartments often consist of an open area and multiple separate rooms. The open area is shared by the room owner and external visitors, with the room owner authorizing entry for each visitor. Because visitors are temporary and cannot be pre-registered for the external door locks, when a visitor arrives unexpectedly, the owner needs to leave their room to unlock the external door, impacting the user experience for both the owner and the visitor. Door locks send alarms point-to-point to the administrator's mobile app, but this method is often affected by network latency, the phone not being readily available, and the administrator's response timeliness, resulting in a poor user experience for visitors. Therefore, deploying a separate management platform within the network is too costly.

[0051] This invention first provides a door-lock coordinated door opening method, which can be applied to electronic devices, such as computer terminals, specifically ordinary computers.

[0052] The following detailed explanation uses a computer terminal as an example. Figure 1 This is a hardware structure block diagram of a computer terminal for a door-lock coordinated door-opening method provided in an embodiment of the present invention. (See diagram below.) Figure 1 As shown, the computer device includes a processor, memory, and network interface connected via a system bus, wherein the memory may include non-volatile storage media and internal memory.

[0053] Non-volatile storage media can store operating systems and computer programs. These computer programs include program instructions that, when executed, cause the processor to perform any door-locking cooperative unlocking method.

[0054] The processor provides computing and control capabilities, supporting the operation of the entire computer device.

[0055] Internal memory provides an environment for the execution of computer programs in non-volatile storage media. When executed by a processor, the computer program enables the processor to perform any door lock cooperative opening method.

[0056] This network interface is used for network communication, such as sending assigned tasks. Those skilled in the art will understand that... Figure 1 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0057] It should be understood that the processor can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among these, a general-purpose processor can be a microprocessor or any conventional processor.

[0058] See Figure 2 The present invention provides a door unlocking method in cooperation with a door lock, which may include the following steps:

[0059] S201, Authorized Person Location Awareness and Registration: Based on the detection results of the authorized person indoors by the multimedia authorization device, the authorized person's identifier is reported to the gateway device via an enhanced multicast protocol message. The gateway device then establishes a binding relationship between the authorized person's identifier and the interface in the multicast forwarding table. Specifically, the authorized person location awareness and registration includes:

[0060] Device registration: Based on the multimedia authorized device startup event, send a device registration message to the preset multicast address to establish a multicast forwarding path;

[0061] Authorized Person Detection: Based on the continuous monitoring of indoor personnel by multimedia authorization devices, capture the best facial images and extract biometric features;

[0062] Identity verification: Based on the comparison results between the extracted biometric features and the locally pre-stored set of authorized person features, the authorized person identifier is determined;

[0063] Location reporting: Based on the identified authorized person identifier, report the presence information of the authorized person to the gateway device via enhanced multicast protocol messages;

[0064] Binding relationship establishment: Based on the gateway device's parsing of the received packets, the authorized person identifier is registered on the corresponding interface of the multicast forwarding table to form an interface-authorized person binding relationship.

[0065] S202, Visitor Information Collection and Distribution: Based on the results of the door lock device's collection of visitor biometrics, a multicast media stream containing the visitor's biometrics is sent to the multicast network; specifically, the visitor information collection and distribution includes:

[0066] Door lock registration: Based on the door lock device startup event, send a door lock device registration message to the preset multicast address to establish a multicast path from the door lock to the aggregation point;

[0067] Visitor feature acquisition: Based on the trigger event of a visitor arriving at the door lock, the optimal facial image of the visitor is acquired and converted into a biometric feature vector;

[0068] Media stream encapsulation: Based on the extracted visitor biometric vector, it is embedded into the first message of the multicast media stream and sent.

[0069] Network distribution: Based on the established multicast path, the multicast media stream containing visitor characteristics is distributed to each gateway device.

[0070] S203, Intelligent Routing and Authorization Decision: Based on the gateway device's reception of the multicast media stream, extract visitor biometrics and query authorization relationships, and execute differentiated forwarding strategies based on the query results; specifically, the intelligent routing and authorization decision includes:

[0071] Feature extraction and query: Extract the embedded visitor biometric vectors from the multicast media stream received by the gateway device;

[0072] Relationship determination: Based on the extracted feature vector, query the local authorization relationship table to determine whether a corresponding authorization relationship exists;

[0073] Initial forwarding process: Based on the determination that there is no authorization association, the multicast media stream is forwarded to all connected multimedia authorization devices;

[0074] Targeted forwarding processing: Based on the judgment result that there is an authorization relationship and the authorizer is online, the multicast media stream is forwarded only to the interface where the corresponding authorizer is located;

[0075] Waiting for forwarding: Based on the judgment result that there is an authorized association relationship but the authorizer is offline, a time-limited waiting mechanism is started. If the association relationship synchronization is received from other gateways during this period, the media stream is discarded; otherwise, it is processed as the initial forwarding after the timeout.

[0076] S204, Authorization Confirmation and Door Opening Execution: Based on the multimedia authorization device's presentation of the forwarded content and the recognition result of the authorization action, authorization confirmation is completed and the door lock opening operation is triggered. Specifically, the authorization confirmation and door opening execution includes:

[0077] Media presentation and motion capture: Based on the multicast media stream received by the multimedia licensing device, decode it and present it on the display device, while capturing the viewer's biometrics and preset authorized actions through the associated camera;

[0078] Local authorization verification: Based on the matching results of the captured biometric features and the local authorized person features, as well as the recognition results of the preset authorization actions, authorization confirmation is completed;

[0079] Door opening command transmission: Based on the authorization confirmation result, the multimedia authorization device directly sends the door opening command to the door lock device;

[0080] Association Reporting: Based on the authorization confirmation result, the multimedia authorization device reports the visitor-authorizer association to the gateway device through the enhanced multicast protocol;

[0081] Relationship synchronization: Based on the association relationships received by the gateway device, it is synchronized to other gateway devices via a dedicated multicast address.

[0082] Furthermore, the method also includes anomaly recovery processing:

[0083] Status query trigger: Based on network reset or management needs, the gateway device sends a coordination information query request to all ports;

[0084] Device status re-reporting: Based on the received query request, the door lock device re-reports the device registration information, and the multimedia authorization device re-reports the device registration information, the information of the authorized person, and the established authorization associations;

[0085] Table Reconstruction: Based on the various types of newly received reported information, the gateway device reconstructs the local multicast forwarding table and authorization association table;

[0086] Relationship resynchronization: Based on the rebuilt authorized association table, the gateway device resynchronizes to other gateway devices via a dedicated multicast address.

[0087] This invention proposes a solution for real-time location sensing of authorized users by door locks and surrounding multimedia authorization devices (smart TVs, set-top boxes, video surveillance, video intercoms, etc.) without a management platform. This allows for autonomous collaboration between the door lock and the multimedia authorization devices, facilitating convenient authorization and improving visitor access efficiency and user experience. For ease of description, set-top boxes will be used as a typical type of multimedia authorization device in the following description.

[0088] I. Technical Solution:

[0089] 1. Core idea:

[0090] The set-top box has a built-in set of feature vectors for all authorized users. When the set-top box detects events such as the appearance, departure, or addition of a target (referring to people in the room) in the room, it compares the facial feature vector of the captured face image with the local set of authorized user feature vectors. After confirming the authorized user's identity, it reports the information to the set-top box gateway via an enhanced IGMP join message. The set-top box gateway then registers the authorized user ID on the interface that receives IGMP messages.

[0091] When a visitor arrives, the door lock captures the visitor's facial feature vector and embeds it into the multicast video stream before sending it. The door lock gateway and the set-top box gateway establish an RPT forwarding path to the RP through a bidirectional PIM mechanism. Therefore, after receiving the multicast video stream sent by the door lock, the door lock gateway forwards it to the set-top box gateway.

[0092] After receiving the multicast stream, the set-top box gateway extracts the visitor feature vector from the multicast stream, queries the local authorization association, and completes the forwarding operation.

[0093] 1) Initially, if no authorization association exists, the video stream is sent to all directly connected set-top boxes for authorization confirmation. The set-top box determines that authorization actions such as nodding originate from locally stored authorized facial feature values ​​in the video stream captured by the connected camera, thus confirming successful authorization. Upon successful authorization, the set-top box directly sends a unicast message to the door lock to unlock the door, and simultaneously reports the authorization association to the set-top box gateway (which includes the authorized person ID and visitor facial feature values) via an IGMP message. The set-top box gateway then synchronizes the authorization association between the authorized person and the visitor via a separate multicast address.

[0094] 2) If an authorization relationship exists, the corresponding authorizing person ID is compared with the set of authorizing persons bound to the local interface one by one. If a match is found, the video stream is sent to the corresponding interface for the set-top box to confirm and unlock.

[0095] 3) If an authorization association exists, but the face feature vector set registered on the local interface is not found based on the corresponding authorization ID, the video stream will be in a 10-second authorization waiting state, and the set-top box gateway will not forward the multicast stream. If, within the 10-second authorization waiting state, the set-top box gateway receives a multicast message of authorization association synchronized by other set-top box gateways, and the authorizing person ID in the authorization association in the message is the same as the authorizing person ID to which the video stream waiting to be forwarded is to be forwarded, it means that other set-top boxes have forwarded the video stream to the authorizing person and completed the authorizing person confirmation, and the local set-top box gateway will discard the forwarding. If no authorization association synchronized by other set-top boxes is received within the 10-second authorization waiting period, the set-top box gateway will treat it as if no authorization association exists, that is, send the multicast stream to all set-top boxes.

[0096] 2. The complete technical implementation process is as follows:

[0097] (1) Pre-set conditions:

[0098] a. Door locks and set-top boxes can capture facial images and convert them into facial feature extraction and feature value comparison based on algorithms. Furthermore, the feature values ​​of door locks and set-top boxes are compatible and mutually recognized.

[0099] b. The set-top box obtains the authorized person's image of the door lock through other management methods and stores a set of characteristic vectors of all authorized persons. All set-top boxes keep the content synchronized.

[0100] Table 1. Authorized Person Information Stored in the Set-Top Box

[0101] ;

[0102] c. The set-top box can receive audio and video multicast messages, decode them and display them on the screen, and can also recognize actions such as nodding in the video feed of people in the camera connected to the set-top box;

[0103] d. The router can provide facial feature comparison services on demand;

[0104] e. Figure 3 In the network topology, routers F, E, and A are the door lock gateways for door locks A, B, and C, respectively; routers B, C, and D are the set-top box gateways for set-top boxes A, B, and C, respectively; and router G is the RP for the entire network.

[0105] (2) The synchronization of information within the network after the set-top box captures a person's image can be completed in the following steps:

[0106] ① The set-top box sends an IGMPv2 “Set-top box device registration” message. The Type field adds an enumeration value of 0x18, the GroupAddress is filled with the multicast address G1 (e.g., 225.1.1.1), and all other fields are left empty. The destination address of the IP layer is filled with G1, and the source address is the set-top box IP address.

[0107] ②The set-top box gateway establishes the RPT forwarding path from the set-top box gateway to the RP by creating (*, G1) entries along the way through the bidirectional PIM mechanism.

[0108] ③ After the set-top box detects that someone is in the room, it tracks all targets to complete the optimal face capture and compares it with the authorized person feature value in the locally stored authorized person information (Table 1) to determine the authorized person ID;

[0109] ④ The set-top box sends an IGMPv2 "Authorization Present" message. The Type field adds an enumeration value of 0x1A, the Group Address is filled with the streaming multicast address G1 (e.g., 225.1.1.1), the Auth Num is filled with the number of authorized users reported, the Auth ID is filled with the confirmed authorized user IDs, the Aux Data Num is filled with 0, the IP layer destination address is filled with G1, and the source address is the set-top box's IP address. For example... Figure 3 As shown, Master 1 and Master 2 appear on set-top box A, and set-top box A sends an IGMPv2 "Authorized Appearance" message containing AuthID-1 and AuthID-2; Master 3 appears on set-top box B, and set-top box B sends an IGMPv2 "Authorized Appearance" message containing AuthID-3. The IGMPv2 enhanced message format is shown in Table 2.

[0110] Table 2 IGMPv2 Enhanced Message Format

[0111] ;

[0112] ⑤ After the set-top box gateway router detects the IGMPv2 "Authorized Present" message, it registers the authorizing person ID included in the message on the receiving interface that receives the IGMP message, forming a forwarding table for multicast data. For example... Figure 3 As shown in Table 3, the multicast forwarding table entries maintained by router B are expanded as shown in Table 4.

[0113] Table 3. Cooperative Interface Binding Table Maintained by Router B

[0114] ;

[0115] Table 4. Cooperative Interface Binding Table Maintained by Router C

[0116] ;

[0117] ⑥ If the set-top box detects the target leaving, it reports an IGMPv2 "Authorized Departure" message, adding an enumeration value of 0x1B to the Type field, and filling in the authorized person ID of the departing entity with the Auxiliary Data; for example... Figure 3 As shown, when owner 1 leaves set-top box A, set-top box A sends an IGMPv2 "Authorized to Leave" message with AuthID-1 as the Auth ID; when owner 3 leaves set-top box B, set-top box B sends an IGMPv2 "Authorized to Leave" message with AuthID-3 as the Auth ID.

[0118] ⑦ The set-top box gateway searches the collaborative interface binding table and deletes the departing authorized person ID on the downlink interface (set-top box);

[0119] (3) The synchronization and forwarding of visitor information captured by the door lock device within the network can be completed in the following steps:

[0120] ① The door lock sends an IGMPv2 "Door Lock Device Registration" message. The Type field adds an enumeration value of 0x19, the GroupAddress is filled with the multicast address G1 (e.g., 225.1.1.1), and all other fields are left empty. The destination address of the IP layer is filled with G1, and the source address is the door lock's IP address.

[0121] ② The door lock gateway uses a bidirectional PIM mechanism to form (*, G1) entries along the way, as well as the RPT forwarding path from the door lock gateway to the RP.

[0122] ③ When visitor A appears in front of door lock A, the door lock extracts the optimal face capture of the visitor, converts it into a feature vector VstFtr-A, and embeds it into the first message in the multicast video stream and sends it out;

[0123] ④ After receiving the multicast video stream sent by the door lock, the door lock gateway forwards it through the RPT forwarding path from the door lock gateway to the RP, the forwarding path from the RP to the set-top box gateway, and then sends it to each set-top box gateway.

[0124] ⑤ After receiving the multicast stream, the set-top box gateway extracts the visitor feature vector VstFtr-A to query whether an authorization association already exists locally. If found, the multicast stream can be sent to the interface where the authorized person resides. Initially, the set-top box and its gateway have no authorization association. Therefore, the following steps are executed:

[0125] 1) Each set-top box gateway forwards the video stream to the local interface where the set-top box is located, based on the collaborative interface binding table, regardless of whether the authorized person ID column matches. Therefore, each owner can authorize visitors on their own set-top box. The collaborative interface binding table also prevents the video stream from being sent to interfaces other than those of the set-top box, such as the interface connected to the router. Figure 3 As shown in Tables 3 and 4, router B forwards the video stream to the VInt1 interface, router C forwards the video stream to the VInt1 interface, and router D forwards the video stream to the VInt1 interface.

[0126] 2) The set-top box decodes and displays the multicast video on the screen. The camera collects the facial feature values ​​corresponding to the target in real time and compares them with the locally stored authorized person feature vector to determine if the person is authorized. It also checks if the target has performed an authorization action such as nodding. If the nodding person is identified as the authorized person, authorization is considered successful. After successful authorization, the set-top box obtains the door lock address from the IP source address of the multicast video stream, directly sends a unicast message to the door lock to open it, and simultaneously reports the authorization association to the set-top box gateway via an IGMP "Authorization Successful" message (containing the visitor's facial feature value and the authorized person's ID). As shown in Table 2, the Type field adds an enumeration value of 0x1C, Group Address is filled with the multicast address G1, Auth ID is filled with the authorized person's ID, Auxiliary Data is filled with the visitor's facial feature value (multiple visitors can be authorized), the IP layer destination address is filled with G1, and the source address is the set-top box's IP address.

[0127] 3) After the set-top box gateway detects the IGMPv2 "authorization successful" message, it saves it to Table 5 below:

[0128] Table 5. Authorization Association Table for Set-Top Box Gateway Maintenance

[0129] ;

[0130] ⑥ The set-top box gateway synchronizes the association relationship between authorized users and visitors to other set-top box gateways via the gateway multicast address G2 (e.g., 224.0.0.100). The message format is shown in Table 6. The Group Address is filled with the stream multicast address G1, the AuthID is filled with the authorized user ID, and the Auxiliary Data is filled with the visitor's facial feature value. Therefore, all set-top box gateways maintain the authorization association table shown in Table 5. The association relationship between authorized users and visitors is synchronized via another gateway multicast address G2.

[0131] Table 6. Message formats used for synchronizing authorization associations between set-top box gateways

[0132]

[0133] ⑦ When the same visitor visits again and the set-top box gateway receives the multicast video stream again, it extracts the visitor feature vector VstFtr-A from the video stream and compares it one by one with the visitor feature vectors in the local authorization association table. If a match is found, it retrieves the corresponding authorized person ID and searches the local collaborative interface binding table (the authorized person ID column in Tables 3 and 4). After finding the match, it sends the video stream to the corresponding downlink interface (set-top box) in Tables 3 and 4, where the set-top box confirms and unlocks the connection. Figure 3As shown, after visitor C leaves the room where door lock B is located, he re-enters the same room. The owner 3 leaves set-top box B and arrives at set-top box C. The collaborative interface binding table maintained by router D is shown in Table 7 below. After the video stream from door lock B arrives at router D, it compares with Table 5 to obtain the authorized person ID AuthID-3. According to Table 7, it looks up the downlink interface corresponding to AuthID-3 and forwards the video stream to the VInt1 interface, where set-top box C performs the authorization confirmation operation.

[0134] Table 7. Cooperative Interface Binding Table Maintained by Router D

[0135] ;

[0136] ⑧ When the set-top box gateway forwards a multicast video stream, if an authorization relationship exists but the authorized person ID is not registered in the local interface (i.e., the authorized ID corresponding to the visitor's facial feature vector is not found in the authorized person IDs registered in the local interface), it indicates that the authorized person is temporarily not local. In this case, the following operation is performed:

[0137] 1) The video stream is in a 10-second authorization wait state, and the set-top box gateway does not forward the multicast stream. For example... Figure 3 As shown, after door lock B sends a video stream containing visitor C to router C, router C queries table 6 and finds that there is an authorized association relationship. However, since owner 3 has left set-top box B, router C has deleted owner 3's authorized person ID AuthID-3 from its downlink interface (set-top box). Therefore, router C does not forward the video stream.

[0138] 2) If, within the 10-second authorization waiting state, the set-top box gateway receives an authorization association message synchronized from another set-top box gateway, and the visitor feature vector and authorizing person ID in the authorization association are the same as the visitor feature vector and authorizing person ID contained in the video stream waiting to be forwarded, it indicates that the other set-top box gateway has forwarded the video stream to the authorizing person and completed the authorization operation. The local set-top box gateway then discards the forwarding. Figure 3 As shown, router C received the authorized association synchronized by router D, and both were VstFtr-C and AuthID-3, so it discarded the forwarding.

[0139] 3) If the set-top box gateway does not receive synchronized authorization association messages from other set-top boxes after the 10-second authorization wait period expires, it indicates that the authorizer is unavailable or has not processed the request in a timely manner. In this case, the set-top box gateway will treat the situation as if no authorization association exists, i.e., it will send the multicast stream to all set-top boxes. For example... Figure 3 As shown, if owner 3 is on set-top box C but does not perform an authorization operation within 10 seconds, routers B and C will forward the video stream to the interface locked by all set-top boxes after the 10-second timeout, for confirmation by owners 1 and 2.

[0140] (4) If the network is reset for some reason, the authorization association table and the collaboration interface binding table will be cleared. The mechanism for their regeneration and synchronization within the network is as follows:

[0141] ① When the router restarts or is needed for management, it will send a query message IGMPv2 "Cooperation Information Query" request to all ports. The Type field will have a new enumeration value of 0x1D, the Group Address will be filled with 0.0.0.0, all other fields will be left empty, and the IP layer destination address will be 224.0.0.1;

[0142] ② Set-top boxes and door locks respond to the query message from 224.0.0.1. The door lock reports a 0x19 "Door lock device registration" message, and the set-top box reports a 0x18 "Set-top box device registration" message, a 0x1A "Authorization appeared" message, and a 0x1C "Authorization successful" message.

[0143] ③ The set-top box gateway refreshes the local collaborative interface binding table and the authorized association table, and synchronizes the association between authorized persons and visitors to other set-top box gateways through the gateway multicast address G2.

[0144] II. Beneficial Effects:

[0145] The door lock collaborative opening method provided by this invention extends the IGMPv2 message fields and multicast messages, enabling routers in the multicast network to establish information synchronization, interface binding, and authorization association between visitors and authorized persons between door locks and set-top boxes based on the bidirectional PIM protocol. This achieves a solution for real-time location sensing of authorized persons, autonomous collaboration between door locks and multimedia authorization devices, and convenient authorization without a management platform. On the one hand, it saves construction costs and reduces maintenance costs; on the other hand, it improves the efficiency of visitor access authorization and enhances the user experience.

[0146] III. Protection Points:

[0147] 1. The set-top box has a built-in set of all authorized person characteristic vectors; after the set-top box camera obtains an indoor face, it compares the face and reports the authorized person ID to the gateway router via an enhanced IGMP message; the set-top box gateway establishes a (*, G) table entry with the RP, and the downlink interface of (*, G) is the interface that receives the IGMP message, and the authorized person ID carried by the IGMP is registered on this interface, forming a binding relationship between the interface and the set of authorized person IDs.

[0148] 2. The set-top box decodes and displays the visitor's video on the screen. After obtaining authorization from the authorized person through actions such as nodding, it binds the authorized person's ID with the visitor's feature vector and reports it to the gateway router via enhanced IGMP join message. The set-top box gateway saves the association between the authorized person's ID and the visitor and synchronizes the authorization association to other set-top box gateways through a separate multicast address.

[0149] 3. The door lock sends a normal IGMP join message, establishes a (*, G) table entry in the door lock gateway to the RP, and captures the visitor's facial feature vector when the visitor appears, embeds it into the multicast video stream and sends it.

[0150] 4. After receiving the multicast stream, the door lock gateway pushes the multicast stream to the set-top box gateway based on the established bidirectional PIM between the door lock gateway and the set-top box gateway. Initially, the visitor has no authorization record. After receiving the multicast stream, the set-top box gateway extracts the visitor's feature vector. If no authorization association is found locally, the video stream is sent to all set-top boxes. The set-top boxes decode the visitor's video and display it on their screens. After the set-top box camera captures a nodding or other recognition action, it compares the nodding person's feature vector stored locally to determine if the person is an authorized person. Then, it sends an enhanced IGMP join message, which includes the authorized person's ID, the visitor's facial feature vector, and the authorization association. The set-top box gateway refreshes the association between the authorized person and the visitor and sends the authorization association through a separate multicast address, notifying other set-top box gateways to wait for authorization release.

[0151] 5. After receiving the multicast stream, the set-top box gateway extracts the visitor's feature vector and checks if an authorization association already exists locally. If an authorized record exists, it compares the corresponding authorized person ID with the set of authorized person IDs bound to the local interface. If a match is found, the video stream is sent to the corresponding interface for authorization confirmation by the set-top box. The set-top box directly sends a unicast message to the door lock to open the door, and simultaneously sends a message indicating that the authorization process is complete, containing the authorized person ID. The set-top box gateway sends the authorization association through a separate multicast address, notifying other set-top box gateways to wait for authorization release. If an authorization association exists, but the authorized person ID corresponding to the visitor's facial feature vector is not found in the set of authorized person IDs registered on the local interface, the video stream enters a 10-second authorization waiting state, and the set-top box gateway does not forward the multicast stream. If the set-top box gateway receives an authorization association message within the 10-second authorization waiting state, it means that other set-top boxes have processed the authorization association, and the local set-top box gateway discards and forwards the multicast video stream. After the 10-second authorization wait period expires, the set-top box gateway will treat it as if there is no authorization association, that is, send the multicast stream to all set-top boxes.

[0152] 6. When the router restarts or is needed for management, it sends an IGMPv2 "Cooperation Information Query" request to all ports. Set-top boxes and door locks respond. Door locks report "Door Lock Device Registration" messages, and set-top boxes report "Set-top Box Device Registration" messages, "Authorization Appeared" messages, and "Authorization Successful" messages. The set-top box gateway refreshes the local collaboration interface binding table and authorization association table, and synchronizes the association between authorized persons and visitors to other set-top box gateways through the gateway multicast address G2.

[0153] As can be seen, based on the detection results of the authorized person indoors by the multimedia authorization device, the authorized person's identifier is reported to the gateway device via an enhanced multicast protocol message. The gateway device then establishes a binding relationship between the authorized person's identifier and the interface in the multicast forwarding table. Based on the door lock device's collection results of the visitor's biometric features, a multicast media stream containing the visitor's biometric features is sent to the multicast network. Based on the gateway device's reception of the multicast media stream, the visitor's biometric features are extracted, and the authorization association is queried. A differentiated forwarding strategy is executed based on the query results. Based on the multimedia authorization device's presentation of the forwarded content and the recognition results of the authorization action, authorization confirmation is completed and the door lock opening operation is triggered. This improves the reliability, immediacy, and flexibility of the door opening authorization process while ensuring the security of the user's biometric data.

[0154] Another embodiment of the present invention provides a door unlocking system that coordinates with a door lock, see [link to relevant documentation]. Figure 4 The system may include:

[0155] The sensing module 401 is used for authorizing the location of the authorized person and registration: based on the detection results of the authorized person indoors by the multimedia authorization device, the authorized person's identifier is reported to the gateway device through an enhanced multicast protocol message, and the gateway device establishes a binding relationship between the authorized person's identifier and the interface in the multicast forwarding table;

[0156] The acquisition module 402 is used for visitor information acquisition and distribution: based on the results of the door lock device's acquisition of visitor biometrics, it sends a multicast media stream containing visitor biometrics to the multicast network;

[0157] The decision module 403 is used for intelligent routing and authorization decision-making: based on the gateway device's reception of the multicast media stream, it extracts the visitor's biometric features and queries the authorization association relationship, and executes a differentiated forwarding strategy based on the query results;

[0158] The determination module 404 is used for authorization confirmation and door opening execution: based on the presentation of forwarded content and the recognition result of authorization actions by the multimedia authorization device, authorization confirmation is completed and the door lock opening operation is triggered.

[0159] This invention also provides a storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above method embodiments when running.

[0160] This invention also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0161] Specifically, the aforementioned electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the aforementioned processor, and the input / output device is connected to the aforementioned processor.

[0162] The above description, based on the embodiments shown in the figures, details the structure, features, and effects of the present invention. The above description is only a preferred embodiment of the present invention, but the present invention is not limited to the scope of implementation shown in the figures. Any changes made in accordance with the concept of the present invention, or equivalent embodiments modified to have equivalent changes, that do not exceed the spirit covered by the specification and figures, should be within the protection scope of the present invention.

Claims

1. A door-opening method in coordination with a door lock, characterized in that, The method includes: Authorized Person Location Awareness and Registration: Based on the detection results of authorized persons indoors by the multimedia authorization device, the authorized person identifier is reported to the gateway device via enhanced multicast protocol messages. The gateway device then establishes a binding relationship between the authorized person identifier and the interface in the multicast forwarding table. The authorized person location awareness and registration includes: Device Registration: Based on the multimedia authorization device's startup event, a device registration message is sent to a preset multicast address to establish a multicast forwarding path; Authorized Person Detection: Based on the continuous monitoring of indoor personnel by the multimedia authorization device, the optimal facial image is captured and biometric features are extracted; Identity Verification: Based on the comparison results between the extracted biometric features and the locally pre-stored authorized person feature set, the authorized person identifier is determined; Location Reporting: Based on the determined authorized person identifier, the authorized person's presence information is reported to the gateway device via enhanced multicast protocol messages; Binding Relationship Establishment: Based on the gateway device's parsing of the received messages, the authorized person identifier is registered on the corresponding interface in the multicast forwarding table, forming an interface-authorized person binding relationship. Visitor information collection and distribution: Based on the biometric data collected by the door lock device, a multicast media stream containing the visitor's biometric data is sent to the multicast network; wherein, the visitor information collection and distribution includes: Door lock registration: Based on the door lock device activation event, a door lock device registration message is sent to a preset multicast address to establish a multicast path from the door lock to the aggregation point; Visitor feature collection: Based on the trigger event of the visitor arriving at the door lock, the optimal facial image of the visitor is collected and converted into a biometric vector; Media stream encapsulation: Based on the extracted visitor biometric vector, it is embedded into the first message of the multicast media stream and sent; Network distribution: Based on the established multicast path, the multicast media stream containing the visitor's biometric data is distributed to each gateway device; Intelligent routing and authorization decision-making: Based on the gateway device's reception of the multicast media stream, visitor biometrics are extracted and authorization relationships are queried. A differentiated forwarding strategy is executed based on the query results. The intelligent routing and authorization decision-making includes: Feature extraction and query: Based on the multicast media stream received by the gateway device, the embedded visitor biometric vector is extracted; Relationship judgment: Based on the extracted feature vector, the local authorization relationship table is queried to determine if a corresponding authorization relationship exists; Initial forwarding processing: Based on the judgment result that no authorization relationship exists, the multicast media stream is forwarded to all connected multimedia authorization devices; Targeted forwarding processing: Based on the judgment result that an authorization relationship exists and the authorizing user is online, the multicast media stream is only forwarded to the interface where the corresponding authorizing user is located; Waiting forwarding processing: Based on the judgment result that an authorization relationship exists but the authorizing user is offline, a time-limited waiting mechanism is initiated. During this period, if a relationship synchronization is received from another gateway, the media stream is discarded; otherwise, after the timeout, the initial forwarding process is executed. Authorization Confirmation and Door Opening Execution: Based on the presentation of forwarded content and the recognition results of authorized actions by the multimedia authorization device, authorization confirmation is completed and the door lock opening operation is triggered. The authorization confirmation and door opening execution includes: Media Presentation and Motion Capture: Based on the multicast media stream received by the multimedia authorization device, it is decoded and presented on the display device, while simultaneously capturing the viewer's biometrics and preset authorized actions through an associated camera; Local Authorization Verification: Based on the matching results of the captured biometrics and local authorized person characteristics, as well as the recognition results of preset authorized actions, authorization confirmation is completed; Door Opening Command Sending: Based on the authorization confirmation result, the multimedia authorization device directly sends a door opening command to the door lock device; Association Relationship Reporting: Based on the authorization confirmation result, the multimedia authorization device reports the visitor-authorized person association relationship to the gateway device through an enhanced multicast protocol; Relationship Synchronization: Based on the association relationship received by the gateway device, it synchronizes it to other gateway devices through a dedicated multicast address.

2. The method according to claim 1, characterized in that, The method also includes anomaly recovery processing: Status query trigger: Based on network reset or management needs, the gateway device sends a coordination information query request to all ports; Device status re-reporting: Based on the received query request, the door lock device re-reports the device registration information, and the multimedia authorization device re-reports the device registration information, the information of the authorized person, and the established authorization associations; Table Reconstruction: Based on the various types of newly received reported information, the gateway device reconstructs the local multicast forwarding table and authorization association table; Relationship resynchronization: Based on the rebuilt authorized association table, the gateway device resynchronizes to other gateway devices via a dedicated multicast address.

3. A door unlocking system in coordination with a door lock, characterized in that, The system includes: The perception module is used for authorized person location perception and registration: based on the detection results of authorized persons indoors by the multimedia authorization device, the authorized person identifier is reported to the gateway device via enhanced multicast protocol messages. The gateway device then establishes a binding relationship between the authorized person identifier and the interface in the multicast forwarding table. The authorized person location perception and registration includes: device registration: based on the multimedia authorization device's startup event, a device registration message is sent to a preset multicast address to establish a multicast forwarding path; authorized person detection: based on the continuous monitoring of indoor personnel by the multimedia authorization device, the optimal facial image is captured and biometric features are extracted; identity comparison: based on the comparison results between the extracted biometric features and the locally pre-stored authorized person feature set, the authorized person identifier is determined; location reporting: based on the determined authorized person identifier, the authorized person's presence information is reported to the gateway device via enhanced multicast protocol messages; binding relationship establishment: based on the gateway device's parsing of the received messages, the authorized person identifier is registered on the corresponding interface in the multicast forwarding table, forming an interface-authorized person binding relationship. The acquisition module is used for visitor information acquisition and distribution: based on the biometric data acquisition results of the door lock device, it sends a multicast media stream containing the visitor's biometric data to the multicast network; wherein, the visitor information acquisition and distribution includes: door lock registration: based on the door lock device start event, a door lock device registration message is sent to a preset multicast address to establish a multicast path from the door lock to the aggregation point; visitor feature acquisition: based on the trigger event of the visitor arriving at the door lock, the optimal facial image of the visitor is acquired and converted into a biometric vector; media stream encapsulation: based on the extracted visitor biometric vector, it is embedded into the first message of the multicast media stream and sent; network distribution: based on the established multicast path, the multicast media stream containing the visitor features is distributed to each gateway device; The decision module is used for intelligent routing and authorization decisions: based on the gateway device's reception of the multicast media stream, it extracts visitor biometric features and queries authorization relationships, and executes differentiated forwarding strategies based on the query results; wherein, the intelligent routing and authorization decisions include: feature extraction and query: extracting the embedded visitor biometric vector from the multicast media stream received by the gateway device; relationship judgment: querying the local authorization relationship table based on the extracted feature vector to determine whether a corresponding authorization relationship exists; initial forwarding processing: based on the judgment result that no authorization relationship exists, forwarding the multicast media stream to all connected multimedia authorization devices; targeted forwarding processing: based on the judgment result that an authorization relationship exists and the authorizing person is online, forwarding the multicast media stream only to the interface where the corresponding authorizing person is located; waiting forwarding processing: based on the judgment result that an authorization relationship exists but the authorizing person is offline, initiating a timed waiting mechanism; if a relationship synchronization is received from other gateways during this period, the media stream is discarded; otherwise, after the timeout, it is processed as initial forwarding. The determination module is used for authorization confirmation and door opening execution: based on the presentation of forwarded content and the recognition results of authorized actions by the multimedia authorization device, authorization confirmation is completed and the door lock opening operation is triggered; wherein, the authorization confirmation and door opening execution includes: media presentation and action capture: based on the multicast media stream received by the multimedia authorization device, it decodes and presents it on the display device, and simultaneously captures the viewer's biometrics and preset authorized actions through the associated camera; local authorization verification: based on the matching results of the captured biometrics and local authorized person characteristics, as well as the recognition results of preset authorized actions, authorization confirmation is completed; door opening command sending: based on the authorization confirmation result, the multimedia authorization device directly sends a door opening command to the door lock device; association relationship reporting: based on the authorization confirmation result, the multimedia authorization device reports the visitor-authorized person association relationship to the gateway device through the enhanced multicast protocol; relationship synchronization: based on the association relationship received by the gateway device, it synchronizes it to other gateway devices through a dedicated multicast address.

4. A storage medium, characterized in that, The storage medium stores a computer program, wherein the computer program is configured to execute the method of any one of claims 1-2 when it is run.

5. An electronic device, comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to perform the method according to any one of claims 1-2.

Images