Door lock camera cooperative processing method and system

Abstract

The invention discloses a door lock camera cooperative processing method and system, and the method comprises the steps: generating an identity verification request message according to a doorbell button signal detected by a door lock, and transmitting the identity verification request message to a corridor camera A covering the door lock; according to the identity checking request message, the camera A obtains a face feature vector of a person in front of the door lock, similarity matching is carried out on the face feature vector and a house owner feature vector, if the similarity exceeds a first threshold value, it is judged that verification is passed, and a face verification result is obtained; according to the face verification result, a robot calling instruction is sent to the platform, and the robot is triggered to move to the door lock position; according to a human face image collected after the robot reaches the position of the door lock, if the similarity between the human face image feature and the target human face feature exceeds a second threshold value, unlocking of the door lock is authorized, and the unlocking state is fed back to the platform. According to the embodiment of the invention, efficient, safe and intelligent access control can be realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of smart door locks, and in particular to a method and system for collaborative processing of door locks and cameras. Background Art

[0002] With the rapid development of smart home technology, the interconnected use of door locks and surveillance cameras is becoming increasingly popular. However, existing technologies still suffer from low authentication efficiency and insufficient collaborative processing capabilities. Traditional door lock systems typically rely on a single biometric feature (such as fingerprint or password), which presents security vulnerabilities in visitor authentication scenarios. Independent surveillance cameras, while capable of recording human activity, lack effective linkage with door locks, making it difficult to promptly address abnormal behavior. Existing solutions often rely on a cloud-based relay model for data exchange between door locks and cameras, which presents drawbacks such as high response latency and a high risk of privacy breaches. Furthermore, the lack of a multi-device collaborative authentication mechanism makes it difficult to address identity verification needs in complex scenarios. In apartments or offices, where frequent visitors frequently come and go, quickly and accurately verifying visitor identities and enabling secure authorization presents a technical challenge. Summary of the Invention

[0003] The purpose of the present invention is to provide a method and system for door lock and camera collaborative processing to address the deficiencies in the prior art and to achieve efficient, safe and intelligent access control.

[0004] An embodiment of the present application provides a method for door lock and camera collaborative processing, the method comprising: Door lock triggering and query request generation: When the door lock detects two consecutive doorbell button signals, it generates an identity verification request message containing the room ID number, door lock IP address, and the owner's facial feature vector, and sends it via the local area network to corridor camera A covering the door lock; Face retrieval and collaborative verification: Based on the identity verification request message, Camera A obtains the facial feature vector of the person in front of the door lock through real-time face monitoring, local historical video backtracking, and broadcast collaborative query. It then performs a similarity match with the homeowner's feature vector. If the similarity exceeds a first threshold, the verification is determined to be successful, and a face verification result is obtained. Robot summoning decision and execution: Based on the facial verification result, a robot summoning instruction containing the room ID number and target facial features is sent to the platform to trigger the robot to move to the door lock position; Robot secondary verification and door lock unlocking: Based on the facial image collected after the robot reaches the door lock location, if the similarity between the facial image features and the target facial features exceeds the second threshold, the door lock is authorized to be unlocked and the unlock status is fed back to the platform.

[0005] Optionally, the door lock triggering and query request generation includes: Doorbell signal detection and condition judgment: When the door lock detects two consecutive doorbell button presses, it is determined to be a valid trigger signal; the door lock records the number of consecutive double doorbell button presses. If the number of consecutive double doorbell button presses exceeds three within five minutes, a voice prompt will be issued to indicate that the door lock is locked. The door lock will no longer receive information about consecutive double doorbell button presses within 10 minutes. Homeowner information extraction and request encapsulation: Based on the homeowner's facial feature vector and room ID number stored in the door lock, an identity verification request message containing a timestamp, door lock IP address, and encrypted feature vector is encapsulated; Target camera positioning and message sending: According to the pre-stored coverage camera list of the door lock, the IP address of the nearest camera A is filtered and a request message is sent through the communication protocol.

[0006] Optionally, the face retrieval and collaborative verification include: Real-time facial monitoring and capture: In response to an identity verification request, Camera A initiates 30 seconds of real-time video stream analysis. Using a dynamic face detection algorithm, it captures the frontal face of the person in front of the door lock and generates a real-time facial feature vector. Historical video backtracking and structured matching: Based on the current person's body structured information, the system backtracks the previous 15 minutes of historical video, matches human targets with a similarity of more than 70% with the body structured information, locates their earliest appearance time point T, and extracts the corresponding facial feature vector; Cross-camera collaborative retrieval: Send a collaborative query request containing human body structured information and the corresponding time window to the cameras on the same floor at time point T, and receive the facial feature vectors returned by each camera; Multi-source feature fusion and decision-making: The facial feature vectors captured in real time, historically searched, and queried across cameras are weighted and fused, and their similarity is calculated with the homeowner's features. If the similarity exceeds 60%, the verification is considered passed.

[0007] Optionally, the robot summoning decision and execution includes: Camera field of view pre-screening: Camera A sends a real-time scene capture request to surrounding cameras, receives the current scene image returned by each camera, analyzes whether each camera covers the door lock area based on the returned image, and screens the camera set that may capture the target person; Dynamic time window estimation: Based on the earliest appearance time T of the target person on camera A and the positional relationship between each camera and the door lock, the possible time interval in which the person is in the field of view of adjacent cameras is estimated as [T-ΔT, T]. Hierarchical cascade retrieval: Prioritizes searching the historical videos of the target camera within the estimated time window. If no hit is found, the search is extended to the entire period of the previous 15 minutes. The matched facial feature vectors are sorted by time proximity and fed back to camera A.

[0008] Optionally, the robot secondary verification and door lock unlocking includes: Robot path planning: The platform retrieves pre-stored map data based on the room ID number. The robot uses the laser SLAM algorithm to plan the path and autonomously avoid obstacles to reach the target door lock. Near-field face acquisition: After the robot docks, it uses multi-angle cameras to capture facial images; Hierarchical verification mechanism: If the facial similarity between the collected face image and the target face features exceeds 90%, the door lock will be unlocked directly; Abnormal behavior handling: If verification fails three times in a row within 5 minutes, a voice prompt indicating that the door lock function has been disabled will be triggered, and the summon function will be locked for 10 minutes.

[0009] Another embodiment of the present application provides a door lock camera collaborative processing system, the system comprising: The trigger module is used to trigger the door lock and generate query requests: when the door lock detects two consecutive doorbell button signals, it generates an identity verification request message containing the room ID number, the door lock IP address, and the owner's facial feature vector, and sends it via the local area network to the corridor camera A covering the door lock; Retrieval module, for face retrieval and collaborative verification: Based on the identity verification request message, camera A obtains the facial feature vector of the person in front of the door lock through real-time face monitoring, local historical video backtracking, and broadcast collaborative query, and performs similarity matching with the homeowner's feature vector. If the similarity exceeds a first threshold, it is determined that the verification is successful and a face verification result is obtained; The summoning module is used for robot summoning decision-making and execution: based on the face verification result, a robot summoning instruction containing the room ID number and target facial features is sent to the platform to trigger the robot to move to the door lock position; The verification module is used for secondary verification of the robot and unlocking the door lock: based on the facial image collected after the robot reaches the door lock position, if the similarity between the facial image features and the target facial features exceeds the second threshold, the door lock is authorized to be unlocked and the unlock status is fed back to the platform.

[0010] Yet another embodiment of the present application provides a storage medium, wherein the storage medium stores a computer program, wherein the computer program is configured to execute any of the above methods when run.

[0011] Yet another embodiment of the present application provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute any of the above methods.

[0012] Compared with the prior art, the present invention provides a method for collaborative processing of door locks and cameras, which generates an identity verification request message based on the doorbell button signal detected by the door lock and sends it to the corridor camera A covering the door lock; based on the identity verification request message, camera A obtains the facial feature vector of the person in front of the door lock and performs similarity matching with the homeowner's feature vector. If the similarity exceeds a first threshold, it is determined that the verification is passed, and a face verification result is obtained; based on the face verification result, a robot summoning command is sent to the platform to trigger the robot to move to the door lock position; based on the face image collected after the robot arrives at the door lock position, if the similarity between the facial image feature and the target facial feature exceeds a second threshold, the door lock is authorized to be unlocked, and the unlocking status is fed back to the platform, thereby realizing efficient, safe and intelligent access control. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 A hardware structure block diagram of a computer terminal for a door lock camera collaborative processing method provided by an embodiment of the present invention; Figure 2 A schematic flow chart of a door lock and camera collaborative processing method provided by an embodiment of the present invention; Figure 3 A schematic diagram of the structure of a door lock and camera collaborative processing system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0014] The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and are not to be construed as limiting the present invention.

[0015] The embodiment of the present invention first provides a door lock camera collaborative processing method, which can be applied to electronic devices such as computer terminals, specifically ordinary computers, etc.

[0016] The following describes it in detail by taking running on a computer terminal as an example. Figure 1 The hardware structure block diagram of a computer terminal for a door lock camera collaborative processing method provided by an embodiment of the present invention. Figure 1 As shown, the computer device includes a processor, a memory, and a network interface connected via a system bus, wherein the memory may include a non-volatile storage medium and an internal memory.

[0017] The non-volatile storage medium can store an operating system and a computer program. The computer program includes program instructions, which, when executed, can cause the processor to execute any one of the door lock camera collaborative processing methods.

[0018] The processor is used to provide computing and control capabilities and support the operation of the entire computer equipment.

[0019] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, the processor can execute any one of the door lock camera collaborative processing methods.

[0020] The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art will understand that Figure 1 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0021] It should be understood that the processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0022] See also Figure 2 An embodiment of the present invention provides a method for door lock camera collaborative processing, which may include the following steps: S201, door lock triggering and query request generation: Based on the door lock detecting two consecutive doorbell button signals, an identity verification request message containing the room ID number, the door lock IP address, and the homeowner's facial feature vector is generated and sent via the local area network to the corridor camera A covering the door lock. Specifically, the door lock triggering and query request generation includes: Doorbell signal detection and condition judgment: When the door lock detects two consecutive doorbell button presses, it is determined to be a valid trigger signal; the door lock records the number of consecutive double doorbell button presses. If the number of consecutive double doorbell button presses exceeds three within five minutes, a voice prompt will be issued to indicate that the door lock is locked. The door lock will no longer receive information about consecutive double doorbell button presses within 10 minutes. Homeowner information extraction and request encapsulation: Based on the homeowner's facial feature vector and room ID number stored in the door lock, an identity verification request message containing a timestamp, door lock IP address, and encrypted feature vector is encapsulated; Target camera positioning and message sending: According to the pre-stored coverage camera list of the door lock, the IP address of the nearest camera A is filtered and a request message is sent through the communication protocol.

[0023] S202, Face Retrieval and Collaborative Verification: Based on the identity verification request message, Camera A obtains the facial feature vector of the person in front of the door lock through real-time facial monitoring, local historical video backtracking, and broadcast collaborative query. It then performs a similarity match with the homeowner's feature vector. If the similarity exceeds a first threshold, the verification is determined to be successful, and a face verification result is obtained. Specifically, the face retrieval and collaborative verification includes: Real-time facial monitoring and capture: In response to an identity verification request, Camera A initiates 30 seconds of real-time video stream analysis. Using a dynamic face detection algorithm, it captures the frontal face of the person in front of the door lock and generates a real-time facial feature vector. Historical video backtracking and structured matching: Based on the current person's body structured information, the system backtracks the previous 15 minutes of historical video, matches human targets with a similarity of more than 70% with the body structured information, locates their earliest appearance time point T, and extracts the corresponding facial feature vector; Cross-camera collaborative retrieval: Send a collaborative query request containing human body structured information and the corresponding time window to the cameras on the same floor at time point T, and receive the facial feature vectors returned by each camera; Multi-source feature fusion and decision-making: The facial feature vectors captured in real time, historically searched, and queried across cameras are weighted and fused, and their similarity is calculated with the homeowner's features. If the similarity exceeds 60%, the verification is considered passed.

[0024] S203, robot summoning decision and execution: Based on the face verification result, a robot summoning instruction including the room ID number and target facial features is sent to the platform to trigger the robot to move to the door lock position. Specifically, the robot summoning decision and execution includes: Camera field of view pre-screening: Camera A sends a real-time scene capture request to surrounding cameras, receives the current scene image returned by each camera, analyzes whether each camera covers the door lock area based on the returned image, and screens the camera set that may capture the target person; Dynamic time window estimation: Based on the earliest appearance time T of the target person on camera A and the positional relationship between each camera and the door lock, the possible time interval in which the person is in the field of view of adjacent cameras is estimated as [T-ΔT, T]. Hierarchical cascade retrieval: Prioritizes searching the historical videos of the target camera within the estimated time window. If no hit is found, the search is extended to the entire period of the previous 15 minutes. The matched facial feature vectors are sorted by time proximity and fed back to camera A.

[0025] S204, robot secondary verification and door lock unlocking: Based on the facial image captured by the robot after it reaches the door lock location, if the similarity between the facial image features and the target facial features exceeds a second threshold, the robot authorizes the door lock to be unlocked and feedback the unlocking status to the platform. Specifically, the robot secondary verification and door lock unlocking includes: Robot path planning: The platform retrieves pre-stored map data based on the room ID number. The robot uses the laser SLAM algorithm to plan the path and autonomously avoid obstacles to reach the target door lock. Near-field face acquisition: After the robot docks, it uses multi-angle cameras to capture facial images; Hierarchical verification mechanism: If the facial similarity between the collected face image and the target face features exceeds 90%, the door lock will be unlocked directly; Abnormal behavior handling: If verification fails three times in a row within 5 minutes, a voice prompt indicating that the door lock function has been disabled will be triggered, and the summon function will be locked for 10 minutes.

[0026] In actual applications, when the owner of a smart door lock forgets to bring their door card, they can press the door lock's "doorbell" button to summon a robot to help them open the door by scanning their face. However, if a visitor or other person (not the homeowner) presses the door lock's "doorbell" button in front of the door, the robot may also be summoned. The ineffective service of the robot's door service will cause those who actually need the robot's service to be delayed in the queue and affected. This invention proposes a method that uses door locks and corridor facial recognition to accurately summon a robot to perform facial recognition on the owner, and the robot will unlock and open the door, thereby eliminating the behavior of non-owners calling the robot. A technical solution specifically includes: 1. Core Concept: To prevent non-homeowners from ineffectively summoning the robot by pressing the door lock, the door lock triggers a nearby corridor camera A to obtain the facial feature vector of the owner of the room corresponding to the door lock. Since camera A cannot see the person's face, it sends the person's structured information to a nearby camera, which responds with a facial feature vector. Camera A compares the facial feature vectors. If the similarity exceeds 60% (due to lighting conditions, the similarity doesn't need to be too high), it summons the robot. The robot obtains the person's face. If the similarity exceeds 90%, it connects to the door lock and opens the door; otherwise, it remains unchanged. A new method is added: Camera A activates reverse playback of historical video to determine the direction from which the person approached the door lock. If the person approaches from the front, the person's face is directly captured. If the person approaches from behind, the precise time period of their movement is notified to the opposite camera, which then directly retrieves the video during that period to capture the person's face. If the person approaches from the side, the side camera is notified of the time period of their movement. Added: This camera requests each of the surrounding cameras to take a picture and send it to this camera to understand the visual range of each camera; this camera then plays the video in reverse to see at what point in time each camera can see the person near the door lock. The purpose of watching in reverse is to quickly understand the above information at the first time; the above time point is sent to the corresponding camera, notifying it to retrieve the image at that time point and obtain the face, so that the corresponding camera can quickly find the required image based on the time point.

[0027] 2. The complete process of technical implementation is as follows: Prerequisites: The owner's entrance door is installed with a smart door lock that supports RFID card reading to unlock the door. The door lock stores the owner's information, including name, RFID-ID number, etc. The door lock is connected to the platform to report the room ID number, door lock ID number, door lock IP, etc.

[0028] Cameras are installed throughout the entrance corridors, ensuring comprehensive coverage (meaning every area of ​​the corridor can be captured by the cameras). Each entrance door is calibrated in the video image (meaning when someone is at a door, the calibration information can be used to determine the room ID of that person). The cameras are connected to the door locks, which record the camera ID and IP address of the lock. The corridor cameras are connected to the platform, reporting their ID numbers and other information. The platform also synchronizes and stores a camera list, including the location and direction of nearby cameras (directly in front of, behind, or to the side of the camera), as well as their IDs and IP addresses.

[0029] The robot has a pre-set navigation map and can autonomously navigate to a designated room door lock based on a pre-set room ID. A facial recognition camera and display screen are installed to perform facial recognition comparisons. The robot is connected to the door lock and supports unlocking authorization for designated doors. The robot is also connected to the platform and receives tasks issued by the platform (such as delivering items to owners, providing on-site facial recognition and unlocking doors for owners), and regularly synchronizes with the resident registration list, which includes room ID numbers, names, facial images / feature vectors, and more.

[0030] Under normal circumstances, the homeowner uses an RFID card to unlock and open the door: Owner A swipes the RFID card on his own door lock, the door lock reads the RFID card ID number and compares it with the ID number in the stored homeowner information. If the match is correct, the door is unlocked for the owner and the process ends.

[0031] The owner forgets to bring the RFID card when going out, and presses the "doorbell" button of the door lock M twice in a row to call the robot. The door lock M sends a "face query" request message to the camera A.

[0032] For example, person B (assuming they are the owner of door lock M) realizes they didn't bring their RFID card with them when they returned home. (This could be because they left their RFID card somewhere else.) Door lock M is closed, so person B can't unlock the door by swiping their RFID card. Person B presses the "Doorbell" button on door lock M twice in succession, hoping to summon a robot to unlock it.

[0033] Door lock M detects two consecutive doorbell button signals, queries the IP address (A) of camera A covering the door lock (for example, the corridor camera covering door lock M is camera A), and sends an "identity verification" request message to camera A, which includes the verification event ID number (generally the door lock ID number + timestamp), room ID number (M), door lock ID number (M), door lock IP address (M), all owner names and face images / feature vectors (in this case, including the face images / feature vectors of all people stored in the door lock where the owner is located), timestamp (the current clock when the "doorbell" button is pressed), etc.

[0034] The corridor camera receives and caches (automatically deletes the cache after 15 minutes) the "identity verification" request message sent by door lock M. The corridor camera performs three types of face recognition comparisons and starts a 30-second timer.

[0035] Camera A receives and caches the "Verify Identity" message sent by door lock M, extracts the facial image and feature vector from the message, and then performs the following actions: 1) Camera A immediately performs 30 seconds of real-time facial recognition monitoring; 2) Extracts the image and structured information of person B in front of door lock M to query the previous 15 minutes of camera A's historical video; 3) Sends the image and structured information of person B in front of door lock M to a "Broadcast Find Person" message, requesting other cameras in the corridor to query their respective 15 minutes of historical video. The 15-minute limit (configurable) is required because recordings taken too early may capture someone wearing similar clothing.

[0036] Step 1: Camera A receives and caches the "Identity Verification" message sent by door lock M, extracts the facial image / feature vector from the message (this may contain multiple facial images / feature vectors), and immediately performs 30 seconds of real-time facial detection. If camera A captures the face of person b in front of door lock M during the 30-second real-time facial detection (at this time, person b in front of door lock A may turn his head toward camera A), the facial image / feature vector of person b is extracted.

[0037] Step 2: Camera A immediately captures a human image of person b in front of the current door lock M and extracts structured information. Using this human image / structured information (the extracted human image / structured information at this point represents person b ringing the doorbell in front of door lock M), camera A searches back through the previous 15 minutes of historical video stored in camera A (i.e., the human image / structured information is matched against the human image / structured information detected in the historical video to determine if the human image / structured information similarity exceeds a threshold. The human image / structured information similarity threshold can generally be set to 70%; the search is performed backwards from the most recent moment in the historical video, and the query ends as soon as the human image of person b is retrieved). If person b's human image / structured information successfully matches a clear facial image of person b at time T (time T is the earliest time person b appears in A's historical video), the facial image / feature vector of person b is extracted.

[0038] Step 3: Camera A sends a "Broadcast Find Person" message to the other corridor cameras on the same floor, using the image / structured information of person b in front of the current door lock M (already implemented in Step 2). The message contains the camera ID (A), camera IP address (A), person b's image / structured information, and time T. After receiving and buffering the "Broadcast Find Person" message (which is automatically deleted after 15 minutes), the other corridor cameras on the same floor use the image / structured information of person b in the message to retrieve the historical video footage stored by their own camera, which dates back to the time T. (At time T, person b has already reached door lock A. Therefore, the other door locks should retrieve historical footage older than time T; otherwise, person b will not be seen in the videos after time T, as person b has already passed by.) If camera Z (using camera Z, which successfully retrieves the facial image corresponding to person b's image / structured information, as an example to illustrate the process) successfully matches person b's image / structured information and detects a clear facial image of person b, the facial image / feature vector of person b is extracted. Based on the camera IP (A) in the "Broadcast Person Search" message, camera A responds with a "Found" message containing camera ID (Z), camera IP (Z), person B's body image / structured information, and person B's facial image / feature vector. Upon receiving the "Found" reply from camera Z, camera A extracts the facial image / feature vector of person B (person B).

[0039] Further improvement: Camera A captures the human image / structured information of person b in front of door lock M, and matches the human image / structured information of person b based on the 15-minute historical video of this camera. If the facial image of person b is not extracted, the camera searches for the cameras in which person b may have been captured in the video, and notifies the corresponding camera of the corresponding time T to accurately jump to that time point to retrieve the face of person b, thereby reducing interference with other cameras.

[0040] In the preceding steps, when camera A extracts the image and structured information of person B in front of door lock M and sends a "broadcast person search" message to other cameras in the corridor to query their 15-minute historical video, all cameras in the corridor are triggered to help query their own 15-minute historical video, causing unnecessary computation for many cameras that did not capture person B. To address this issue, the following optimization is implemented: 1) Camera A captures and extracts the image / structured information of person b in front of door lock M. It then searches the previous 15 minutes of camera A's historical video in a backwards manner to match the image / structured information of person b (person b). The process is then processed based on whether or not a match is found for person b. (There are three possible cases: Case 1: Person b is not matched; Case 2: Person b is matched but no face image exists (face b exists); and Case 3: Person b is matched and both face b exist.)

[0041] 2) Case 1: Camera A fails to match person B (possibly due to lighting conditions, focus, and other factors). Following the previous steps, camera A sends a "Broadcast Find Person" message about person B, requesting other cameras in the corridor to query their respective 15-minute historical videos.

[0042] 3) Case 2: Camera A matches person b but fails to extract face b.

[0043] For example, camera A captures an image of person B, but fails to capture a clear facial image that meets the facial recognition standards (image requirements for facial recognition: generally, a frontal face with a minimum of 80*80 pixels). Alternatively, camera A cannot accurately match the detected facial image to person B's image, meaning the facial image is not extracted from person B's image. Therefore, it's impossible to confirm whether the captured facial image is of the same person as person B's image.

[0044] When camera A matches person b from its own 15 minutes of historical video but fails to extract face b, camera A caches the earliest appearance timestamp (time T) of person b in the previous 15 minutes of historical video of camera A and the body image / structured information (person b) of person b in front of the current door lock M based on the time point (time T) when person b appeared.

[0045] Camera A captures an image P(A) from its current video (at this moment, person B is in front of door lock M, and A can capture person B). At the same time, based on the floor information of the camera in the camera list, it sends a "request image" message to the surrounding cameras on the same floor (requesting to send a scene image captured by this camera at the current time, in order to understand the visual range of each surrounding camera). The message includes the camera ID (A), camera IP (A), etc. Other cameras on the floor (for example, cameras B, C, D, and E on the same floor) receive the "request image" message, each captures an image from the current video, and sends it back to camera A (for example, the images sent in the reply messages by cameras B, C, D, and E are P(B), P(C), P(D), and P(E) respectively). Camera A receives and caches images P(B), P(C), P(D), and P(E), along with its own captured image P(A). Using intelligent analysis (using the image of person b in P(A) to calculate whether the similarity exceeds a threshold, which can be set to 60%), it compares P(B), P(C), P(D), and P(E) to see if the same person b as in P(A) is present. If person b is found in P(B) or P(C), A sends an "Assist in Finding Person" message to cameras B and C, including the earliest appearance time T of person b in camera A's 15-minute historical video and the current person b. Cameras B and C receive the "Assist in Finding Person" message and search for person b in their own historical video clips prior to time T. If camera B finds person b and detects a clear facial image of person b, it extracts the facial image and feature vector (person b). Camera B uses the facial image / feature vector of person b based on the IP address of the "Assisting in Person Search" camera (A) to reply to camera A with a "Found" message. This message contains the camera ID (B), the camera IP address (B), the image / structured information of person b, and the facial image / feature vector of person b. Upon receiving the "Found" reply from camera B, camera A extracts the facial image / feature vector of person b (person b). If person b is not found in P(B), P(C), P(D), or P(E), a "Broadcast Person Search" message is sent to the other cameras on the floor using the same method as described above. This message contains the camera ID (A), the camera IP address (A), the image / structured information of person b, and the time T of the earliest appearance.

[0046] 4) Case 3: Camera A matches person b and there is face b, then the facial image / feature vector of the person (face b) is extracted.

[0047] Within 30 seconds, camera A uses real-time facial recognition monitoring of its own camera, reverse review of the previous 15 minutes of historical video from its own camera, and reverse review of the previous 15 minutes of historical video from other cameras. If it extracts the facial image / feature vector (face b) of person b in front of door lock M in any of these ways, it immediately matches face b with each facial image / feature vector in the "Verify Identity" message sent by door lock M. If the face of person b is successfully matched, the verification result is sent to door lock M.

[0048] Detailed description of the process: After receiving and buffering the "Verify Identity" message from door lock M, camera A begins a 30-second countdown based on the timestamp in the "Verify Identity" message (typically set to 30 seconds; setting it too long may affect the waiting time at the door lock). If, within 30 seconds, any of the aforementioned steps successfully matches and extracts the facial image / feature vector of person b (face b) in front of door lock M, camera A then performs facial matching (calculates facial similarity) against each of the names and facial images / feature vectors in the cached "Verify Identity" request message sent by M. If a match is found, camera A immediately sends a "Verify Identity Result" reply message to door lock M, containing the event ID, room ID (M), door lock ID (M), door lock IP address (M), whether it matches (yes), and the facial image / feature vector (face b). The received matching message will be automatically deleted after 15 minutes of buffering (there may be multiple reply messages from other cameras within 30 seconds), and the reply message of "identity verification result" will not be sent repeatedly.

[0049] If the facial image / feature vector (face b) of person b in front of door lock M is not successfully matched and extracted according to the above steps within 30 seconds, camera A will reply to door lock M with an "identity verification result" message indicating that it was not found, including the summoning event ID number, room ID number (M), door lock ID number (M), door lock IP (M), and whether it matched / not matched.

[0050] The door lock M receives the "identity verification result" message sent by the camera A and determines whether to summon the robot based on whether the result in the message matches.

[0051] Detailed description of the process: When door lock M receives the "Identity Verification Result" message from camera A, if the "Yes" result in the message is "Yes," door lock M immediately sends a "Call Robot" message to the platform, containing information such as camera ID (A), room ID (M), door lock ID (M), door lock IP (M), owner's name (b), and facial image / feature vector (b). The platform receives the "Call Robot" message from M, extracts the information, and sends a "Departure" message to the robot, containing the summon event ID, room ID (M), door lock ID (M), door lock IP (M), owner's name (b), and facial image / feature vector (b). The robot receives and caches the "Departure" message from the platform, then autonomously moves (typically using autonomous navigation and autonomous obstacle avoidance technologies, which are not the focus of this invention and will not be elaborated on) to door lock M and performs facial recognition on person B. If the facial comparison is successful, the door is unlocked for person B. The robot then proceeds to the next task address or returns to the default destination, depending on the task, and the process ends.

[0052] When door lock M receives the "Identity Verification Result" message from camera A, if the comparison in the message returns "No," door lock M issues a voice prompt, "Unable to summon robot." This indicates that camera A and the other cameras in the corridor have successfully identified person B. If person B leaves on their own, the process ends. If person B does not leave and cooperates (such as actively approaching and facing camera A in the corridor, or walking in front of the corridor and looking at the camera), then pressing the "Doorbell" button on door lock M twice in succession again may successfully summon the robot according to the aforementioned process. If the voice prompt "Unable to summon robot" is still heard from door lock M, it indicates that person B is not the owner of door lock M or has entered the wrong door lock.

[0053] Door lock M records the number of times the "Doorbell" button is pressed twice in succession. If the number of times the "Doorbell" button is pressed twice in succession exceeds 3 times within 5 minutes, a voice prompt of "Door lock locked" will be issued, and no more messages of pressing the "Doorbell" button twice in succession will be received within 10 minutes.

[0054] The door lock confirms that the person calling the robot is the door lock owner by requesting the corridor camera to perform facial comparison, thereby reducing the number of people who are not the door lock owner calling the robot and causing invalid robot calls.

[0055] It can be seen that according to the doorbell button signal detected by the door lock, an identity verification request message is generated and sent to the corridor camera A covering the door lock; according to the identity verification request message, camera A obtains the facial feature vector of the person in front of the door lock, and performs similarity matching with the homeowner's feature vector. If the similarity exceeds the first threshold, it is determined that the verification is passed, and the face verification result is obtained; according to the face verification result, a robot summoning command is sent to the platform to trigger the robot to move to the door lock position; according to the face image collected after the robot reaches the door lock position, if the similarity between the facial image feature and the target facial feature exceeds the second threshold, the door lock is authorized to be unlocked, and the unlocking status is fed back to the platform, thereby realizing efficient, safe and intelligent access control.

[0056] Another embodiment of the present invention provides a door lock camera collaborative processing system, see Figure 3 , the system may include: Trigger module 301, for door lock triggering and query request generation: upon detecting two consecutive doorbell button signals, generates an identity verification request message containing the room ID number, door lock IP address, and the homeowner's facial feature vector, and sends it via the local area network to corridor camera A covering the door lock; Retrieval module 302 is used for face retrieval and collaborative verification: Based on the identity verification request message, camera A obtains the facial feature vector of the person in front of the door lock through real-time face monitoring, local historical video backtracking, and broadcast collaborative query. It then performs a similarity match with the homeowner's feature vector. If the similarity exceeds a first threshold, the verification is determined to be successful, and a face verification result is obtained; The summoning module 303 is used for robot summoning decision and execution: according to the face verification result, a robot summoning instruction containing the room ID number and target facial features is sent to the platform to trigger the robot to move to the door lock position; Verification module 304 is used for secondary verification of the robot and unlocking the door lock: based on the facial image collected after the robot reaches the door lock position, if the similarity between the facial image features and the target facial features exceeds the second threshold, the door lock is authorized to be unlocked and the unlock status is fed back to the platform.

[0057] An embodiment of the present invention further provides a storage medium, in which a computer program is stored. The computer program is configured to execute the steps of any one of the above method embodiments when running.

[0058] An embodiment of the present invention further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to perform the steps in any one of the above method embodiments.

[0059] Specifically, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0060] It can be seen that according to the doorbell button signal detected by the door lock, an identity verification request message is generated and sent to the corridor camera A covering the door lock; according to the identity verification request message, camera A obtains the facial feature vector of the person in front of the door lock, and performs similarity matching with the homeowner's feature vector. If the similarity exceeds the first threshold, it is determined that the verification is passed, and the face verification result is obtained; according to the face verification result, a robot summoning command is sent to the platform to trigger the robot to move to the door lock position; according to the face image collected after the robot reaches the door lock position, if the similarity between the facial image feature and the target facial feature exceeds the second threshold, the door lock is authorized to be unlocked, and the unlocking status is fed back to the platform, thereby realizing efficient, safe and intelligent access control.

[0061] The above describes in detail the structure, features and effects of the present invention based on the embodiments shown in the drawings. The above is only a preferred embodiment of the present invention, but the scope of implementation of the present invention is not limited to what is shown in the drawings. Any changes made in accordance with the concept of the present invention, or modifications to equivalent embodiments with equivalent changes, which do not exceed the spirit covered by the description and drawings, should be within the scope of protection of the present invention.

Claims

1. A door lock camera collaborative processing method, characterized in that: The method comprises: Door lock triggering and query request generation: When the door lock detects two consecutive doorbell button signals, it generates an identity verification request message containing the room ID number, door lock IP address, and the owner's facial feature vector, and sends it via the local area network to corridor camera A covering the door lock; Face retrieval and collaborative verification: Based on the identity verification request message, Camera A obtains the facial feature vector of the person in front of the door lock through real-time face monitoring, local historical video backtracking, and broadcast collaborative query. It then performs a similarity match with the homeowner's feature vector. If the similarity exceeds a first threshold, the verification is determined to be successful, and a face verification result is obtained. Robot summoning decision and execution: Based on the facial verification result, a robot summoning instruction containing the room ID number and target facial features is sent to the platform to trigger the robot to move to the door lock position; Robot secondary verification and door lock unlocking: Based on the facial image collected after the robot reaches the door lock location, if the similarity between the facial image features and the target facial features exceeds the second threshold, the door lock is authorized to be unlocked and the unlock status is fed back to the platform.

2. The method according to claim 1, characterized in that The door lock triggering and query request generation includes: Doorbell signal detection and condition judgment: When the door lock detects two consecutive doorbell button presses, it is determined to be a valid trigger signal; the door lock records the number of consecutive double doorbell button presses. If the number of consecutive double doorbell button presses exceeds three within five minutes, a voice prompt will be issued to indicate that the door lock is locked. The door lock will no longer receive information about consecutive double doorbell button presses within 10 minutes. Homeowner information extraction and request encapsulation: Based on the homeowner's facial feature vector and room ID number stored in the door lock, an identity verification request message containing a timestamp, door lock IP address, and encrypted feature vector is encapsulated; Target camera positioning and message sending: According to the pre-stored coverage camera list of the door lock, the IP address of the nearest camera A is filtered and a request message is sent through the communication protocol.

3. The method according to claim 2, characterized in that The face retrieval and collaborative verification include: Real-time facial monitoring and capture: In response to an identity verification request, Camera A initiates 30 seconds of real-time video stream analysis. Using a dynamic face detection algorithm, it captures the frontal face of the person in front of the door lock and generates a real-time facial feature vector. Historical video backtracking and structured matching: Based on the current person's body structured information, the system backtracks the previous 15 minutes of historical video, matches human targets with a similarity of more than 70% with the body structured information, locates their earliest appearance time point T, and extracts the corresponding facial feature vector; Cross-camera collaborative retrieval: Send a collaborative query request containing human body structured information and the corresponding time window to the cameras on the same floor at time point T, and receive the facial feature vectors returned by each camera; Multi-source feature fusion and decision-making: The facial feature vectors captured in real time, historically searched, and queried across cameras are weighted and fused, and their similarity is calculated with the homeowner's features. If the similarity exceeds 60%, the verification is considered passed.

4. The method according to claim 3, characterized in that The robot summoning decision and execution includes: Camera field of view pre-screening: Camera A sends a real-time scene capture request to surrounding cameras, receives the current scene image returned by each camera, analyzes whether each camera covers the door lock area based on the returned image, and screens the camera set that may capture the target person; Dynamic time window estimation: Based on the earliest appearance time T of the target person on camera A and the positional relationship between each camera and the door lock, the possible time interval in which the person is in the field of view of adjacent cameras is estimated as [T-ΔT, T]. Hierarchical cascade retrieval: Prioritizes searching the historical videos of the target camera within the estimated time window. If no hit is found, the search is extended to the entire period of the previous 15 minutes. The matched facial feature vectors are sorted by time proximity and fed back to camera A.

5. The method according to claim 4, characterized in that The robot secondary verification and door lock unlocking include: Robot path planning: The platform retrieves pre-stored map data based on the room ID number. The robot uses the laser SLAM algorithm to plan the path and autonomously avoid obstacles to reach the target door lock. Near-field face acquisition: After the robot docks, it uses multi-angle cameras to capture facial images; Hierarchical verification mechanism: If the facial similarity between the collected face image and the target face features exceeds 90%, the door lock will be unlocked directly; Abnormal behavior handling: If verification fails three times in a row within 5 minutes, a voice prompt indicating that the door lock function has been disabled will be triggered, and the summon function will be locked for 10 minutes.

6. A door lock camera collaborative processing system, characterized in that: The system comprises: The trigger module is used to trigger the door lock and generate query requests: when the door lock detects two consecutive doorbell button signals, it generates an identity verification request message containing the room ID number, the door lock IP address, and the owner's facial feature vector, and sends it via the local area network to the corridor camera A covering the door lock; Retrieval module, for face retrieval and collaborative verification: Based on the identity verification request message, camera A obtains the facial feature vector of the person in front of the door lock through real-time face monitoring, local historical video backtracking, and broadcast collaborative query, and performs similarity matching with the homeowner's feature vector. If the similarity exceeds a first threshold, it is determined that the verification is successful and a face verification result is obtained; The summoning module is used for robot summoning decision-making and execution: based on the face verification result, a robot summoning instruction containing the room ID number and target facial features is sent to the platform to trigger the robot to move to the door lock position; The verification module is used for secondary verification of the robot and unlocking the door lock: based on the facial image collected after the robot reaches the door lock position, if the similarity between the facial image features and the target facial features exceeds the second threshold, the door lock is authorized to be unlocked and the unlock status is fed back to the platform.

7. The system according to claim 6, characterized in that The trigger module is specifically used to: Doorbell signal detection and condition judgment: When the door lock detects two consecutive doorbell button presses, it is determined to be a valid trigger signal; the door lock records the number of consecutive double doorbell button presses. If the number of consecutive double doorbell button presses exceeds three within five minutes, a voice prompt will be issued to indicate that the door lock is locked. The door lock will no longer receive information about consecutive double doorbell button presses within 10 minutes. Homeowner information extraction and request encapsulation: Based on the homeowner's facial feature vector and room ID number stored in the door lock, an identity verification request message containing a timestamp, door lock IP address, and encrypted feature vector is encapsulated; Target camera positioning and message sending: According to the pre-stored coverage camera list of the door lock, the IP address of the nearest camera A is filtered and a request message is sent through the communication protocol.

8. The system according to claim 7, characterized in that The retrieval module is specifically used to: Real-time facial monitoring and capture: In response to an identity verification request, Camera A initiates 30 seconds of real-time video stream analysis. Using a dynamic face detection algorithm, it captures the frontal face of the person in front of the door lock and generates a real-time facial feature vector. Historical video backtracking and structured matching: Based on the current person's body structured information, the system backtracks the previous 15 minutes of historical video, matches human targets with a similarity of more than 70% with the body structured information, locates their earliest appearance time point T, and extracts the corresponding facial feature vector; Cross-camera collaborative retrieval: Send a collaborative query request containing human body structured information and the corresponding time window to the cameras on the same floor at time point T, and receive the facial feature vectors returned by each camera; Multi-source feature fusion and decision-making: The facial feature vectors captured in real time, historically searched, and queried across cameras are weighted and fused, and their similarity is calculated with the homeowner's features. If the similarity exceeds 60%, the verification is considered passed.

9. A storage medium, characterized in that: The storage medium stores a computer program, wherein the computer program is configured to execute the method according to any one of claims 1 to 5 when executed.

10. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to perform the method according to any one of claims 1 to 5.

Images