Intelligent door lock unlocking method and system

Abstract

One or more embodiments of the present application provide an unlocking method and system for an intelligent door lock. The method is applied to the intelligent door lock and includes: when it is detected that a user enters the detection range, sending a first message to the door lock management platform so that the door lock management platform sends the device information of the public camera device corresponding to the intelligent door lock to the intelligent door lock; based on the device information, sending a second message to the public camera device so that the public camera device sends the historical video of a preset duration captured by it to the intelligent door lock; performing face detection on the video frames in the historical video to obtain the target face information included in the video frame with the latest shooting time in the video frames containing face information; when the target face information matches the face information of the user who can unlock the door, performing an unlocking operation. The present application can improve the security and reliability of the automatic unlocking operation for the intelligent door lock while realizing the automatic unlocking of the intelligent door lock.

Description

Technical Field

[0001] One or more embodiments of the present application relate to the technical field of smart home, and particularly to an unlocking method and system for a smart lock. Background Art

[0002] The Internet of Things refers to a network that connects various information sensing devices (such as radio frequency identification, infrared sensors, laser scanners, global positioning systems, etc.) to objects (such as home appliances, household appliances, cars, buildings, etc.) through the Internet for communication and information exchange to achieve intelligent identification, positioning, tracking, monitoring, and management. It expands the concept of the Internet, no longer limited to information exchange between people, but enabling information interaction between things and between things and people. Internet of Things devices refer to those physical objects that can be connected through the Internet and exchange data with other devices or systems. These devices are usually embedded with sensors, software, and other technologies to achieve data collection, data interaction with the Internet or other devices, and automation functions. Common types of Internet of Things devices include smart home devices (such as smart locks, smart cameras, etc.), smart home appliances (such as smart refrigerators, smart air conditioners, etc.), wearable devices (such as smart watches, fitness trackers, etc.), vehicle networking devices (such as in-vehicle units, smart traffic lights, etc.), healthcare Internet of Things devices (such as remote patient monitoring devices, drug management devices, etc.).

[0003] The Internet of Things has become a bridge connecting the physical world and the digital world, greatly promoting the intelligent process of all walks of life. With the rapid development of Internet of Things technology and the in-depth promotion of smart city construction, the application of smart home devices in communities is becoming increasingly popular. Currently, many families have deployed smart locks with cameras and displays on their entrance doors. These smart locks usually have functions such as face recognition, automatic and remote unlocking, and visitor visual intercom, providing users with convenient access management and security protection.

[0004] In practical applications, specifically how to achieve automatic unlocking of smart locks, so as to provide convenience for users through automatic unlocking, avoid users unlocking manually by themselves, and at the same time, be able to improve the security and reliability of automatic unlocking as much as possible, has become an urgent problem to be solved. Summary of the Invention

[0005] One or more embodiments of the present application provide the following technical solutions:

[0006] The present application provides an unlocking method for a smart lock, which is applied to a smart lock; the method includes:

[0007] When it is detected that a user enters the detection range of the smart door lock, send a first message to the door lock management platform, so that in response to receiving the first message, the door lock management platform determines a public camera device corresponding to the smart door lock based on the corresponding relationship between the smart door lock and the public camera device maintained, and sends the device information of the public camera device to the smart door lock;

[0008] In response to receiving the device information, based on the device information, send a second message to the public camera device, so that in response to receiving the second message, the public camera device acquires the historical video of a preset duration captured by it and sends the historical video to the smart door lock;

[0009] In response to receiving the historical video, perform face detection on the video frames in the historical video to obtain target face information; wherein, the video frame containing the target face information is the video frame with the latest shooting time among the video frames containing face information;

[0010] Match the target face information with the face information of the unlockable users maintained, and perform an unlocking operation when the target face information matches the face information of the unlockable users.

[0011] This application also provides an unlocking system for a smart door lock. The system includes a smart door lock, a public camera device, and a door lock management platform; wherein:

[0012] When the smart door lock detects that a user enters the detection range of the smart door lock, it sends a first message to the door lock management platform;

[0013] In response to receiving the first message, the door lock management platform determines a public camera device corresponding to the smart door lock based on the corresponding relationship between the smart door lock and the public camera device maintained, and sends the device information of the public camera device to the smart door lock;

[0014] In response to receiving the device information, the smart door lock sends a second message to the public camera device based on the device information;

[0015] In response to receiving the second message, the public camera device acquires the historical video of a preset duration captured by it and sends the historical video to the smart door lock;

[0016] In response to receiving the historical video, the smart door lock performs face detection on the video frames in the historical video to obtain target face information; wherein, the video frame containing the target face information is the video frame with the latest shooting time among the video frames containing face information;

[0017] The intelligent door lock matches the target face information with the face information of the unlockable users it maintains, and performs an unlocking operation when the target face information matches the face information of the unlockable users.

[0018] In the above technical solution, when the intelligent door lock first detects that a user enters its detection range, it sends a first message to the door lock management platform for unified management of the intelligent door lock. In response to the received first message, the door lock management platform determines the public camera device corresponding to the intelligent door lock based on the corresponding relationship between the intelligent door lock and the public camera device maintained by the door lock management platform, and returns the device information of the public camera device. Then, in response to the received device information, based on the device information, it sends a second message to the public camera device. In response to the received second message, the public camera device acquires the historical video of a preset duration it has captured and returns the historical video. Then, in response to the received historical video, it performs face detection on the video frames in the historical video to obtain the target face information included in the video frame with the latest shooting time in the video frames containing face information. Finally, it can match the target face information with the face information of the unlockable users maintained by the intelligent door lock, and perform an unlocking operation when the target face information matches the face information of the unlockable users.

[0019] By adopting the above method, the intelligent door lock, the public camera device, and the door lock management platform can be linked to jointly complete the automatic unlocking of the intelligent door lock. In this process, the intelligent door lock can obtain the face information of the user through the historical video captured by the public camera device, and perform an unlocking operation for the user when the face information of the user matches the face information of the unlockable users maintained by the intelligent door lock. In this way, while providing convenience for the user through automatic unlocking and avoiding the user from manually unlocking, the security and reliability of the automatic unlocking operation for the intelligent door lock can be improved. Description of the Drawings

[0020] The following will explain the drawings required for the description of the exemplary embodiments, where:

[0021] Figure 1 is a schematic diagram of an intelligent door lock management system shown in an exemplary embodiment of the present application.

[0022] Figure 2 is a flowchart of an unlocking method for an intelligent door lock shown in an exemplary embodiment of the present application.

[0023] Figure 3 is a schematic diagram of another intelligent door lock management system shown in an exemplary embodiment of the present application.

[0024] Figure 4It is a flowchart of another unlocking method for an intelligent door lock shown in an exemplary embodiment of the present application.

[0025] Figure 5 It is a multi-party interaction flowchart of an unlocking method for an intelligent door lock shown in an exemplary embodiment of the present application.

[0026] Figure 6 It is a schematic structural diagram of a device shown in an exemplary embodiment of the present application.

[0027] Figure 7 It is a block diagram of an unlocking device for an intelligent door lock shown in an exemplary embodiment of the present application. Detailed implementation manners

[0028] Here, the exemplary embodiments will be described in detail, and the examples are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with one or more embodiments of the present application. On the contrary, they are only examples consistent with some aspects of one or more embodiments of the present application.

[0029] It should be noted that in other embodiments, the steps of the corresponding method are not necessarily executed in the order shown and described in the present application. In some other embodiments, the steps included in the method may be more or less than those described in the present application. In addition, a single step described in the present application may be decomposed into multiple steps for description in other embodiments; and multiple steps described in the present application may also be combined into a single step for description in other embodiments.

[0030] For an intelligent door lock, it usually has functions such as face recognition, automatic and remote unlocking, and visitor visual intercom, providing users with convenient access management and security protection.

[0031] In practical applications, how to specifically implement the automatic unlocking of the intelligent door lock, so as to provide convenience for users through automatic unlocking, avoid users manually unlocking by themselves, and at the same time, be able to improve the security and reliability of automatic unlocking as much as possible, has become an urgent problem to be solved.

[0032] This application provides a technical solution for unlocking an intelligent door lock. In this technical solution, when the intelligent door lock first detects that a user enters its detection range, it sends a first message to the door lock management platform for unified management of the intelligent door lock. In response to receiving this first message, the door lock management platform determines the public camera device corresponding to the intelligent door lock based on the corresponding relationship between the intelligent door lock and the public camera device maintained by the door lock management platform, and returns the device information of the public camera device. Then, in response to receiving this device information, based on this device information, it sends a second message to the public camera device. In response to receiving this second message, the public camera device acquires the historical video of a preset duration that it has captured and returns this historical video. Then, in response to receiving this historical video, it performs face detection on the video frames in the historical video to obtain the target face information included in the video frame with the latest shooting time among the video frames containing face information. Finally, it matches the target face information with the face information of the users who can unlock the door maintained by the intelligent door lock, and when the target face information matches the face information of the user who can unlock the door, it performs the unlocking operation.

[0033] By adopting the above method, the intelligent door lock, the public camera device, and the door lock management platform can be linked to jointly complete the automatic unlocking of the intelligent door lock. During this process, the intelligent door lock can obtain the user's face information through the historical video captured by the public camera device, and when the user's face information matches the face information of the users who can unlock the door maintained by the intelligent door lock, it performs the unlocking operation for this user. In this way, while providing convenience for the user through automatic unlocking and avoiding the user from manually unlocking by themselves, the security and reliability of the automatic unlocking operation for the intelligent door lock can be improved.

[0034] Reference Figure 1 , Figure 1 is a schematic diagram of an intelligent door lock management system shown in an exemplary embodiment of this application.

[0035] As Figure 1 shown, in the above intelligent door lock management system, it may include multiple intelligent door locks and a door lock management platform for unified management of these intelligent door locks. Each intelligent door lock within a specific area (such as a community, a building, or a unit within a building) can access the Internet through Wi-Fi, cellular network, or other wireless network communication technologies. At the same time, the door lock management platform within this area can also access the Internet through wired network communication technology or wireless network communication technology, so that these intelligent door locks and the door lock management platform can be connected through the Internet for communication and information exchange.

[0036] Specifically, each intelligent door lock can register the device on the above-mentioned door lock management platform to upload its own device information (such as: device communication address, device geographical location, device operating status, etc.) to the door lock management platform for maintenance. Moreover, each intelligent door lock can also periodically synchronize the current device information of the intelligent door lock to the door lock management platform to update the device information of the intelligent door lock maintained by the door lock management platform.

[0037] Taking one of the intelligent door locks as an example, when the intelligent door lock is first started after deployment, it can send a registration message to the above-mentioned door lock management platform. The registration message can include the device unique identifier of the intelligent door lock (such as: device unique number), communication address, geographical location, associated household information, etc. Among them, the geographical location can specifically be data encoded by using the standard geographical coding format for information such as building number, floor number, room number, etc.; the associated household information can include the identity information of one or more users using the intelligent door lock (such as: personal password, face information, fingerprint information, etc., where the face information or fingerprint information can be the feature vectors extracted from the face image or fingerprint image). After receiving the registration message, the door lock management platform can store the corresponding relationship between various information such as the device unique identifier, communication address, geographical location, and associated household information in the registration message. In addition, the intelligent door lock can periodically send an information update message to the door lock management platform. The information update message can include the updated communication address, updated associated household information, etc. After receiving the information update message, the door lock management platform can update the corresponding stored information according to the updated information in the information update message.

[0038] The device information of each intelligent door lock maintained by the above-mentioned door lock management platform can be as shown in Table 1 below:

[0039] Table 1

[0040]

[0041] In the intelligent door lock management system as Figure 1 shown, it can also include multiple public camera devices. Among them, the public camera device means that in addition to being an independent camera device, the camera device can also be an external camera device of the intelligent door lock and provide camera services for one or more intelligent door locks. For example: the camera device can be the one at the elevator entrance and exit on the floor where it is located and used to photograph pedestrians entering and leaving the elevator, or it can be the one above the opposite side of the room entrance door and used to photograph the access situation of multiple room entrance doors. The public camera devices in a specific area can also be connected to the Internet through wired network communication technology or wireless network communication technology, and thus be connected to the above-mentioned door lock management platform through the Internet for communication and information exchange.

[0042] Similarly to the above intelligent door lock, each public camera device can also upload its own device information to the above door lock management platform for maintenance.

[0043] The device information of each public camera device maintained by the above door lock management platform can be as shown in Table 2 below:

[0044] Table 2

[0045]

[0046] It should be noted that for a public camera device, the geographical location of the public camera device can also include the rooms within the shooting range of the public camera device.

[0047] In this application, building numbers, floor numbers, room numbers, etc. can be referred to by information such as building numbers, floor numbers, room numbers, etc. (the information itself or the encoded data of the information).

[0048] In the above intelligent door lock management system, the intelligent door lock and the public camera device can both communicate and exchange information with the door lock management platform. In this case, the door lock management platform can integrate and analyze the information reported by the intelligent door lock and the public camera device, and generate corresponding control instructions to be sent to the intelligent door lock and the public camera device to control them and make them perform corresponding operations.

[0049] Therefore, through the above door lock management platform, the above intelligent door lock and the above public camera device can be linked to jointly complete the automatic unlocking of the intelligent door lock, and the security and reliability of the automatic unlocking operation for the intelligent door lock can be improved.

[0050] Based on Figure 1 , referring to Figure 2 , Figure 2 is a flowchart of an unlocking method for an intelligent door lock shown in an exemplary embodiment of this application.

[0051] As Figure 2 shown, the unlocking method for the above intelligent door lock can be applied to the intelligent door lock as shown in Figure 1 . This method can specifically include the following steps:

[0052] Step 201: When it is detected that a user enters the detection range of the intelligent door lock, send a first message to the door lock management platform, so that in response to the received first message, the door lock management platform determines the public camera device corresponding to the intelligent door lock based on the corresponding relationship between the intelligent door lock and the public camera device maintained, and sends the device information of the public camera device to the intelligent door lock.

[0053] In this embodiment, the above intelligent door lock can detect whether a user has entered the detection range of the intelligent door lock. Herein, the detection range refers to the area or distance within which a device or system can effectively identify, analyze, and respond. If it is detected that a user has entered the detection range of the intelligent door lock, since the intelligent door lock is usually deployed on the entrance door of a room, it indicates that the user is very likely a resident of the room corresponding to the intelligent door lock and wants to enter the room. At this time, the intelligent door lock can provide the user with an automatic unlocking service.

[0054] In practical applications, to ensure the timeliness of automatic unlocking, the above intelligent door lock can usually detect in real time whether a user has entered the detection range of the intelligent door lock.

[0055] When the above intelligent door lock is not equipped with a camera and does not have a shooting function, it cannot perform user confirmation through face recognition on its own, resulting in difficulty in providing a highly secure automatic unlocking service for the user relying solely on the intelligent door lock itself. In this case, the intelligent door lock can, through the above door lock management platform, link with the above public camera device to jointly complete the automatic unlocking of the intelligent door lock.

[0056] Therefore, when the above intelligent door lock detects that a user has entered the detection range of the intelligent door lock, it can send a message (which can be referred to as the first message) to the above door lock management platform to obtain the historical video recently captured by the public camera device, and determine whether an automatic unlocking operation can be performed based on this historical video.

[0057] In some embodiments, when the above intelligent door lock is not equipped with a camera and thus cannot implement structured light ranging, it can detect whether a user has entered the detection range of the intelligent door lock through a sensor carried by it for detecting the movement of an object.

[0058] Specifically, the detection range of the above intelligent door lock can be the sensing range of the sensor carried by it for detecting the movement of an object. In this case, when the intelligent door lock detects through this sensor that a user has entered the sensing range of the sensor, it can determine that the user has entered the detection range of the intelligent door lock.

[0059] In some embodiments, the above sensor for detecting the movement of an object can specifically be a Passive Infrared (PIR) sensor or a radar sensor, and their main functions are both used to detect the presence or movement of an object (usually a person or an animal) in a certain area.

[0060] Among them, the PIR sensor senses whether a warm object enters its sensing range by detecting changes in infrared radiation in the environment; when a warm object moves, it changes the infrared radiation distribution in this area, and the PIR sensor can capture this change. The sensing range of the PIR sensor refers to the maximum distance and angular range within which the sensor can detect changes in infrared radiation.

[0061] The radar sensor detects the presence and movement of an object by emitting microwaves and receiving the reflected signals; according to the Doppler effect, if the frequency of the reflected wave changes, it indicates that an object is in motion. The sensing range of the radar sensor refers to the effective distance within which the sensor emits microwaves and receives the reflected signals.

[0062] In some embodiments, the above smart door lock can be equipped with a camera, enabling the smart door lock to call the camera to implement structured light ranging. That is, if the above smart door lock is equipped with a camera, the smart door lock can perform structured light ranging on the user, obtain the distance between the user and the smart door lock, and compare this distance with a preset threshold (for example: 5 meters) to detect whether the distance between the user and the smart door lock is greater than the threshold. If the distance between the user and the smart door lock is not greater than the threshold, it can be considered that the user has entered the detection range of the smart door lock.

[0063] Structured light ranging is a three-dimensional measurement technique. It projects a specific pattern of light (usually stripes or grids) onto the target object, and then uses a camera to capture the deformed image formed by these lights on the object surface, thereby calculating the three-dimensional shape and distance information of the object. During the process of structured light ranging, first, the structured light system emits a set of known pattern of lights to the object to be measured, and this pattern is usually a set of parallel lines or a complex grid pattern; then, one or more cameras can be used to capture the light pattern projected onto the object from different angles. Due to the undulations on the object surface, the originally regular light pattern will be distorted; finally, by analyzing the differences between these deformed patterns and the original pattern, the three-dimensional contour of the object can be reconstructed, and the specific position and distance of the object relative to the camera can be calculated.

[0064] In this embodiment, when the above-mentioned door lock management platform receives the above-mentioned first message, it can respond. Specifically, based on the corresponding relationship between the maintained intelligent door locks and public camera devices, it can determine the public camera device corresponding to the intelligent door lock, and send the device information of this public camera device to the intelligent door lock. Specifically, in the corresponding relationship between the intelligent door locks and public camera devices maintained by the door lock management platform, both the intelligent door locks and public camera devices can be referred to by their device information (for example: device unique identifier, device communication address, device geographical location, etc.); the door lock management platform can, based on this corresponding relationship, according to the device information of the intelligent door lock, determine the device information of the public camera device corresponding to the device information of this intelligent door lock, and send the device information of this public camera device to the intelligent door lock.

[0065] It should be noted that the above-mentioned door lock management platform may not maintain a direct corresponding relationship between the intelligent door locks and public camera devices. In this case, the intelligent door locks and public camera devices that match the building, floor, and room where they are located can be used as the corresponding intelligent door locks and public camera devices. For example, assuming that the geographical location of an intelligent door lock includes building 1, unit 2, floor 10, room 1002, and the geographical location of a public camera device includes building 1, unit 2, floor 10, rooms 1001 and 1002, then it can be considered that the building, floor, and room where this intelligent door lock and this public camera device are located match, and thus this intelligent door lock and this public camera device can be used as the corresponding intelligent door lock and public camera device.

[0066] In some embodiments, in order to reduce the power consumption of the above-mentioned intelligent door lock and extend the operable duration of the intelligent door lock under the same power, the default state of the intelligent door lock can be the sleep state, that is, it is by default in a low-power, low-activity standby mode, temporarily stopping the operation of most functions, but retaining basic data and settings.

[0067] For example, when the above-mentioned intelligent door lock is in the sleep state, its network function can also stop running. When the intelligent door lock detects that a user enters the detection range of the intelligent door lock, it can wake up the network module (for example: WiFi module) in the intelligent door lock to achieve the normal operation of the network function.

[0068] In the above case, when the above-mentioned intelligent door lock detects that a user enters the detection range of the intelligent door lock, it can switch from the sleep state to the running state, that is, switch to the normal working mode, all functions can be used, and corresponding tasks can be executed, so that the intelligent door lock can send the above-mentioned first message to the above-mentioned door lock management platform.

[0069] After the unlocking operation is completed, the above-mentioned intelligent door lock can switch back from the running state to the sleep state.

[0070] Step 202: In response to the received device information, based on the device information, send a second message to the public camera device, so that in response to the received second message, the public camera device acquires a historical video of a preset duration captured by it and sends the historical video to the intelligent door lock.

[0071] In this embodiment, when the above intelligent door lock receives the device information of the above public camera device, it can respond. Specifically, based on the device information, it can send a message (which can be called a second message) to the public camera device.

[0072] When the above public camera device receives the above second message, it can respond. Specifically, it can acquire a historical video of a preset duration captured by the public camera device and send the historical video to the above intelligent door lock.

[0073] In some embodiments, the above intelligent door lock can send the second message to the above door lock management platform, and the door lock management platform forwards the second message to the public camera device according to the device information of the public camera device included in the second message.

[0074] In some embodiments, in order to shorten the data transmission link and improve the data transmission efficiency, the device information of the above public camera device may include the communication address of the public camera device (for example: IP address, port number, MAC address, etc.). In this case, since both the above intelligent door lock and the above public camera device are connected to the Internet, the above intelligent door lock can directly send the second message to the public camera device according to the communication address.

[0075] Alternatively, the above intelligent door lock can re - establish a communication connection with the above public camera device according to the communication address of the public camera device. When a communication connection is established between the intelligent door lock and the public camera device, the intelligent door lock can directly send the second message to the public camera device through the communication connection; and the public camera device can, in response to the received second message, send the stored historical video of a preset duration to the intelligent door lock through the communication connection.

[0076] In some embodiments, since the storage space of the public camera device itself is usually relatively limited and it is unable to store historical videos with a long duration, to ensure the traceability of the historical videos captured by the public camera device, the public camera device can regularly upload the historical videos it has captured to the above-mentioned door lock management platform for storage. For example, the public camera device can upload the historical video it captured from 16:00 to 16:15 on a certain day to the door lock management platform for storage at 16:15 on that day; at 16:30 on that day, upload the historical video it captured from 16:15 to 16:30 on that day to the door lock management platform for storage; at 16:45 on that day, upload the historical video it captured from 16:30 to 16:45 on that day to the door lock management platform for storage; and so on.

[0077] In the above case, when the duration of the historical video stored by the public camera device is not less than the above-mentioned preset duration, the public camera device can extract the historical video of the preset duration that is closest to the current moment in terms of the shooting time period from the historical videos it stores, and send the extracted historical video to the above-mentioned intelligent door lock. For example, assume that the historical video stored by the public camera device is for the shooting time period from 16:30 to 16:40 on a certain day, and assume that the preset duration is 5 minutes, then the public camera device can send the historical video for the shooting time period from 16:35 to 16:40 on that day to the intelligent door lock.

[0078] When the duration of the historical video stored by the public camera device is less than the above-mentioned preset duration, the public camera device can request the door lock management platform to obtain a part of the historical video, and merge this part of the historical video with the historical video stored by the public camera device, so that the merged historical video can be sent to the above-mentioned intelligent door lock. For example, assume that the historical video stored by the public camera device is for the shooting time period from 16:30 to 16:33 on a certain day, and assume that the preset duration is 5 minutes, then the public camera device can request the door lock management platform to obtain the historical video for the shooting time period from 16:28 to 16:30 on that day, and merge the historical video obtained from the door lock management platform with the historical video stored by the public camera device to obtain the historical video for the shooting time period from 16:28 to 16:33 on that day, and then send the merged historical video to the intelligent door lock.

[0079] Specifically, when the above-mentioned public camera device determines that the duration of the historical video stored by itself is less than the above-mentioned preset duration, it can send a message (which can be called the third message) to the above-mentioned door lock management platform. The third message includes the difference between the duration of the historical video stored by the public camera device and the preset duration, so that the door lock management platform can extract from the historical videos it stores the historical video with the closest shooting time period to the current moment and a duration equal to the difference, and send the extracted historical video to the public camera device.

[0080] Step 203: In response to the received historical video, perform face detection on the video frames in the historical video to obtain target face information; among them, the video frame containing the target face information is the video frame with the latest shooting moment among the video frames containing face information.

[0081] In this embodiment, when the above-mentioned intelligent door lock receives the above-mentioned historical video, it can make a response. Specifically, it can perform face detection on the video frames in the historical video. For the video frames in the historical video, the face information detected from the video frame with the latest shooting moment (i.e., the shooting moment closest to the current moment) among the video frames containing face information can be used as the target face information.

[0082] For example, assume that the video frames included in the above-mentioned historical video are in the order of shooting time from early to late: video frame 1, video frame 2, video frame 3, video frame 4, and video frame 5. After performing face detection on these 5 video frames respectively, it is found that face information can be detected from video frame 2, video frame 3, and video frame 4. In this case, since the shooting moment of video frame 4 is the latest, the face information detected from video frame 4 can be used as the above-mentioned target face information.

[0083] In some embodiments, in order to reduce the number of video frames for which face detection is actually performed, the above-mentioned intelligent door lock can start from the video frame with the latest shooting moment in the above-mentioned historical video and perform face detection on the video frames in the historical video in the order of shooting time from late to early until face information is detected, so that the detected face information can be used as the above-mentioned target face information.

[0084] Continuing with the above example, the above-mentioned intelligent door lock can first perform face detection on video frame 5. Since no face information is detected from video frame 5, it can continue to perform face detection on video frame 4. Since face information can be detected from video frame 4, the face information detected from video frame 4 can be used as the above-mentioned target face information. After face information is detected from video frame 4, face detection does not need to be performed on video frame 3, video frame 2, and video frame 1 anymore.

[0085] In some cases, when the above intelligent door lock performs face detection on the video frames in the above historical video starting from the video frame with the latest shooting time in the above historical video and in the order from the latest to the earliest shooting time, if no face information can be detected all the time, the face detection can be stopped when the time interval between the video frames for which face detection has been completed and the video frame with the latest shooting time is not less than a threshold value (for example: 2 minutes).

[0086] In practical applications, starting from the video frame with the latest shooting time in the above historical video and performing face detection on the video frames in the above historical video in the order from the latest to the earliest shooting time can be regarded as playing back the above historical video in reverse and performing face detection on the video frames in the above historical video in the reverse order. For example, play back at most 2 minutes of the historical video and perform face detection on the video frames in this 2-minute-long historical video in the reverse order.

[0087] The specific value of the above threshold can be determined by the time required for the user to go from the elevator entrance of the floor where the room corresponding to the above intelligent door lock is located to the entrance door of the room corresponding to the intelligent door lock. For example, assume that the room corresponding to the intelligent door lock is Room 502 and the floor where the room is located is the 5th floor. Then the specific value of the threshold can be determined by the time required for the user to go from the elevator entrance on the 5th floor to the entrance door of Room 502. The threshold can be set according to general experience, and the same threshold is set for different users; or, the threshold can also be set according to the user's own walking speed, and different thresholds are set for different users.

[0088] It should be noted that when the face detection is stopped, if no face information has been detected yet, the above intelligent door lock can directly refuse to perform the unlocking operation, that is, the intelligent door lock can remain in the locked state. Or, the above target face information can be determined to be an empty set; the target face information being an empty set means that the target face information will surely not match the face information of the users who can unlock the door maintained by the intelligent door lock.

[0089] Step 204: Match the target face information with the face information of the users who can unlock the door maintained, and perform the unlocking operation when the target face information matches the face information of the users who can unlock the door.

[0090] In this embodiment, when the above intelligent door lock obtains the above target face information, it can match the target face information with the face information of the users who can unlock the door maintained by the intelligent door lock, and perform the unlocking operation when the target face information matches the face information of the users who can unlock the door.

[0091] Specifically, when the above intelligent door lock maintains the face information of multiple unlockable users, the above target face information can be respectively matched with the face information of each of the unlockable users, and when the target face information is matched with the face information of any one of the unlockable users, the unlocking operation is performed.

[0092] In practical applications, as described above, the face information can be a feature vector extracted from a face image. In this case, for the above target face information and the face information of a certain unlockable user, if the vector similarity between the feature vector as the target face information and the feature vector as the face information of this unlockable user is not less than a preset threshold (for example: 95%), it can be considered that the target face information is matched with the face information of this unlockable user.

[0093] It should be noted that the above intelligent door lock can call the camera it is equipped with to take pictures of the above unlockable users and perform face detection on the taken pictures, so that the detected face information can be used as the face information of this unlockable user, thereby completing the entry of the face information of the unlockable user at the intelligent door lock.

[0094] In some embodiments, the shooting range of the above public camera device can be non-fixed. For example, the base of the public camera device can be rotatable, so that the lens direction of the public camera device can change, thereby changing the shooting range of the public camera device.

[0095] In practical applications, the above public camera device can independently maintain the correspondence between the device position and the shooting range, so that when the public camera device needs to adjust its shooting range to a specific shooting range, it can be adjusted according to the set position corresponding to the specific shooting range. Among them, the device position can include the base rotation angle and / or lens direction of the public camera device, etc., and the shooting range can be represented by elevator number, floor number, room number, and / or intelligent door lock number, etc. For example, assume that when the base rotation angle of the public camera device is 0 degrees, the shooting range of the public camera device is the elevator entrance and exit on the 5th floor, when the base rotation angle of the public camera device is 20 degrees, the shooting range of the public camera device is Room 501, and when the base rotation angle of the public camera device is 40 degrees, the shooting range of the public camera device is Room 502. Then, when the public camera device needs to adjust its shooting range to Room 501, it can control the rotation angle of its base to be adjusted to 20 degrees, and when it needs to adjust its shooting range to Room 502, it can control the rotation angle of its base to be adjusted to 40 degrees.

[0096] The default shooting range of the above-mentioned public camera device can be the elevator entrance of the floor where the above-mentioned intelligent door lock is located (i.e., the floor where the room corresponding to the intelligent door lock is located).

[0097] To improve the reliability and security of the automatic unlocking operation for the intelligent door lock, while obtaining the above-mentioned target face information by performing face detection on the video frames in the above-mentioned historical video, the above-mentioned intelligent door lock can also obtain portrait auxiliary information corresponding to the target face information from the video frame. For example, while detecting the target face information from a certain video frame, the portrait area in the video frame can also be structurally analyzed to obtain portrait auxiliary information as the portrait auxiliary information corresponding to the target face information. Among them, the portrait auxiliary information can include the color of the clothes worn by the person, the shape of the accessories worn by the person, etc.

[0098] Correspondingly, when the above-mentioned intelligent door lock obtains the above-mentioned target face information and its corresponding portrait auxiliary information, it can send a message (which can be called the third message) to the above-mentioned public camera device. When receiving the third message, the public camera device can respond. Specifically, it can first adjust its shooting range to the room corresponding to the intelligent door lock (specifically, it can be the room entrance door), and then, after the shooting range adjustment is completed, perform a shooting operation, that is, shoot the room entrance door corresponding to the intelligent door lock and send the captured image to the intelligent door lock.

[0099] The above-mentioned intelligent door lock can respond to the received captured image, obtain portrait auxiliary information from the image, and match the portrait auxiliary information obtained at this time with the portrait auxiliary information corresponding to the above-mentioned target face information obtained previously. If the two match, and the target face information matches the face information of the unlockable user maintained by the intelligent door lock, it can be confirmed that the user currently located at the entrance door where the intelligent door lock is located is an unlockable user, so the intelligent door lock can perform an unlocking operation.

[0100] In some embodiments, to reduce the matching workload of the face information and the portrait auxiliary information, when receiving the above-mentioned third message sent by the intelligent door lock, the above-mentioned public camera device can respond. Specifically, it can first adjust its shooting range to the room corresponding to the intelligent door lock (specifically, it can be the room entrance door), and then, after the shooting range adjustment is completed, perform a shooting operation and send a message (which can be called the fourth message) to the intelligent door lock. The fourth message can be used to notify the intelligent door lock that the shooting range adjustment of the public camera device has been completed.

[0101] The above intelligent door lock can, in response to receiving the above fourth message, obtain in real time from the above public camera device the video frames captured by the public camera device. For example, the intelligent door lock can send a message for requesting to obtain real-time video to the public camera device, so that the public camera device can, in response to receiving the message, send the video frames captured by itself to the intelligent door lock in real time.

[0102] When the above intelligent door lock obtains in real time from the above public camera device the video frames captured by the public camera device, it can first perform human face detection on the video frames to determine whether there is a human face in the video frames, and when it is determined that the video frames contain a human face, obtain human face auxiliary information from the video frames. Subsequently, the human face auxiliary information obtained at this time can be matched with the human face auxiliary information corresponding to the above target face information obtained previously. If the two match, and the target face information matches the face information of the unlockable user maintained by the intelligent door lock, it can be confirmed that the user currently located at the entrance door where the intelligent door lock is located is an unlockable user, so the intelligent door lock can perform an unlocking operation.

[0103] In the above technical solution, the intelligent door lock can first, when detecting that a user enters its detection range, send a first message to the door lock management platform for uniformly managing the intelligent door lock. The door lock management platform, in response to receiving the first message, determines the public camera device corresponding to the intelligent door lock based on the corresponding relationship between the intelligent door lock and the public camera device maintained by the door lock management platform, and returns the device information of the public camera device. Then, in response to receiving the device information, based on the device information, it sends a second message to the public camera device. The public camera device, in response to receiving the second message, obtains the historical video of a preset duration captured by it and returns the historical video. Then, in response to receiving the historical video, it performs human face detection on the video frames in the historical video to obtain the target face information contained in the video frame with the latest shooting time among the video frames containing face information. Finally, it matches the target face information with the face information of the unlockable user maintained by the intelligent door lock, and when the target face information matches the face information of the unlockable user, performs an unlocking operation.

[0104] By adopting the above method, the intelligent door lock, the public camera device and the door lock management platform can be linked to jointly complete the automatic unlocking of the intelligent door lock. In this process, the intelligent door lock can obtain the face information of the user through the historical video captured by the public camera device, and when the face information of the user matches the face information of the unlockable user maintained by the intelligent door lock, perform an unlocking operation for the user. In this way, while providing convenience for the user through automatic unlocking and avoiding the user unlocking manually, the safety and reliability of the automatic unlocking operation for the intelligent door lock can be improved.

[0105] Based on Figure 1 , with reference to Figure 3 , Figure 3 FIG. is a schematic diagram of another intelligent door lock management system shown in an exemplary embodiment of the present application.

[0106] As shown in Figure 3 , in order to distinguish devices such as intelligent door locks and public camera devices (or systems, collectively referred to as devices in the present application) in different regions, avoid unnecessary data transmission between devices in different regions, and facilitate the management of these devices in the same region, multicast can be used. Intelligent door locks and public camera devices in the same region can join the same multicast group. Further, a routing device supporting the multicast function (such as a router, a layer 3 switch, etc.) can be used to manage and forward the multicast data between the door lock management platform and each multicast group. This routing device can be directly connected to the door lock management platform and each device in the multicast group respectively.

[0107] In practical applications, in order to reduce the deployment cost of the door lock management platform, the door lock management platform can be directly deployed on the above-mentioned routing device. That is, this routing device can not only provide the multicast function, but also realize the unified management of intelligent door locks through the deployed door lock management platform.

[0108] Taking one of the multicast groups as an example, a suitable IP address can be assigned to this multicast group as the communication address of this multicast group, so that the above-mentioned door lock management platform can use this communication address to send multicast data to this multicast group. In practical applications, each device in this multicast group can also use this communication address to send multicast data to other devices in this multicast group. Each intelligent door lock or each public camera device that needs to join this multicast group can indicate that the device hopes to join this multicast group by sending an IGMP (Internet Group Management Protocol) join message to the routing device corresponding to this multicast group. This IGMP join message can include the device information of the device (such as device unique identifier, device geographical location, device type, etc.); after receiving this IGMP join message, this routing device can record the corresponding relationship between the device information of this device and the interface in this routing device used to forward multicast data to this device (usually the interface used to receive this IGMP join message) in the multicast forwarding table maintained inside. And when this device no longer needs to receive multicast data, it can send an IGMP leave message to this routing device, so that this routing device stops forwarding multicast data to this device.

[0109] The correspondence between the device information of each device in the multicast group maintained by the above routing device and the interface can be as shown in Table 3 below:

[0110] Table 3

[0111]

[0112] It should be noted that although Figure 3 only one routing device is shown between the above door lock management platform and the multicast group joined by the above intelligent door lock, in fact, in a multicast network, multiple routing devices can participate in the forwarding of multicast data. These routing devices all support and participate in the multicast routing protocol (such as Protocol Independent Multicast, i.e., PIM).

[0113] Specifically, a multicast network can be divided into multiple subnets. For example, assume that the intelligent door locks corresponding to all rooms in a community (e.g., the intelligent door locks deployed on the entrance doors of the rooms) join the same multicast group, forming a multicast network. Then, in this multicast network, the subnets can be divided according to the buildings, that is, the network formed by the intelligent door locks corresponding to the rooms in the same building is taken as a subnet. In a subnet, there is at least one local multicast routing device, which is responsible for receiving the multicast data from the upstream and forwarding the multicast data to all multicast members in this subnet. It should be noted that if there is only one local multicast routing device in a subnet, then this routing device is directly connected to the devices that are multicast members in this subnet. The upstream multicast routing device refers to the multicast routing device that is closer to the multicast source or Rendezvous Point (RP) relative to the local multicast routing device; they are responsible for receiving the multicast data from the multicast source or RP and forwarding the multicast data to the downstream multicast routing devices according to the multicast routing protocol.

[0114] In a multicast network, the flow direction of multicast data usually includes: the multicast source sends data to the multicast routing device, the multicast routing devices forward the data to each other, reach the target multicast routing device, and the target multicast routing device sends data to the multicast members. Specifically, the multicast data is first sent by the multicast source to the multicast routing device directly connected to this multicast source; then, through the multicast routing protocol, these multicast data are forwarded to other multicast routing devices that need to receive this multicast stream. In this process, a multicast distribution tree is formed, which can be a source-based (*, G) or specific-source (S, G) tree, where S represents the multicast source and G represents the communication address of the multicast group; then, the multicast data will be forwarded to the target multicast routing device located at the edge of the target subnet; finally, the target multicast routing device forwards the received multicast data to the multicast members in the target subnet.

[0115] When constructing an arbitrary source multicast tree of (*, G), this process allows multicast data from any source to be sent to the multicast address represented by G, and dynamically establishes and maintains a multicast distribution path through the PIM protocol. This means that any routing device can indicate its desire to join this multicast group by sending a PIM join message, thereby ensuring that member devices in the subnet where the routing device is located can receive the corresponding multicast data. After receiving the PIM join message, the upstream multicast routing device will correspondingly update its multicast forwarding table to ensure that the multicast data can be forwarded along the correct path.

[0116] Based on Figure 3 and referring to Figure 4 , Figure 4 is a flowchart of another unlocking method for an intelligent door lock shown in an exemplary embodiment of the present application.

[0117] As Figure 4 shown, the above unlocking method for an intelligent door lock can be applied to an intelligent door lock as shown in Figure 3 . The method may specifically include the following steps:

[0118] Step 401: When detecting that a user enters the detection range of the intelligent door lock, send a first message to the door lock management platform, so that in response to the received first message, the door lock management platform determines a public camera device corresponding to the intelligent door lock based on the maintained correspondence between the intelligent door lock and the public camera device, and sends the device information of the public camera device to the intelligent door lock.

[0119] Step 402: In response to the received device information, send a second message to a routing device corresponding to the multicast group according to the communication address of the multicast group. The routing device determines an interface corresponding to the public camera device according to the device information included in the second message, and sends the second message to the public camera device through the interface, so that in response to the received second message, the public camera device acquires historical video of a preset duration captured by it and sends the historical video to the intelligent door lock.

[0120] Step 403: In response to the received historical video, perform face detection on the video frames in the historical video to obtain target face information; among them, the video frame containing the target face information is the video frame with the latest shooting time among the video frames containing face information.

[0121] Step 404: Match the target face information with the face information of the unlockable users maintained, and perform an unlocking operation when the target face information matches the face information of the unlockable users.

[0122] In this embodiment, the above-mentioned intelligent door lock and the above-mentioned public camera device can join the same multicast group; the routing device corresponding to the multicast group can maintain the correspondence between the device information of each device in the multicast group and the interface used to forward multicast data to the device.

[0123] In the above case, when the above-mentioned intelligent door lock sends a second message to the above-mentioned public camera device based on the device information of the above-mentioned public camera device, specifically, it can send the second message to the routing device corresponding to the multicast group according to the communication address of the multicast group. The routing device determines the interface corresponding to the public camera device according to the device information included in the second message, and sends the second message to the public camera device through the interface.

[0124] Correspondingly, since in bidirectional data transmission, the data transmission paths in both directions can be the same, the above-mentioned public camera device can send the above-mentioned historical video to the above-mentioned routing device through the above-mentioned interface, and the routing device forwards the historical video to the above-mentioned intelligent door lock.

[0125] It should be noted that when the duration of the historical video stored in the above-mentioned public camera device is less than the above-mentioned preset duration, a second message can be sent to the above-mentioned door lock management platform through the above-mentioned routing device. The second message includes the difference between the duration of the historical video stored in the public camera device and the preset duration, so that the door lock management platform can extract from the historical videos stored by it the historical video with the closest shooting time period to the current moment and a duration equal to the difference, and according to the communication address of the above-mentioned multicast group, send the extracted historical video to the routing device. The routing device determines the interface corresponding to the public camera device according to the device information of the public camera device, and sends the historical video to the public camera device through the interface.

[0126] And so on, when the above-mentioned door lock management platform, any intelligent door lock or any public camera device needs to send specific data to any one or more devices (which can be called target devices) in the above-mentioned multicast group, it can send the data and the device information of the target device (for example: the floor where it is located, the corresponding room, the device type, the identification information, etc.) to the routing device corresponding to the multicast group according to the communication address of the multicast group. The routing device determines the interface corresponding to the target device according to the device information, and sends the data to the target device through the interface.

[0127] Such as Figure 4 shown, the specific implementation manner of the embodiment can refer to the embodiment shown in Figure 2 shown, and this application will not elaborate here.

[0128] Specifically, refer to Figure 5 ,Figure 5 It is a multi - party interaction flowchart of an unlocking method for an intelligent door lock shown in an exemplary embodiment of the present application.

[0129] As Figure 5 shown, the unlocking method of the above - mentioned intelligent door lock may include the following steps:

[0130] Step 501: The intelligent door lock detects whether the user enters the detection range of the intelligent door lock.

[0131] Step 502: If it is detected that the user enters the detection range of the intelligent door lock, the intelligent door lock switches from the sleep state to the running state.

[0132] Step 503: The intelligent door lock sends a first message to the door lock management platform.

[0133] Step 504: The door lock management platform determines the public camera device corresponding to the intelligent door lock based on the corresponding relationship between the intelligent door lock and the public camera device maintained.

[0134] Step 505: The door lock management platform sends the device information of the public camera device to the intelligent door lock.

[0135] Step 506: The intelligent door lock sends a second message to the routing device corresponding to the multicast group according to the communication address including the intelligent door lock and the public camera device multicast group.

[0136] Step 507: The routing device determines the interface corresponding to the public camera device according to the device information included in the second message.

[0137] Step 508: The routing device sends the second message to the public camera device through the interface.

[0138] Step 509: The public camera device detects whether the duration of the stored historical video is not less than the preset duration.

[0139] Step 510: If the duration of the stored historical video is not less than the preset duration, the public camera device acquires the historical video of the preset duration that is closest to the current moment among the stored shooting time periods.

[0140] Step 511: If the duration of the stored historical video is less than the preset duration, the public camera device sends a third message to the door lock management platform through the routing device.

[0141] Step 512: The door lock management platform acquires the historical video of the duration equal to the difference between the duration of the historical video stored in the public camera device and the preset duration that is closest to the current moment among the stored shooting time periods.

[0142] Step 513: The door lock management platform sends the historical video to the routing device according to the communication address of the multicast group.

[0143] Step 514: The routing device determines the interface corresponding to the public camera device according to the device information of the public camera device.

[0144] Step 515: The routing device sends the historical video to the public camera device through the interface.

[0145] Step 516: The public camera device merges the received historical video with the stored historical video to obtain the merged historical video.

[0146] Step 517: The public camera device sends the historical video to the smart door lock through the routing device.

[0147] Step 518: The smart door lock starts from the video frame with the latest shooting time in the historical video, and performs face detection on the video frames in the historical video in the order from the latest to the earliest shooting time to obtain the target face information; wherein, the video frame containing the target face information is the video frame with the latest shooting time among the video frames containing face information.

[0148] Step 519: The smart door lock matches the target face information with the face information of the unlockable users it maintains.

[0149] Step 520: When the target face information matches the face information of the unlockable users, the smart door lock performs an unlocking operation.

[0150] As Figure 5 shown, the specific implementation manners of the embodiments can refer to the embodiments as Figure 2 and Figure 4 shown, and the present application will not elaborate herein.

[0151] Corresponding to the embodiments of the foregoing method, the present application also provides embodiments of a device.

[0152] Refer to Figure 6 , Figure 6It is a schematic structural diagram of a device shown in an exemplary embodiment of the present application. At the hardware level, the device includes a processor 601, an internal bus 602, a network interface 603, a memory 604, and a non-volatile memory 605. Of course, other required hardware may also be included. One or more embodiments of the present application may be implemented in a software manner. For example, the processor 601 reads the corresponding computer program from the non-volatile memory 605 into the memory 604 and then runs it. Of course, in addition to the software implementation manner, one or more embodiments of the present application do not exclude other implementation manners, such as a logic device or a combination of software and hardware. That is to say, the execution subject of the following processing flow is not limited to each logic module, but may also be hardware or a logic device.

[0153] Reference Figure 7 , Figure 7 It is a block diagram of an unlocking device of an intelligent door lock shown in an exemplary embodiment of the present application.

[0154] The above-mentioned unlocking device of the intelligent door lock can be applied to Figure 6 the device shown in the figure. The device can be an intelligent door lock to implement the technical solution of the present application.

[0155] As Figure 7 shown, the above-mentioned unlocking device of the intelligent door lock can specifically include:

[0156] A device acquisition module 701, configured to send a first message to a door lock management platform when detecting that a user enters the detection range of the intelligent door lock, so that the door lock management platform, in response to the received first message, determines a public camera device corresponding to the intelligent door lock based on the maintained correspondence between the intelligent door lock and the public camera device, and sends the device information of the public camera device to the intelligent door lock;

[0157] A video acquisition module 702, configured to send a second message to the public camera device based on the received device information in response to the received device information, so that the public camera device, in response to the received second message, acquires a historical video of a preset duration captured by it and sends the historical video to the intelligent door lock;

[0158] A face detection module 703, configured to perform face detection on video frames in the historical video in response to the received historical video to obtain target face information; wherein, the video frame containing the target face information is the video frame with the latest shooting time among the video frames containing face information.

[0159] A face matching module 704, configured to match the target face information with the face information of the unlockable users maintained, and perform an unlocking operation when the target face information matches the face information of the unlockable users.

[0160] In some embodiments, the intelligent door lock is equipped with a sensor for detecting the movement of an object;

[0161] The device further includes:

[0162] A user detection module, configured to determine that the user enters the detection range of the intelligent door lock when it is detected by the sensor that the user enters the sensing range of the sensor.

[0163] In some embodiments, when the duration of the historical video stored by the public camera device is not less than the preset duration, the historical video is the historical video stored by the public camera device;

[0164] When the duration of the historical video stored by the public camera device is less than the preset duration, the historical video is the historical video obtained by merging the historical video stored by the public camera device and the historical video uploaded by the public camera device to the door lock management platform for storage.

[0165] In some embodiments, the performing face detection on video frames in the historical video to obtain target face information includes:

[0166] Starting from the video frame with the latest shooting time in the historical video, performing face detection on the video frames in the historical video in the order from the latest to the earliest shooting time until face information is detected, and determining the detected face information as the target face information; or,

[0167] Starting from the video frame with the latest shooting time in the historical video, performing face detection on the video frames in the historical video in the order from the latest to the earliest shooting time until the shooting time interval between the video frames with face detection completed and the video frame with the latest shooting time is not less than a preset threshold, and determining that no face information is detected.

[0168] In some embodiments, the performing face detection on video frames in the historical video to obtain target face information includes:

[0169] Performing face detection on the video frames in the historical video to obtain target face information, and obtaining portrait auxiliary information corresponding to the target face information from the video frames;

[0170] The device further includes:

[0171] An information matching module, configured to send a third message to the public camera device before performing an unlocking operation, so that the public camera device performs a shooting operation in response to receiving the third message and sends the captured image to the intelligent door lock;

[0172] In response to the received image, obtain portrait auxiliary information from the image;

[0173] Match the portrait auxiliary information with the portrait auxiliary information corresponding to the target face information, and perform an unlocking operation when the portrait auxiliary information matches the portrait auxiliary information corresponding to the target face information.

[0174] In some embodiments, the sending the third message to the public camera device, so that the public camera device performs a shooting operation in response to receiving the third message and sends the captured image to the intelligent door lock includes:

[0175] Send a third message to the public camera device, so that the public camera device performs a shooting operation in response to receiving the third message and sends the captured video frame to the intelligent door lock in real time;

[0176] The obtaining portrait auxiliary information from the image in response to the received image includes:

[0177] In response to the received video frame, perform portrait detection on the video frame, and when it is determined that the video frame contains a portrait, obtain portrait auxiliary information from the video frame.

[0178] In some embodiments, the sending a first message to the door lock management platform when detecting that a user enters the detection range of the intelligent door lock includes:

[0179] When detecting that a user enters the detection range of the intelligent door lock, switch from the sleep state to the running state to send a first message to the door lock management platform.

[0180] In some embodiments, the device information includes a communication address;

[0181] The sending the second message to the public camera device based on the device information includes:

[0182] Send the second message to the public camera device according to the communication address.

[0183] In some embodiments, the intelligent door lock and the public camera device join the same multicast group; the routing device corresponding to the multicast group maintains the correspondence between device information and interfaces;

[0184] Sending a second message to the public camera device based on the device information includes:

[0185] According to the communication address of the multicast group, the second message is sent to the routing device corresponding to the multicast group. The routing device determines the interface corresponding to the public camera device based on the device information included in the second message, and sends the second message to the public camera device through the interface.

[0186] In some embodiments, the correspondence between the device information and the interface is created by the routing device according to the device information included in the IGMP join messages sent by each device in the multicast group and the interface in the routing device for receiving the IGMP join messages.

[0187] Corresponding to the embodiments of the foregoing method, the present application also provides embodiments of a system.

[0188] An unlocking system for an intelligent door lock, which specifically may include a door lock management platform, an elevator control system, an information acquisition device, and an intelligent door lock; wherein:

[0189] When the intelligent door lock detects that a user enters the detection range of the intelligent door lock, it sends a first message to the door lock management platform;

[0190] In response to the received first message, the door lock management platform determines the public camera device corresponding to the intelligent door lock based on the correspondence between the intelligent door lock and the public camera device maintained, and sends the device information of the public camera device to the intelligent door lock;

[0191] In response to the received device information, the intelligent door lock sends a second message to the public camera device based on the device information;

[0192] In response to the received second message, the public camera device acquires the historical video of a preset duration captured by it and sends the historical video to the intelligent door lock;

[0193] In response to the received historical video, the intelligent door lock performs face detection on the video frames in the historical video to obtain target face information; wherein, the video frame containing the target face information is the video frame with the latest shooting time among the video frames containing face information.

[0194] The intelligent door lock matches the target face information with the face information of the users who can unlock the door, and performs an unlocking operation when the target face information matches the face information of the users who can unlock the door.

[0195] In some embodiments, the smart door lock is equipped with a sensor for detecting the movement of an object;

[0196] The smart door lock also performs:

[0197] When it is detected by the sensor that the user enters the sensing range of the sensor, it is determined that the user enters the detection range of the smart door lock.

[0198] In some embodiments, when the duration of the historical video stored by the public camera device is not less than the preset duration, the historical video is the historical video stored by the public camera device;

[0199] When the duration of the historical video stored by the public camera device is less than the preset duration, the historical video is the historical video obtained by merging the historical video stored by the public camera device and the historical video uploaded by the public camera device to the door lock management platform for storage.

[0200] In some embodiments, the smart door lock specifically performs:

[0201] Starting from the video frame with the latest shooting time in the historical video, in the order of shooting time from late to early, perform face detection on the video frames in the historical video until face information is detected, and determine the detected face information as the target face information; or,

[0202] Starting from the video frame with the latest shooting time in the historical video, in the order of shooting time from late to early, perform face detection on the video frames in the historical video until the shooting time interval between the video frame with the completed face detection and the video frame with the latest shooting time is not less than a preset threshold, and determine that no face information is detected.

[0203] In some embodiments, the smart door lock specifically performs:

[0204] Perform face detection on the video frames in the historical video to obtain target face information, and obtain portrait auxiliary information corresponding to the target face information from the video frames;

[0205] The smart door lock also performs:

[0206] Before performing the unlocking operation, send a third message to the public camera device;

[0207] The public camera device also performs:

[0208] In response to the received third message, perform a shooting operation and send the captured image to the smart door lock;

[0209] The smart door lock also performs:

[0210] In response to the received image, obtain portrait auxiliary information from the image;

[0211] Match the portrait auxiliary information with the portrait auxiliary information corresponding to the target face information, and perform an unlocking operation when the portrait auxiliary information matches the portrait auxiliary information corresponding to the target face information.

[0212] In some embodiments, the public camera device specifically performs:

[0213] In response to the received third message, perform a shooting operation and send the captured video frame to the smart door lock in real time;

[0214] The smart door lock specifically performs:

[0215] In response to the received video frame, perform portrait detection on the video frame, and when it is determined that the video frame contains a portrait, obtain portrait auxiliary information from the video frame.

[0216] In some embodiments, the smart door lock specifically performs:

[0217] When it is detected that the user enters the detection range of the smart door lock, switch from the sleep state to the running state to send a first message to the door lock management platform.

[0218] In some embodiments, the device information includes a communication address;

[0219] The smart door lock specifically performs:

[0220] According to the communication address, send a second message to the public camera device.

[0221] In some embodiments, the smart door lock and the public camera device join the same multicast group; the system further includes a routing device corresponding to the multicast group; the routing device corresponding to the multicast group maintains the correspondence between device information and interfaces;

[0222] The smart door lock specifically performs:

[0223] According to the communication address of the multicast group, send a second message to the routing device corresponding to the multicast group;

[0224] The routing device performs:

[0225] According to the device information included in the second message, determine the interface corresponding to the public camera device, and send the second message to the public camera device through the interface.

[0226] In some embodiments, the correspondence between the device information and the interfaces is created by the routing device according to the device information included in the IGMP join messages sent by each device in the multicast group and the interfaces in the routing device for receiving the IGMP join messages.

[0227] For the apparatus embodiments and system embodiments, they basically correspond to the method embodiments. Therefore, for the related parts, refer to the partial descriptions of the method embodiments. The apparatus embodiments and system embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed to multiple network modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the technical solution of this application.

[0228] The systems, apparatuses, modules or units illustrated in the above embodiments can be specifically implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, and the specific form of the computer can be a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email receiving and sending device, a game console, a tablet computer, a wearable device or a combination of any several of these devices.

[0229] In a typical configuration, a computer includes one or more processors (CPUs), an input / output interface, a network interface, and a memory.

[0230] The memory may include non-permanent memory in the computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of, for example, read-only memory (ROM) or flash memory (flash RAM). The memory is an example of the computer-readable medium.

[0231] A computer-readable medium includes both permanent and non-permanent, removable and non-removable media and can store information by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory, or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD), or other optical storage, magnetic cassettes, magnetic disk storage, quantum memory, graphene-based storage media, or other magnetic storage devices, or any other non-transitory media that can store information accessible by a computing device. As defined herein, a computer-readable medium does not include transitory media such as modulated data signals and carrier waves.

[0232] It should be noted that the term "comprising", "including", or any other variation thereof is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising a..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0233] The above describes specific embodiments of the present application. Other embodiments are within the scope of the present application. In some cases, the acts or steps recited in the present application may be performed in a different order than in the embodiments and still achieve the desired results. Additionally, the processes depicted in the figures do not necessarily require the particular order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0234] The terms used in one or more embodiments of the present application are for the purpose of describing particular embodiments only and are not intended to limit one or more embodiments of the present application. The singular forms "a", "the", and "that" are also intended to include the plural forms unless the context clearly dictates otherwise. The term "and / or" means and includes any and all possible combinations of one or more of the associated listed items.

[0235] The descriptions of terms such as "one embodiment", "some embodiments", "example", "specific example", or "a kind of implementation manner" used in one or more embodiments of this application mean that the specific features or characteristics described in connection with the embodiment are included in at least one embodiment of this application. The schematic descriptions of these terms do not necessarily refer to the same embodiment. Moreover, the specific features or characteristics described can be combined in a suitable manner in one or more embodiments of this application. In addition, different embodiments and the specific features or characteristics in different embodiments can be combined without contradiction.

[0236] It should be understood that although the terms first, second, third, etc. may be used in one or more embodiments of this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "when" or "while" or "in response to determining".

[0237] The above description is only the preferred embodiment of one or more embodiments of this application, and is not used to limit one or more embodiments of this application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of one or more embodiments of this application shall be included within the scope protected by one or more embodiments of this application.

[0238] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. And the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions, and corresponding operation entrances are provided for users to choose to authorize or refuse.

Claims

1. An unlocking method for an intelligent door lock, applied to an intelligent door lock; the method includes: When detecting that a user enters the detection range of the intelligent door lock, sending a first message to a door lock management platform, so that in response to receiving the first message, the door lock management platform determines a public camera device corresponding to the intelligent door lock based on the corresponding relationship between the intelligent door lock and the public camera device maintained, and sends the device information of the public camera device to the intelligent door lock; In response to receiving the device information, based on the device information, sending a second message to the public camera device, so that in response to receiving the second message, the public camera device obtains a historical video of a preset duration captured by it and sends the historical video to the intelligent door lock; In response to receiving the historical video, performing face detection on the video frames in the historical video to obtain target face information; wherein, the video frame containing the target face information is the video frame with the latest shooting time among the video frames containing face information; Matching the target face information with the face information of the unlockable users maintained, and performing an unlocking operation when the target face information matches the face information of the unlockable users.

2. The method according to claim 1, wherein the intelligent door lock is equipped with a sensor for detecting the movement of an object; The method further includes: When detecting that a user enters the sensing range of the sensor through the sensor, determining that the user enters the detection range of the intelligent door lock.

3. The method according to claim 1, when the duration of the historical video stored by the public camera device is not less than the preset duration, the historical video is the historical video stored by the public camera device; When the duration of the historical video stored by the public camera device is less than the preset duration, the historical video is the historical video obtained by merging the historical video stored by the public camera device and the historical video uploaded by the public camera device to the door lock management platform for storage.

4. The method according to claim 1, the performing face detection on the video frames in the historical video to obtain target face information includes: Starting from the video frame with the latest shooting time in the historical video, performing face detection on the video frames in the historical video in the order of shooting time from late to early until face information is detected, and determining the detected face information as target face information; or, Starting from the video frame with the latest shooting time in the historical video, performing face detection on the video frames in the historical video in the order of shooting time from late to early until the shooting time interval between the video frames for which face detection has been completed and the video frame with the latest shooting time is not less than a threshold value, and determining that no face information is detected.

5. The method according to claim 1, the default shooting range of the public camera device is the elevator entrance and exit on the floor where the intelligent door lock is located; The performing face detection on the video frames in the historical video to obtain target face information includes: Perform face detection on the video frames in the historical video to obtain target face information, and obtain portrait auxiliary information corresponding to the target face information from the video frames; Before performing the unlocking operation, the method further includes: Send a third message to the public camera device, so that in response to the received third message, the public camera device adjusts the shooting range to the room corresponding to the smart door lock, performs a shooting operation after the shooting range adjustment is completed, and sends the captured image to the smart door lock; In response to the received image, obtain portrait auxiliary information from the image; Match the portrait auxiliary information with the portrait auxiliary information corresponding to the target face information, and perform the unlocking operation when the portrait auxiliary information matches the portrait auxiliary information corresponding to the target face information.

6. The method according to claim 5, where the sending the third message to the public camera device, so that in response to the received third message, the public camera device adjusts the shooting range to the room corresponding to the smart door lock, performs a shooting operation, and sends the captured image to the smart door lock, includes: Send a third message to the public camera device, so that in response to the received third message, the public camera device adjusts the shooting range to the room corresponding to the smart door lock, performs a shooting operation after the shooting range adjustment is completed, and sends a fourth message to the smart door lock; In response to the received fourth message, obtain the video frames captured by the public camera device from the public camera device in real time.

7. The method according to claim 1, where the device information includes a communication address; The sending the second message to the public camera device based on the device information includes: Send the second message to the public camera device according to the communication address.

8. The method according to claim 1, where the smart door lock and the public camera device are joined to the same multicast group; a routing device corresponding to the multicast group maintains the correspondence between device information and interfaces; The sending the second message to the public camera device based on the device information includes: Send the second message to the routing device corresponding to the multicast group according to the communication address of the multicast group. The routing device determines the interface corresponding to the public camera device according to the device information included in the second message, and sends the second message to the public camera device through the interface.

9. An unlocking system for a smart door lock, the system includes a smart door lock, a public camera device, and a door lock management platform; where: When the smart door lock detects that a user enters the detection range of the smart door lock, send a first message to the door lock management platform; In response to the received first message, the door lock management platform determines the public camera device corresponding to the smart door lock based on the maintained correspondence between the smart door lock and the public camera device, and sends the device information of the public camera device to the smart door lock; The intelligent door lock responds to the received device information and sends a second message to the public camera device based on the device information; The public camera device responds to the received second message, obtains the historical video of a preset duration captured by it, and sends the historical video to the intelligent door lock; The intelligent door lock responds to the received historical video, performs face detection on the video frames in the historical video to obtain target face information; wherein, the video frame containing the target face information is the video frame with the latest shooting time among the video frames containing face information; The intelligent door lock matches the target face information with the face information of the unlockable users maintained by it, and performs an unlocking operation when the target face information matches the face information of the unlockable users.

10. The system according to claim 9, wherein the intelligent door lock and the public camera device are joined to the same multicast group; the system further includes a routing device corresponding to the multicast group; the routing device corresponding to the multicast group maintains the correspondence between device information and interfaces; The intelligent door lock sends a second message to the public camera device based on the device information, including: The intelligent door lock sends the second message to the routing device corresponding to the multicast group according to the communication address of the multicast group; The routing device determines the interface corresponding to the public camera device according to the device information included in the second message, and sends the second message to the public camera device through the interface.

Images