Pan-tilt control method, device and equipment for intelligent door lock and medium

Abstract

The invention relates to the technical field of intelligent door locks, and discloses a pan-tilt control method, device and equipment for an intelligent door lock and a medium, the intelligent door lock comprises a door lock body, and a radar module, a voice module, a camera module and a pan-tilt module which are arranged on the door lock body; wherein a camera shooting module is carried on the holder module, when the holder module rotates in the horizontal direction and the vertical direction, the camera shooting module rotates correspondingly, and the method comprises the steps that recognition results of a radar module, a voice module and the camera shooting module on a target object are obtained respectively; determining a target control module based on the recognition result of each module on the target object; and based on the target control module and the recognition result of the target control module on the target object, the holder module is controlled to correspondingly rotate in the horizontal and vertical directions, so that a camera module on the holder module tracks the target object in real time. According to the invention, accurate identification and real-time tracking of an object outside a door can be realized, the door lock control efficiency is improved, and the user experience is improved at the same time.

Description

Technical Field

[0001] The present invention relates to the technical field of smart door locks, and in particular to a pan / tilt control method, device, equipment and medium for a smart door lock. Background Art

[0002] At present, more and more smart door locks have face recognition or cat's eye monitoring functions, but their camera modules (including face or cat's eye, etc.) are fixed and cannot be rotated or adjusted. In actual applications, due to the difference in height of different users, the corresponding user will appear above or below the lens; or the user sometimes stands on the side of the door to open the door, that is, the user will appear on the left or right side of the lens. The above situations are likely to cause the collected face information to be biased or unclear, and thus the face cannot be well recognized. Therefore, the user needs to walk to a specific position in front of the door to perform accurate face recognition. In addition, if the user moves fast or is in motion, it is difficult to recognize the corresponding face. Face recognition can only be performed after the user stops stably. Therefore, the current smart door lock face recognition conditions are harsh, and there are defects such as slow recognition speed and low accuracy.

[0003] In addition, some smart door locks can view and monitor the scene outside the door through mobile phone APP or devices such as the rear screen of the door lock, but can only view the position of a certain angle area in front of the camera, and cannot track the target object in real time so that the target object is always in the center of the monitoring screen; and it is also impossible to adjust any angle through the terminal interface to view or zoom in on a certain area, and thus it is difficult to meet the user's needs for precise target tracking with adaptive angle adjustment. For example, if a package or express is placed directly below the door lock body or in other blind spots, the corresponding object cannot be monitored.

[0004] To solve the above problems, relevant smart door locks have introduced fisheye lenses to increase the imaging wide angle, but the imaging of scenes at the edge will also produce certain distortions. Once the target object (such as a face) stands at the edge, it will also affect the accurate identification of the target object, that is, the target object must be close to the front of the camera to be accurately identified, and there are certain limitations on the identification area and height. Or by adding another lower camera to the smart door lock to monitor the target object under the door, but this method greatly increases the control cost, power consumption and complexity of the smart door lock. What's more, for the application scenario of multiple people at the door, the existing smart door locks are prone to distortion due to angles and other reasons for people standing at the edge or at a lower position, making it difficult to accurately identify each person one by one, and the identification priority is not clear, and subsequent related analysis cannot be performed.

[0005] In summary, existing smart door locks have monitoring blind spots and are unable to accurately identify objects outside the door, which in turn leads to low control accuracy and poor efficiency of smart door locks, seriously affecting the user experience and making it difficult to meet the needs of smart door locks for accurate identification and real-time tracking of target objects. Summary of the invention

[0006] In view of this, the present invention provides a pan / tilt control method, device, equipment and medium for a smart door lock to solve the problem that the existing smart door lock is unable to accurately identify and track objects outside the door in real time within the monitoring blind area, which leads to low control accuracy and poor efficiency of the door lock and is difficult to meet the needs of smart use of the door lock.

[0007] In a first aspect, the present invention provides a pan / tilt control method for a smart door lock, wherein the smart door lock comprises a door lock body, and a radar module, a voice module, a camera module and a pan / tilt module arranged on the door lock body; wherein the pan / tilt module is equipped with a camera module, and when the pan / tilt module rotates in the horizontal and vertical directions, the camera module is rotated accordingly, and the method comprises: Obtain recognition results of the target object by the radar module, voice module and camera module respectively; Determine the target control module based on the recognition results of the target object by each module; Based on the target control module and its recognition result of the target object, the pan-tilt module is controlled to rotate accordingly in the horizontal and vertical directions, so that the camera module on the pan-tilt module can track the target object in real time.

[0008] The present invention determines the target control module of the pan-tilt module through the recognition results of the target object by the radar module, the voice module and the camera module, and controls the rotation of the pan-tilt module according to the recognition result of the target control module, so that the camera module on the pan-tilt module tracks the target object in real time, which can realize accurate recognition and real-time tracking of objects outside the door, improves the control accuracy and efficiency of the door lock, enhances the user experience, and meets the intelligent use requirements of the door lock.

[0009] In an optional implementation, determining the target control module based on the recognition results of the target object by each module includes: Determine whether the radar module recognizes the target object; When the radar module recognizes the target object, it is determined whether the camera module recognizes the target object; if the camera module recognizes the target object, the camera module is determined to be the target control module; if the camera module does not recognize the target object, the radar module is determined to be the target control module; When the radar module fails to recognize the target object, it is determined whether the voice module recognizes the target object. If the voice module recognizes the target object, the voice module is determined to be the target control module. If the voice module fails to recognize the target object, the process returns to the steps of respectively obtaining the recognition results of the target object by the radar module, the voice module and the camera module.

[0010] The present invention determines the control right of the pan-tilt module through the recognition results of the target object by the radar module, camera module and voice module and the set control priority, which can ensure that the pan-tilt module can quickly and accurately recognize the objects outside the door, and helps to improve the recognition accuracy and efficiency of the objects.

[0011] In an optional embodiment, the camera module includes a face recognition module and a cat's eye module; if the camera module is a target control module, then based on the target control module and its recognition result of the target object, the pan-tilt module is controlled to rotate accordingly in the horizontal and vertical directions, so that the camera module on the pan-tilt module tracks the target object in real time, including: Determine the current face of the face recognition module and the current picture of the cat's eye module according to the recognition result of the target object by the camera module; Determine whether the current face is in the center of the current picture; When the current face is not at the center of the current picture, the pan / tilt module is controlled to rotate in the horizontal and vertical directions so that the current face is at the center of the current picture; When the current face is at the center of the current picture, the current face is recognized based on the preset face information and the opening and closing of the door lock body is controlled according to the recognition result, or the target object is monitored in real time through the camera module.

[0012] When the camera module controls the pan-tilt module to work, the present invention obtains face information and monitoring images in real time through the face recognition module and the cat's eye module, and when the current face is not in the center of the current picture, controls the pan-tilt module to rotate in the horizontal and vertical directions to make the current face in the center of the current picture, thereby ensuring that the door lock can perform face recognition more clearly, improving the face recognition accuracy to a certain extent, and ensuring the accuracy of the door lock opening and closing control.

[0013] In an optional implementation, if the radar module is a target control module, then based on the target control module and its recognition result of the target object, the pan / tilt module is controlled to rotate accordingly in the horizontal and vertical directions, so that the camera module on the pan / tilt module tracks the target object in real time, including: Determine the current detection position according to the recognition result of the target object by the radar module; Control the PTZ module to rotate to the current detection position; Get the current recognition result of the camera module on the PTZ module at the current detection position; Determine whether the current recognition result contains the target object; When the current recognition result includes the target object, the target control module is updated to the camera module, and the pan / tilt module is controlled to rotate accordingly in the horizontal and vertical directions based on the camera module and its current recognition result of the target object, so that the camera module on the pan / tilt module can track the target object in real time; When the current recognition result does not include the target object, the process returns to the step of determining the current detection position according to the recognition result of the target object by the radar module.

[0014] When the radar module controls the pan-tilt module to work, the present invention starts the pan-tilt module to rotate to the current detection position recognized by the radar module, so that the camera module on the pan-tilt module can realize real-time tracking of the target object, which not only helps the target object to appear in the face recognition area of ​​the camera module faster and more accurately, but also speeds up the realization of face recognition and real-time monitoring of the target object.

[0015] In an optional implementation, if the voice module is a target control module, then based on the target control module and its recognition result of the target object, the pan / tilt module is controlled to rotate accordingly in the horizontal and vertical directions, so that the camera module on the pan / tilt module tracks the target object in real time, including: Determine the current sound position according to the recognition result of the target object by the voice module; Control the PTZ module to rotate to the current sound position; Get the current detection result of the radar module at the current sound emission position; Determine whether the current detection result contains the target object; When the current detection result includes the target object, the target control module is updated to the radar module, and the pan / tilt module is controlled to rotate accordingly in the horizontal and vertical directions based on the radar module and its current detection result of the target object, so that the camera module on the pan / tilt module can track the target object in real time; When the current detection result does not include the target object, the process returns to the step of determining the current sounding position according to the recognition result of the target object by the speech module.

[0016] When the voice module controls the pan-tilt module to work, the present invention starts the pan-tilt module to rotate to the current sound position recognized by the voice module. At the current sound position, if the current detection result of the radar module includes the target object, the control right of the pan-tilt is updated to the radar module, and based on the current detection result of the radar module, the pan-tilt module is controlled to rotate in the horizontal and vertical directions so that the camera module on the pan-tilt module can track the target object in real time, which helps to realize fast and accurate recognition and tracking of the target object outside the door.

[0017] In an optional implementation, before determining whether the current face is at the center of the current picture, the pan / tilt control method of the smart door lock further includes: When the number of faces in the current face is equal to 1, executing the step of determining whether the current face is located at the center of the current picture; When the number of faces in the current face count is greater than 1, the pan / tilt module is controlled to rotate in sequence according to the preset direction, and the recognition result of at least one person by the camera module at each rotation position is obtained accordingly; the face recognition result of the corresponding person is obtained based on the recognition result of each person corresponding to each rotation position, and the door lock body is controlled to be opened and closed based on the face recognition result; wherein, when the face recognition result is a registered person of the smart door lock, the door lock body is controlled to open, and the registered person is a person whose identity has been entered in the smart door lock; when the face recognition result is a non-registered person of the smart door lock, the door lock body is controlled to be closed.

[0018] The present invention not only designs real-time monitoring of a single person outside the door, but also designs an unlocking and identification process for multiple people, that is, by controlling the pan-tilt module to rotate in a preset direction to achieve a full scan of the scene outside the door, and then obtaining the identity information of all personnel and performing identity authentication in turn. When a registered person of the smart door lock is identified, the door lock body is controlled to open; when a non-registered person of the smart door lock is identified, the door lock body is controlled to close. This can significantly enhance the control efficiency of the pan-tilt and improve the user experience.

[0019] In an optional implementation, when the face recognition result is a non-registered person of the smart door lock, the pan / tilt control method of the smart door lock further includes: Count the number of identifications of non-registered persons; Determine whether the non-registered person belongs to the preset marked person and whether the number of identifications is greater than the preset number threshold; When the non-registered person does not belong to the preset marked person and the number of identification times is greater than the preset number threshold, the non-registered person is determined to be a risk person of the smart door lock, and the smart door lock is controlled to issue a risk warning.

[0020] When the current person identified as a non-registered person of the smart door lock is identified by the present invention, risk determination of the non-registered person is designed to identify the risk person, and the smart door lock is controlled to issue a risk warning, which can greatly ensure the safe operation of the door lock.

[0021] In an optional implementation, before respectively obtaining the recognition results of the target object by the radar module, the voice module and the camera module, the pan / tilt control method of the smart door lock further includes: Detect whether a pan-tilt control signal corresponding to an operation of a preset pan-tilt control interface by a target user is received; wherein the target user is a registered person of the smart door lock; the preset pan-tilt control interface includes an operation interface set on a mobile terminal APP that communicates with the smart door lock, and an operation interface set on a preset screen installed in the indoor part of the door lock body of the smart door lock, and the operation interface is used to control, configure and view the pan-tilt module of the smart door lock; When no pan / tilt control signal generated by the target user operating the preset pan / tilt interface is received, the steps of respectively obtaining recognition results of the target object by the radar module, the voice module and the camera module are performed; When a pan / tilt control signal corresponding to an operation of a preset pan / tilt interface by a target user is received, the pan / tilt module of the smart door lock is controlled accordingly based on the pan / tilt control signal.

[0022] The present invention designs automatic control processes and manual control processes for the pan-tilt module, and when receiving the pan-tilt control signal corresponding to the target user's operation on the preset pan-tilt interface, the manual control process is preferred, that is, the pan-tilt module of the smart door lock is controlled accordingly based on the pan-tilt control signal, which can meet the user's active monitoring needs of manually controlling the pan-tilt rotation to view the situation outside the door at any angle.

[0023] In an optional implementation, function activation buttons corresponding to the package monitoring function and the privacy avoidance function are also provided in the mobile terminal APP. The target user generates corresponding function control signals by operating different function activation buttons, and executes the corresponding pan / tilt control function of the smart door lock based on the function control signals.

[0024] The present invention satisfies the user's monitoring needs for different objects outside the door by enabling multiple monitoring functions set in the mobile terminal APP and configuring function buttons, which helps to improve the user's door lock usage experience.

[0025] In an optional implementation, when the target user presses the function activation button corresponding to the package monitoring function through the mobile terminal APP, the target user generates a corresponding function control signal by operating different function activation buttons, and executes the corresponding pan / tilt control function of the smart door lock based on the function control signal, including: The pan / tilt module of the smart door lock is controlled to rotate in a preset direction in sequence until the monitoring screen contains a package or other persons carrying a package and stops rotating; Identify the current identity information of the package or other persons carrying the package according to the camera module on the pan / tilt module, and determine whether the current identity information belongs to a preset marked person; When the current identity information belongs to a preset marked person, the radar module is used to identify whether there is someone in the room, and the smart door lock is controlled to perform corresponding processing based on the identification result.

[0026] The present invention takes into account the actual scenario requirements of couriers delivering packages. By rotating the pan / tilt module, the camera module can realize real-time monitoring and identity recognition of packages outside the door or people carrying packages. When the current identity information identified belongs to a preset marked person, the smart door lock is processed accordingly in combination with the indoor personnel recognition situation of the radar module, which helps to improve real-time monitoring and accurate identification of packages.

[0027] In an optional implementation, the smart door lock is controlled to perform corresponding processing according to the personnel situation recognition result, including: When someone is identified as being inside the room, the smart door lock is controlled to perform a first preset voice broadcast process to remind the person inside to open the door and take the package; When it is recognized that there is no one indoors, the smart door lock is controlled to perform a second preset voice broadcast to remind other people outside to place the package at the door. The camera module of the smart door lock identifies the current location information of the package and the current facial information of other people, and sends the current location information and current facial information to the administrator of the smart door lock. At the same time, the pan / tilt module of the smart door lock is controlled to rotate downward to monitor the package in real time.

[0028] The present invention designs two package handling processes corresponding to when someone is indoors and when no one is indoors, which not only realizes the safe and effective handling of packages, but also simplifies the operation process of couriers and ensures the efficiency of door lock registration personnel in picking up packages.

[0029] In an optional implementation, when the target user presses the function activation button corresponding to the privacy avoidance function through the mobile terminal APP, the target user generates a corresponding function control signal by operating different function activation buttons, and executes the corresponding pan / tilt control function of the smart door lock based on the function control signal, including: The pan / tilt module of the smart door lock is controlled to rotate to the neighbor identification area. The neighbor identification area refers to the monitoring screen containing the neighbor smart door lock; Identify the current door lock status of the neighbor's smart door lock based on the camera module on the pan / tilt module; When it is identified that the neighbor smart door lock is currently in the door lock body open state, the pan-tilt module is controlled to rotate to a direction away from the neighbor recognition area, and the pan-tilt module is controlled to rotate back to the neighbor recognition area after a preset time; if it is identified that the neighbor smart door lock is still in the door lock body open state, the pan-tilt module is again controlled to rotate to a direction away from the neighbor recognition area until it is identified that the neighbor smart door lock is currently in the door lock body closed state, and the rotation of the pan-tilt module is stopped.

[0030] The present invention takes into account the actual need that the current door lock is easy to expose its own privacy after the neighbor opens the door. When it is recognized that the neighbor has opened the door or the door lock is open, the pan-tilt head is rotated to other positions to avoid it, which can prevent the camera module of the current door lock from monitoring the situation of the other neighbor's home, thereby protecting the other party's privacy and security to a great extent.

[0031] In an optional implementation, in the process of controlling the pan / tilt module to rotate toward a position away from the neighbor recognition area, the pan / tilt control method of the smart door lock further includes: If the radar module recognizes the location information of other people, the privacy avoidance function is exited, and the gimbal module is controlled to rotate accordingly in the horizontal and vertical directions based on the location information, so that the gimbal module can track other people in real time.

[0032] When executing the privacy avoidance function, the present invention exits the currently executed monitoring function when other persons are identified, and controls the pan / tilt module to rotate so that the camera module on the pan / tilt module can track other newly appeared persons in real time, thus having the advantages of flexible and accurate monitoring.

[0033] In a second aspect, the present invention provides a pan-tilt control device for a smart door lock, wherein the smart door lock comprises a door lock body, and a radar module, a voice module, a camera module and a pan-tilt module arranged on the door lock body; wherein the pan-tilt module is equipped with a camera module, and when the pan-tilt module rotates in the horizontal and vertical directions, the camera module rotates accordingly, and the device comprises: An acquisition unit, used to respectively acquire recognition results of the target object by the radar module, the voice module and the camera module; An identification unit, used to determine a target control module based on the identification results of each module on the target object; The control unit is used to control the pan / tilt module to rotate accordingly in the horizontal and vertical directions based on the target control module and its recognition result of the target object, so that the camera module on the pan / tilt module can track the target object in real time.

[0034] The present invention determines the target control module of the pan-tilt module based on the recognition results of the target object by the radar module, the voice module and the camera module, and controls the rotation of the pan-tilt module through the recognition result of the target control module so that the camera module on the pan-tilt module can track the target object in real time, which can reduce the monitoring blind area outside the door and realize the effective monitoring of the objects outside the door to a great extent. It can not only improve the control accuracy and efficiency of the door lock, but also enhance the user experience and meet the intelligent use requirements of the door lock.

[0035] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the pan / tilt control method of a smart door lock according to the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0036] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute a pan / tilt control method for an intelligent door lock according to the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0038] Figure 1 is a flow chart of a pan / tilt control method of an intelligent door lock according to an embodiment of the present invention; Figure 2 is a flow chart of another pan / tilt control method of an intelligent door lock according to an embodiment of the present invention; Figure 3 This is a flowchart of the PTZ control rights confirmation process; Figure 4 This is a schematic diagram of the structure of the door lock installation pan / tilt; Figure 5 It is a schematic diagram of target recognition in a multi-person scenario; Figure 6 This is a schematic diagram of personnel classification processing in a multi-person scenario; Figure 7 This is a schematic diagram of the PTZ terminal control interface; Figure 8 It is a schematic diagram of package monitoring; Fig. 9 It is a flowchart of the execution of the package monitoring function; Fig.10 It is a privacy monitoring schematic diagram; Fig.11 It is a flowchart of the execution of the privacy avoidance function; Fig.12 It is a structural diagram of the intelligent door lock pan-tilt control system; Fig.13 is a structural block diagram of a pan / tilt control device of an intelligent door lock according to an embodiment of the present invention; Fig.14It is a schematic diagram of the structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0040] An embodiment of the present invention provides an embodiment of a pan / tilt control method for a smart door lock. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer device such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0041] In this embodiment, the smart door lock includes a door lock body and a radar module, a voice module, a camera module and a PTZ module arranged on the door lock body; the PTZ module is equipped with a camera module, and when the PTZ module rotates in the horizontal and vertical directions, the camera module rotates accordingly; the radar module, the voice module and the camera module are used to identify objects outside the door. It should be noted that the specific composition of each module is adaptively set based on actual needs.

[0042] In this embodiment, a pan / tilt control method for an intelligent door lock is provided. Figure 1 is a flow chart of a pan / tilt control method of an intelligent door lock according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps: Step S101, respectively obtaining the recognition results of the radar module, the voice module and the camera module on the target object.

[0043] It should be noted that the specific content of the target object in this embodiment is not limited here, and it is adaptively adjusted based on actual needs. For example, the target object is people such as young people, the elderly and children, or pets, express deliveries and other items, which is only used as an exemplary description.

[0044] Step S102: determining a target control module based on the recognition results of each module on the target object.

[0045] In this embodiment, the target control module is used to control the rotation of the pan / tilt module, which is determined based on the recognition results of each module and the set control priority.

[0046] Step S103, based on the target control module and its recognition result of the target object, the pan / tilt module is controlled to rotate accordingly in the horizontal and vertical directions, so that the camera module on the pan / tilt module can track the target object in real time.

[0047] The pan / tilt control method of the smart door lock of the embodiment of the present invention can realize accurate recognition and real-time tracking of objects outside the door, improve the control accuracy and efficiency of the door lock, further enhance the user experience, and meet the intelligent use requirements of the door lock.

[0048] In this embodiment, a pan / tilt control method for an intelligent door lock is provided. Figure 2 FIG. 1 is a flow chart of another PTZ control method of an intelligent door lock according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps: Step S201, respectively obtain the recognition results of the radar module, the voice module and the camera module on the target object. Figure 1 Step S101 of the illustrated embodiment will not be described in detail here.

[0049] Step S202: determining a target control module based on the recognition results of each module on the target object.

[0050] In this embodiment, the control priority of each module over the PTZ module can be determined according to the actual demand adaptability, that is, once each module recognizes the target object, the corresponding high-priority module has priority control over the PTZ module. Specifically, the above step S202 includes: Step S2021, determine whether the radar module recognizes the target object.

[0051] In this embodiment, the smart door lock is equipped with a millimeter wave radar (i.e., radar module), which uses electromagnetic waves in the millimeter wave frequency band (usually 30-300GHz) to detect targets. Specifically, by emitting millimeter wave signals, when the signals encounter the target object, they will be reflected back. The radar receives the reflected signals and analyzes their time delay, frequency change and other information to calculate the distance, speed and angle of the target object.

[0052] Step S2022, when the radar module recognizes the target object, determine whether the camera module recognizes the target object; if the camera module recognizes the target object, determine the camera module as the target control module; if the camera module does not recognize the target object, determine the radar module as the target control module.

[0053] In this embodiment, the camera module includes a face recognition module and a cat's eye module, which are used to capture face information and monitor target objects, and the above two modules are used to identify target objects.

[0054] Step S2023, when the radar module fails to recognize the target object, determine whether the voice module recognizes the target object. If the voice module recognizes the target object, determine that the voice module is the target control module; if the voice module fails to recognize the target object, return to the step of respectively obtaining the recognition results of the radar module, voice module and camera module for the target object.

[0055] In this embodiment, the process of each module controlling the PTZ module to achieve target monitoring includes: 1. When the radar recognizes the target object, the camera module will be started. If the face recognition module can quickly and accurately recognize the entered face information without the pan-tilt rotation, the door will be opened. If the face recognition cannot accurately recognize the face information, the pan-tilt module will be started to rotate the pan-tilt so that the current person is in the center of the picture, even if the pan-tilt is aimed at the face, so as to accurately recognize the person. In the cat's eye monitoring picture, the pan-tilt rotates with the movement of the target object, so that the target object is always in the center of the picture.

[0056] 2. When the voice module recognizes a voice signal, it processes the voice signal, that is, a sound wave that changes with time, is collected by the microphone and converted into an electrical signal. The electrical signal is framed, that is, the continuous voice signal is divided into multiple short time periods (frames), and then the acoustic feature is extracted for each frame. This feature can effectively characterize the spectral characteristics of the voice (that is, the "sound fingerprint" of the voice signal); when a specific sound feature is obtained, the target object is identified. In addition, a specific voice command can be set, such as "Xiaoman, hello, I'm back", and the voice recognition will collect and recognize this voice command, and then start the pan / tilt based on the command, and turn on the camera module for target recognition.

[0057] In a specific embodiment, Figure 3 This is a flowchart of the PTZ control right confirmation process. Figure 3 It can be seen that the process includes: ① Once the radar module identifies the target object in the monitoring area, the camera module is started, and the camera module detects whether the target object is identified; if the camera module identifies the target object, the pan-tilt module is immediately started, and the camera module controls the pan-tilt so that the camera module on the pan-tilt rotates left and right and up and down with the target object, so that the target object is always in the center of the picture, which can perform face recognition or target object monitoring faster and clearer.

[0058] ② When the radar identifies the target object, but the camera module does not, the gimbal module is started, and then the gimbal rotates synchronously according to the direction, speed, distance and other information of the object identified by the radar until the camera module captures the target object. The control of the gimbal is then transferred to the camera module, that is, the lens is rotated in advance to aim at the target object as soon as the object enters the radar area. When the object enters the camera area, it can be recognized by the face recognition module at the first time, and combined with the cat's eye monitoring, it can capture and track the movement of objects outside the door faster and clearer, realize accurate recognition and then control the door lock to open.

[0059] ③When the radar does not recognize the target object, but the voice module recognizes the target object, the gimbal module is started, and then the gimbal rotates synchronously according to the information recognized by the voice module. When the radar module captures the target object, the control of the gimbal is transferred to the radar module.

[0060] It should be noted that the control priority of the gimbal of the above three modules that recognize the target objects is: camera module>radar module>voice module, that is, once the camera module detects the target object, the camera module controls the gimbal to monitor the target object; in the voice module recognition and tracking process, once the target object is recognized by the radar, the radar monitors the target object and controls the gimbal; in the radar recognition and tracking process, once the target object is recognized by the camera module, the camera module monitors the target object and controls the gimbal.

[0061] It should be noted that the voice module of this embodiment can record the corresponding voiceprint or specific voice command in advance, and the pan / tilt module will be activated when the corresponding information is recognized. In addition, if the voice module includes two or more microphones, the sound signals are obtained respectively, and the sound source is located according to the time difference. For example, if a sound is emitted from the far right side of the door (different thresholds can be set), the camera module can control the pan / tilt to turn to the right in advance according to the sound source, so as to monitor the situation on the right side in advance, and capture the object's entry picture at the first time, avoiding the limited distance of the usual camera or radar to capture the situation outside the door, expanding the monitoring distance and capturing more pictures at an earlier time.

[0062] In this embodiment, the pan-tilt module includes a camera module (i.e., a face recognition module and a cat's eye module), a display screen, a pan-tilt drive module, and dual motors; wherein the pan-tilt drive module is integrated on the pan-tilt circuit board; the display screen is on the outer surface of the pan-tilt, and can display weather forecasts, emoticons, or self-set reminder information, etc., and can also display the monitoring screen of the camera module. Specifically, the pan-tilt can drive the camera module to rotate left and right, up and down, and the pan-tilt speed (i.e., the corresponding frequency) is adjustable. In a specific embodiment, a smart door lock is provided after installing a pan-tilt module, Figure 4 This is a schematic diagram of the structure of the door lock installation pan / tilt. Figure 4As shown, a pan / tilt is installed above the outdoor panel of the smart door lock, and a display screen is also installed on the indoor panel inside the door.

[0063] In the embodiment of the present invention, the control right of the pan-tilt module is determined by the recognition results of the target object by the radar module, camera module and voice module and the set control priority, which can ensure that the pan-tilt module can quickly and accurately identify objects outside the door, and help improve the recognition accuracy and efficiency of objects.

[0064] Step S203, based on the target control module and its recognition result of the target object, the pan / tilt module is controlled to rotate accordingly in the horizontal and vertical directions, so that the camera module on the pan / tilt module can track the target object in real time.

[0065] It should be noted that the target control modules in this embodiment include three types. When the camera module is the target control module, the above step S203 includes: Step A1, determining the current face of the face recognition module and the current picture of the cat's eye module respectively according to the recognition result of the target object by the camera module.

[0066] It should be noted that in this embodiment, there may be multiple target objects, that is, there are multiple facial information, which need to be identified separately to achieve real-time monitoring of each person. Therefore, after executing step A1, the pan / tilt control method of the smart door lock in this embodiment also includes: Step A11, when the number of faces in the current face is equal to 1, execute step A2 of determining whether the current face is at the center position of the current picture.

[0067] Step A12, when the number of faces in the current face count is greater than 1, the pan / tilt module is controlled to rotate in sequence according to the preset direction, and the recognition result of the camera module for at least one person is obtained correspondingly at each rotation position; the face recognition result of the corresponding person is obtained based on the recognition result of each person corresponding to each rotation position, and the door lock body is controlled to be opened and closed based on the face recognition result; wherein, when the face recognition result is a registered person of the smart door lock, the door lock body is controlled to open, and the registered person is a person whose identity has been entered in the smart door lock; when the face recognition result is a non-registered person of the smart door lock, the door lock body is controlled to be closed.

[0068] In this embodiment, the specific content of the preset direction is not limited here. For example, the area outside the door is divided into blocks, and people are identified by rotating in sequence from left to right and from top to bottom. This is only an exemplary description.

[0069] It should be noted that in this embodiment, the non-registered person may be a person who needs to frequently enter and exit the location of the door lock, such as a food delivery or courier delivery worker; or a stranger living in the current environment, or a risk person with potential harm. Therefore, when the non-registered person outside the door is identified in this embodiment, it is necessary to further identify him to ensure the safety of the registered person of the smart door lock. Therefore, when the face recognition result is a non-registered person of the smart door lock, the pan / tilt control method of the smart door lock in this embodiment also includes: Step A121, counting the number of times non-registered persons are identified.

[0070] Step A122, determining whether the unregistered person belongs to a preset marked person and whether the number of recognitions is greater than a preset number threshold.

[0071] In this embodiment, the preset marked persons are people who need to pass through the door lock (such as couriers), and the specific contents of the preset marked persons and the preset number of thresholds are not limited here, and can be adaptively determined according to actual needs.

[0072] Step A123, when the non-registered person does not belong to the preset marked person and the number of identification times is greater than the preset number threshold, the non-registered person is determined to be a risk person of the smart door lock, and the smart door lock is controlled to issue a risk warning.

[0073] In this embodiment, the specific method of risk warning is not limited here and can be adaptively determined according to actual needs, such as voice warning through door locks.

[0074] When the embodiment of the present invention identifies that the current person is an unregistered person of the smart door lock, the risk person is identified by designing a risk assessment for the unregistered person, and the smart door lock is controlled to issue a risk warning, which can greatly ensure the safe operation of the door lock.

[0075] In a specific embodiment, Figure 5 This is a schematic diagram of target recognition in a multi-person scenario. Figure 5 It can be seen that for multi-person recognition application scenarios, see Figure 5 In the schematic diagram of part (a), the pan / tilt of the smart door lock of this embodiment will be adjusted according to the number of people (i.e. Figure 5 The identification application scenario of 5 people is described in the figure. Each person is identified by numbers in sequence. The area outside the door is divided into 4 areas. Figure 5The schematic diagram of part (b) in the figure is the upper left area: A, the upper right area: B, the lower right area: C and the lower left area: D. Among them, the order of pan-tilt rotation recognition is: upper left area A->upper right area B->lower right area C->lower left area D. Specifically, the pan-tilt rotation in this embodiment can accurately identify each person in a way that identifies people, whether they are people at the edge of the door or children. In addition, the identified people can be classified, such as door lock administrators, family members, couriers, strangers and other categories. At the same time, for the same stranger, each recognition accumulates attention once, and when the number of recognitions of the stranger reaches a certain threshold, the administrator of the door lock will be actively reminded, indicating that the stranger often comes and goes, please confirm whether he is a risk person or a familiar person; if it is determined to be a risk person, a voice warning can be issued, such as "You have stayed many times, please leave this area." For the classification process of identified personnel, please refer to Figure 6 .

[0076] In the embodiment of the present invention, not only real-time monitoring of a single person outside the door is designed, but also a unlocking and identification process for multiple people is designed, that is, by controlling the pan-tilt module to rotate in a preset direction to achieve a full scan of the scene outside the door, and then obtaining the identity information of all personnel and performing identity authentication in turn, when the person is identified as a registered person of the smart door lock, the door lock body is controlled to open, and when the person is identified as a non-registered person of the smart door lock, the door lock body is controlled to close, which can significantly enhance the control efficiency of the pan-tilt and improve the user experience.

[0077] Step A2, determining whether the current face is in the center of the current picture.

[0078] In this embodiment, the method for determining the positional relationship between the current face and the current picture is not limited here, such as calculating the distance between the center of the face and the center of the picture, that is, first determine the center position of the face, and then calculate the distance between the position and the center of the picture; if the distance is less than a certain threshold (for example, half the width and height of the picture), it can be considered that the face is in the center of the picture, which is only for exemplary explanation.

[0079] Step A3, when the current face is not at the center of the current picture, control the pan / tilt module to rotate in the horizontal and vertical directions so that the current face is at the center of the current picture.

[0080] Step A4, when the current face is at the center of the current picture, the current face is recognized based on the preset face information and the door lock body is controlled to be opened and closed according to the recognition result, or the target object is monitored in real time through the camera module.

[0081] In this embodiment, the preset facial information is the facial information saved by the door lock user when entering the identity information.

[0082] In the embodiment of the present invention, when the camera module controls the pan-tilt module to work, face information and monitoring images are obtained in real time through the face recognition module and the cat's eye module, and when the current face is not in the center of the current picture, the pan-tilt module is controlled to rotate in the horizontal and vertical directions to make the current face in the center of the current picture, thereby ensuring that the door lock can perform face recognition more clearly, improving the face recognition accuracy to a certain extent, and at the same time ensuring the accuracy of the door lock opening and closing control.

[0083] In this embodiment, when the radar module is a target control module, the above step S203 includes: Step B1, determining the current detection position according to the recognition result of the target object by the radar module.

[0084] Step B2, controlling the pan / tilt module to rotate to the current detection position.

[0085] Step B3, obtaining the current recognition result of the camera module on the pan / tilt module at the current detection position.

[0086] Step B4, determining whether the current recognition result contains the target object.

[0087] Step B5, when the current recognition result includes the target object, the target control module is updated to the camera module, and based on the camera module and its current recognition result of the target object, the pan-tilt module is controlled to rotate accordingly in the horizontal and vertical directions, so that the camera module on the pan-tilt module can track the target object in real time.

[0088] Step B6, when the current recognition result does not include the target object, return to the step of determining the current detection position according to the recognition result of the target object by the radar module.

[0089] In the embodiment of the present invention, when the radar module controls the operation of the pan-tilt module, the pan-tilt module is started to rotate to the current detection position recognized by the radar module, so that the camera module on the pan-tilt module can realize real-time tracking of the target object, which not only helps the target object to appear in the face recognition area of ​​the camera module faster and more accurately, but also speeds up the realization of face recognition and real-time monitoring of the target object.

[0090] In this embodiment, when the radar module is a target control module, the above step S203 includes: Step C1, determining the current sound position according to the recognition result of the target object by the voice module.

[0091] Step C2, controlling the pan / tilt module to rotate to the current sound emitting position.

[0092] Step C3, obtaining the current detection result of the radar module at the current sound emission position.

[0093] Step C4, determining whether the current detection result contains a target object.

[0094] Step C5, when the current detection result includes the target object, the target control module is updated to the radar module, and based on the radar module and its current detection result of the target object, the gimbal module is controlled to rotate accordingly in the horizontal and vertical directions, so that the camera module on the gimbal module can track the target object in real time.

[0095] Step C6, when the current detection result does not include the target object, return to the step of determining the current sound position according to the recognition result of the target object by the voice module.

[0096] In the embodiment of the present invention, when the voice module controls the pan-tilt module to work, the pan-tilt module is started to rotate to the current sound position recognized by the voice module. When the current detection result of the radar module at the current sound position includes the target object, the control right of the pan-tilt is updated to the radar module, and based on the current detection result of the radar module, the pan-tilt module is controlled to rotate accordingly in the horizontal and vertical directions, so that the camera module on the pan-tilt module can track the target object in real time, which helps to achieve fast and accurate identification and tracking of target objects outside the door.

[0097] It should be noted that the pan-tilt control process of the smart door lock in this embodiment includes an automatic control process and a manual control process; the signal source of the automatic control process is the signal recognition module, that is, the pan-tilt operation is started only after any of the three modules, namely the radar module, the voice module and the camera module, recognizes the target object information; the signal source of the manual control process is the relevant key operation of the door lock administrator on the set function button on the pan-tilt terminal control interface, and the manual control request of this embodiment is higher than the signal request issued by the signal recognition module. Therefore, before obtaining the recognition results of the target object by the radar module, the voice module and the camera module respectively, the pan-tilt control method of the smart door lock in this embodiment also includes: Step D1, detects whether a pan-tilt control signal corresponding to the target user's operation on the preset pan-tilt control interface is received; wherein the target user is a registered person of the smart door lock; the preset pan-tilt control interface includes an operation interface set on a mobile terminal APP that communicates with the smart door lock, and an operation interface set on a preset screen installed on the indoor part of the door lock body of the smart door lock, and the operation interface is used to control, configure and view the pan-tilt module of the smart door lock.

[0098] It should be noted that, in this embodiment, the preset PTZ control interface can be set on the screen configured by the indoor operation panel, such as a touch screen, a mobile visual screen, etc. After the operator passes the administrator identity authentication, he can perform relevant operations on the function buttons set on the interface.

[0099] In this embodiment, the mobile terminal APP is also provided with function activation buttons corresponding to the package monitoring function and the privacy avoidance function. The target user generates corresponding function control signals by operating different function activation buttons, and executes the corresponding pan / tilt control function of the smart door lock based on the function control signals. Specifically, by enabling multiple monitoring functions set in the mobile terminal APP and configuring the function buttons, the user's monitoring needs for different objects outside the door are met, which helps to improve the user's door lock usage experience.

[0100] In a specific embodiment, Figure 7 This is a schematic diagram of the PTZ terminal control interface. Figure 7 It can be seen that the interface can display the monitoring screen in real time, and the interface is set with buttons for recording, taking photos, and shooting videos. Specifically, when the operator of the door lock enters the PTZ terminal control interface, he can click the PTZ control button on the real-time video screen, and the left, right, up, and down control buttons will pop up. The PTZ direction can be adjusted according to the user's own needs, and the control command is sent to the PTZ drive module, thereby driving the horizontal motor and the vertical motor to rotate to view the required monitoring screen.

[0101] Step D2, when no pan / tilt control signal corresponding to the target user operating the preset pan / tilt interface is received, the steps of respectively obtaining the recognition results of the target object by the radar module, the voice module and the camera module are executed.

[0102] Step D3, when receiving the pan / tilt control signal generated corresponding to the target user operating the preset pan / tilt interface, the pan / tilt module of the smart door lock is controlled accordingly based on the pan / tilt control signal.

[0103] In the embodiment of the present invention, an automatic control process and a manual control process are designed for the pan-tilt module. When the pan-tilt control signal generated corresponding to the target user's operation on the preset pan-tilt interface is received, the manual control process is preferred, that is, the pan-tilt module of the smart door lock is controlled accordingly based on the pan-tilt control signal, which can meet the user's active monitoring needs of manually controlling the rotation of the pan-tilt to view the situation outside the door at any angle.

[0104] In this embodiment, when the target user presses the function activation button corresponding to the package monitoring function through the mobile terminal APP, the target user generates a corresponding function control signal by operating different function activation buttons, and executes the corresponding pan / tilt control function of the smart door lock based on the function control signal, including: Step E1, controlling the pan / tilt module of the smart door lock to rotate in a preset direction in sequence until the monitoring screen contains a package or other persons carrying a package and stops rotating.

[0105] In this embodiment, the contents related to the preset direction are as mentioned above and will not be repeated here.

[0106] Step E2, identifying the current identity information of the package or other persons carrying the package according to the camera module on the pan / tilt module, and determining whether the current identity information belongs to a preset marked person.

[0107] Step E3, when the current identity information belongs to a preset marked person, the radar module is used to identify whether there is anyone in the room, and the smart door lock is controlled to perform corresponding processing according to the result of the personnel identification.

[0108] Specifically, in the above step E3, the smart door lock is controlled to perform corresponding processing according to the personnel situation recognition result, including: Step E31, when it is recognized that there is someone indoors, the smart door lock is controlled to perform a first preset voice broadcast process to remind the person indoors to open the door and take the package.

[0109] Step E32, when it is recognized that there is no one indoors, the smart door lock is controlled to perform a second preset voice broadcast process to remind other people outside to place the package at the door, and the current location information of the package and the current facial information of other people are identified through the camera module of the smart door lock, and the current location information and the current facial information are sent to the administrator of the smart door lock, and at the same time, the pan / tilt module of the smart door lock is controlled to rotate downward to monitor the package in real time.

[0110] It should be noted that, in this embodiment, the specific contents of the first preset voice and the second preset voice are adaptively set based on actual needs.

[0111] In the embodiment of the present invention, two package handling processes corresponding to when someone is indoors and when no one is indoors are designed, which not only realizes the safe and effective handling of packages, but also simplifies the operation process of couriers and ensures the efficiency of door lock registration personnel in picking up packages.

[0112] In a specific embodiment, Figure 8 This is a schematic diagram of package monitoring. Figure 8 It can be seen that the package may be placed at the bottom outside the door or in the hands of the courier. When the target user of the door lock presses the function start button corresponding to the package monitoring function through the mobile terminal APP, the pan / tilt rotates to make the camera rotate downward to see the objects at the bottom outside the door. There is no need to add a new bottom camera, which can conveniently monitor the package information, making the package safer. Fig. 9, when the package monitoring function is turned on in the mobile door lock APP, when a stranger is detected approaching, the pan-tilt system is started to scan around, and when the relevant information "marking express delivery" is recognized on the clothes or package, it is determined that it is a courier; at the same time, it is determined whether there is anyone indoors. If it is recognized that there is someone indoors: that is, when the door is in a locked state, the indoor radar detects that there is someone indoors, the indoor speaker will issue a voice broadcast "You have a new package", which is convenient for indoor personnel to quickly check the information and open the door to take the express, and the courier does not need to put down the package and press the doorbell; when it is recognized that there is no one indoors: that is, when the door is in a locked state, the indoor radar does not detect anyone, press the doorbell, no one opens the door, and a message is received at the door: "Please leave the package at the door, thank you", the courier can directly put down the package and leave; the pan-tilt system records the facial information of the courier, takes a photo of the package and sends it to the administrator, and at the same time rotates the pan-tilt system to the position aimed at the package to monitor the package information. Even when someone approaches and quickly takes the package, the movement trajectory and facial information of the person who takes it can be immediately captured.

[0113] In the embodiment of the present invention, the actual scenario requirements of couriers delivering packages are taken into consideration. The pan / tilt module is rotated to enable the camera module to monitor and identify packages outside the door or other persons carrying packages in real time. When the current identity information identified belongs to a preset marked person, the smart door lock is processed accordingly in combination with the indoor personnel recognition situation of the radar module, which helps to improve real-time monitoring and accurate identification of packages.

[0114] In this embodiment, when the target user presses the function activation button corresponding to the privacy avoidance function through the mobile terminal APP, the target user generates a corresponding function control signal by operating different function activation buttons, and executes the corresponding pan / tilt control function of the smart door lock based on the function control signal, including: Step F1, control the pan / tilt module of the smart door lock to rotate to the neighbor identification area, the neighbor identification area refers to the neighbor smart door lock included in the monitoring screen.

[0115] Step F2, identifying the current door lock status of the neighbor's smart door lock according to the camera module on the pan / tilt module.

[0116] Step F3, when it is identified that the neighbor smart door lock is currently in the door lock body open state, the pan-tilt module is controlled to rotate to a direction away from the neighbor recognition area, and the pan-tilt module is controlled to rotate back to the neighbor recognition area after a preset time; if it is identified that the neighbor smart door lock is still in the door lock body open state, the pan-tilt module is controlled to rotate to a direction away from the neighbor recognition area again until it is identified that the neighbor smart door lock is currently in the door lock body closed state, and the rotation of the pan-tilt module is stopped.

[0117] Specifically, in the process of controlling the pan-tilt module to rotate away from the neighbor recognition area in the above step F3, the pan-tilt control method of the smart door lock also includes: if the radar module recognizes the location information of other people, exit the privacy avoidance function, and control the pan-tilt module to rotate accordingly in the horizontal and vertical directions based on the location information, so that the pan-tilt module can track other people in real time. When executing the privacy avoidance function, if other people are recognized, the embodiment of the present invention exits the currently executed monitoring function, and controls the pan-tilt module to rotate so that the camera module on the pan-tilt module can track other newly appeared people in real time, which has the advantages of flexible and accurate monitoring.

[0118] In a specific embodiment, Fig.10 is a privacy monitoring diagram. Fig.10 It can be seen that when the neighbor's door is opened, the PTZ camera rotates to another direction to avoid privacy. Fig.11 When the privacy avoidance function is turned on in the APP, when the other party's door is opened in the cat's eye monitoring screen, the pan-tilt is immediately rotated so that the cat's eye is at the bottom and the other party's state cannot be seen, and privacy avoidance is performed; after setting a certain threshold, such as 5 minutes, the pan-tilt returns to its position. If it is detected that the other party's door is still open, the pan-tilt is rotated again to the avoidance position until the other party's door is closed. It should be noted that when the pan-tilt is in the avoidance position, if the radar detects that a person is approaching the door lock, the pan-tilt avoidance is lifted and the person is tracked and identified normally.

[0119] In the embodiment of the present invention, the actual need that the current door lock is easy to expose its own privacy after the neighbor opens the door is taken into consideration. When it is recognized that the neighbor has opened the door or the door lock is open, the pan / tilt is rotated to other positions to avoid it. This can prevent the camera module of the current door lock from monitoring the situation of the other neighbor's home, thereby protecting the other party's privacy and security to a great extent.

[0120] In this embodiment, a smart door lock pan / tilt control system is provided that integrates the pan / tilt control method of the smart door lock. Fig.12As shown, the system includes: a signal recognition module, a PTZ terminal control interface, a main control module and a PTZ control module; wherein the signal recognition module includes a radar module, a camera module and a voice module; when any of the radar module, the camera module and the voice module recognizes a target object, the PTZ is started, and the control priority of the PTZ is: the camera module> the radar module> the voice module, and these signal recognition modules send information such as the detected position of the target object to the main control module, and the main control module sends a control instruction to the PTZ control module, thereby driving the motor 1 (horizontal direction) and the motor 2 (vertical direction) to rotate. The PTZ terminal control interface includes a mobile door lock APP interface, a touch screen interface behind the door, a mobile visual screen interface, etc. (the interface can be adaptively adjusted according to actual needs), that is, on this interface, the PTZ is manually controlled to rotate according to the administrator's needs of the door lock. The PTZ control module includes a PTZ drive module, a motor 1 and a motor 2. Specifically, the main control module processes data according to the identification information sent by the signal recognition module or the control command sent by the PTZ terminal control interface and sends the processed signal to the PTZ drive module, which then controls motors 1 and 2 to rotate accordingly according to the control command, that is, controls the rotation angle, speed, minimum step, etc. of each motor. It should be noted that the button request on the PTZ terminal control interface has a higher priority than the signal request issued by the signal recognition module.

[0121] In summary, the smart door lock and smart door lock PTZ control system of this embodiment have the following advantages: 1. The face recognition module and cat's eye module carried by the gimbal can rotate left and right and up and down with the gimbal, and can quickly and efficiently identify and monitor different targets according to different scenarios; for example, when a short child is detected, the gimbal can rotate downward to perform more accurate face recognition; or when a person enters the door from the side, the door can be opened without taking a few more steps to the front.

[0122] 2. The smart door lock includes a pan / tilt terminal control interface, which can manually control the pan / tilt rotation to view the situation outside the door at any angle.

[0123] 3. The advantage of the smart door lock PTZ control system is that when the radar detects an object but the camera module does not recognize the object, the PTZ can be started and the camera module can be rotated so that the target object appears in the face recognition area faster and more accurately, and the target object can be monitored or identified earlier.

[0124] 4. The advantage of the smart door lock PTZ control system is that it also includes a voice module. When the voice module detects an object but the radar module does not recognize the object, the PTZ can be started and the camera module can be rotated so that the target object appears in the camera recognition area faster and more accurately, and the target object can be monitored or identified earlier.

[0125] 5. The advantage of the smart door lock PTZ control system is that when the target object is in motion, the PTZ can synchronously track the target object and perform face recognition more clearly, solving the problem of blurry and difficult to identify due to the object in motion.

[0126] 6. The smart door lock PTZ control system has a package monitoring function. It detects objects directly under the door by adjusting the PTZ rotation angle, and can check and monitor package or express information at any time.

[0127] 7. The smart door lock PTZ control system has a privacy avoidance function, which can avoid shooting the situation inside the opposite door by rotating the PTZ, thereby protecting the privacy and security of the opposite door.

[0128] 8. The smart door lock PTZ control system can accurately identify multiple people outside the door one by one, classify the identified people and determine their priority.

[0129] In this embodiment, a pan / tilt control device for an intelligent door lock is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and will not be repeated hereafter. As used below, the term "unit" refers to a combination of software and / or hardware that can implement a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0130] The present invention provides a pan / tilt control device for an intelligent door lock, such as Fig.13 As shown, the device comprises: The acquisition unit 1301 is used to respectively acquire the recognition results of the radar module, the voice module and the camera module on the target object.

[0131] The recognition unit 1302 is used to determine the target control module based on the recognition results of each module on the target object.

[0132] The control unit 1303 is used to control the pan / tilt module to rotate accordingly in the horizontal and vertical directions based on the target control module and its recognition result of the target object, so that the camera module on the pan / tilt module can track the target object in real time.

[0133] In some optional embodiments, the identification unit 1302 includes: a first identification subunit, a second identification subunit and a third identification subunit; wherein the first identification subunit is used to determine whether the radar module recognizes the target object; the second identification subunit is used to determine whether the camera module recognizes the target object when the radar module recognizes the target object; if the camera module recognizes the target object, the camera module is determined to be a target control module; if the camera module does not recognize the target object, the radar module is determined to be a target control module; the third identification subunit is used to determine whether the voice module recognizes the target object when the radar module does not recognize the target object, and if the voice module recognizes the target object, the voice module is determined to be a target control module; if the voice module does not recognize the target object, the step of returning to respectively obtain the recognition results of the target object by the radar module, the voice module and the camera module.

[0134] In some optional embodiments, the control unit 1303 includes: a first control subunit, a second control subunit and a third control subunit; wherein the first control subunit is used to determine the current face of the face recognition module and the current picture in the cat's eye module according to the recognition result of the target object by the camera module; determine whether the current face is in the center of the current picture; when the current face is not in the center of the current picture, control the pan-tilt module to rotate in the horizontal and vertical directions so that the current face is in the center of the current picture; when the current face is in the center of the current picture, recognize the current face based on the preset face information and control the opening and closing of the door lock body according to the recognition result, or monitor the target object in real time through the camera module; the second control subunit is used to determine the current detection position according to the recognition result of the target object by the radar module; control the pan-tilt module to rotate to the current detection position; obtain the current recognition result of the camera module on the pan-tilt module at the current detection position; determine whether the current recognition result contains the target object; when the current recognition result contains the target object, The target control module is updated to a camera module, and based on the camera module and its current recognition result of the target object, the pan-tilt module is controlled to rotate accordingly in the horizontal and vertical directions, so that the camera module on the pan-tilt module can track the target object in real time; when the current recognition result does not include the target object, the step of determining the current detection position according to the recognition result of the target object by the radar module is returned; a third control subunit is used to determine the current sound position according to the recognition result of the target object by the voice module; control the pan-tilt module to rotate to the current sound position; obtain the current detection result of the radar module at the current sound position; determine whether the current detection result includes the target object; when the current detection result includes the target object, the target control module is updated to a radar module, and based on the radar module and its current detection result of the target object, the pan-tilt module is controlled to rotate accordingly in the horizontal and vertical directions, so that the camera module on the pan-tilt module can track the target object in real time; when the current detection result does not include the target object, the step of determining the current sound position according to the recognition result of the target object by the voice module is returned.

[0135] In some optional embodiments, the first control subunit includes: a multi-person determination level and a risk determination level; wherein the multi-person determination level is used to, when the number of faces in the current face is equal to 1, execute the step of determining whether the current face is in the center of the current picture; when the number of faces in the current face is greater than 1, control the pan / tilt module to rotate in sequence in a preset direction, and obtain the recognition result of at least one person by the camera module at each rotation position; obtain the face recognition result of the corresponding person based on the recognition result of each person corresponding to each rotation position, and perform opening and closing control on the door lock body based on the face recognition result. control; among them, when the face recognition result is a registered person of the smart door lock, the door lock body is controlled to open, and the registered person is a person whose identity has been entered in the smart door lock; when the face recognition result is a non-registered person of the smart door lock, the door lock body is controlled to close; the risk warning level is used to count the number of recognitions of non-registered persons; it is judged whether the non-registered person belongs to the preset marked person and whether the number of recognitions is greater than the preset number threshold; when the non-registered person does not belong to the preset marked person and the number of recognitions is greater than the preset number threshold, it is determined that the non-registered person is a risk person of the smart door lock, and the smart door lock is controlled to issue a risk warning.

[0136] In some optional embodiments, the device also includes: a detection unit, used to detect whether a pan-tilt control signal generated corresponding to the target user's operation on a preset pan-tilt control interface is received; wherein the target user is a registered person of the smart door lock; the preset pan-tilt control interface includes an operation interface set on a mobile terminal APP that communicates with the smart door lock, and an operation interface set on a preset screen installed on the indoor part of the door lock body of the smart door lock, and the operation interface is used to control, configure and view the pan-tilt module of the smart door lock; when the pan-tilt control signal generated corresponding to the target user's operation on the preset pan-tilt interface is not received, the steps of respectively obtaining the recognition results of the target object by the radar module, the voice module and the camera module are executed; when the pan-tilt control signal generated corresponding to the target user's operation on the preset pan-tilt interface is received, the pan-tilt module of the smart door lock is controlled accordingly based on the pan-tilt control signal.

[0137] In some optional embodiments, the device also includes: a configuration unit, which is used to set function activation buttons corresponding to the package monitoring function and the privacy avoidance function in the mobile terminal APP, and the target user generates corresponding function control signals by operating different function activation buttons, and executes the corresponding pan-tilt control function of the smart door lock based on the function control signals.

[0138] In some optional embodiments, the configuration unit includes: a first configuration subunit and a second configuration subunit; wherein the first configuration subunit is used to control the pan-tilt module of the smart door lock to rotate in sequence in a preset direction until the monitoring screen contains a package or other persons carrying the package and stops rotating; identify the current identity information of the package or other persons carrying the package according to the camera module on the pan-tilt module, and determine whether the current identity information belongs to a preset marked person; when the current identity information belongs to the preset marked person, identify whether there is someone in the room according to the radar module, and control the smart door lock to perform corresponding processing according to the result of the personnel identification; the second configuration subunit, The pan / tilt module used to control the smart door lock is rotated to the neighbor identification area, and the neighbor identification area refers to the neighbor smart door lock contained in the monitoring screen; the current door lock state of the neighbor smart door lock is identified according to the camera module on the pan / tilt module; when it is identified that the neighbor smart door lock is currently in the door lock body open state, the pan / tilt module is controlled to rotate to a direction away from the neighbor identification area, and the pan / tilt module is controlled to rotate back to the neighbor identification area after a preset time; if it is identified that the neighbor smart door lock is still in the door lock body open state, the pan / tilt module is controlled to rotate to a direction away from the neighbor identification area again, and the rotation of the pan / tilt module is stopped when it is identified that the neighbor smart door lock is currently in the door lock body closed state.

[0139] In some optional embodiments, the first configuration subunit includes: a package processing level, which is used to control the smart door lock to perform a first preset voice broadcast processing when it is recognized that there is someone indoors, so as to remind the person indoors to open the door and pick up the package; when it is recognized that there is no one indoors, the smart door lock is controlled to perform a second preset voice broadcast processing to remind other people outside to place the package at the door, and the current location information of the package and the current facial information of other people are identified through the camera module of the smart door lock, and the current location information and the current facial information are sent to the manager of the smart door lock, and at the same time, the pan / tilt module of the smart door lock is controlled to rotate downward to monitor the package in real time.

[0140] In some optional embodiments, the second configuration subunit includes: a privacy avoidance level, which is used to exit the privacy avoidance function if the radar module recognizes the location information of other people, and control the gimbal module to rotate accordingly in the horizontal and vertical directions based on the location information, so that the gimbal module can track other people in real time.

[0141] The further functional description of each of the above units is the same as that of the above corresponding embodiments, and will not be repeated here. In this embodiment, the pan / tilt control device of the smart door lock is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0142] The pan / tilt control device of the smart door lock of the embodiment of the present invention can realize accurate recognition and real-time tracking of objects outside the door, which not only improves the control efficiency of the door lock, but also enhances the user experience and meets the demand for smart use of the door lock.

[0143] The present invention also provides a computer device, see Fig.14 , Fig.14 is a schematic diagram of the structure of the above-mentioned computer device provided in an optional embodiment of the present invention, such as Fig.14 As shown, the computer device includes: one or more processors 10, memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components utilize different buses to communicate with each other, and can be installed on a common mainboard or installed in other ways as required. The processor can process the instructions executed in the computer device, including instructions stored in the memory or on the memory to display the graphical information of the GUI on an external input / output system (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each computer device provides part of the necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Fig.14 A processor 10 is taken as an example.

[0144] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0145] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.

[0146] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0147] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0148] The computer device also includes a communication interface 30, which is used for the main control chip to communicate with other devices or a communication network.

[0149] A computer-readable storage medium is also provided in an embodiment of the present invention. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium and downloaded through a network, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor main control chip or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor or hardware, the method shown in the above embodiment is implemented.

[0150] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A PTZ control method for a smart door lock, the smart door lock comprising a door lock body and a radar module, a voice module, a camera module and a PTZ module arranged on the door lock body; wherein: The camera module is mounted on the pan-tilt module, and when the pan-tilt module rotates in the horizontal and vertical directions, the camera module rotates accordingly, and the method includes: Obtain recognition results of the target object by the radar module, voice module and camera module respectively; Determine a target control module based on the recognition results of the target object by each module; Based on the target control module and its recognition result of the target object, the pan-tilt module is controlled to rotate accordingly in the horizontal and vertical directions, so that the camera module on the pan-tilt module can track the target object in real time.

2. The PTZ control method of the smart door lock according to claim 1, characterized in that: The determining of the target control module based on the recognition results of the target object by each module includes: Determine whether the radar module recognizes the target object; When the radar module recognizes the target object, it is determined whether the camera module recognizes the target object; if the camera module recognizes the target object, it is determined that the camera module is a target control module; if the camera module does not recognize the target object, it is determined that the radar module is a target control module; When the radar module fails to recognize the target object, it is determined whether the voice module recognizes the target object. If the voice module recognizes the target object, the voice module is determined to be the target control module. If the voice module fails to recognize the target object, the process returns to the step of respectively obtaining the recognition results of the target object by the radar module, the voice module and the camera module.

3. The PTZ control method of the smart door lock according to claim 1, characterized in that: The camera module includes a face recognition module and a cat's eye module; if the camera module is a target control module, then the target control module and its recognition result of the target object are used to control the pan-tilt module to rotate accordingly in the horizontal and vertical directions, so that the camera module on the pan-tilt module can track the target object in real time, including: Determine the current face of the face recognition module and the current picture of the cat's eye module respectively according to the recognition result of the target object by the camera module; Determining whether the current face is located at the center of the current picture; When the current face is not at the center of the current picture, controlling the pan / tilt module to rotate in the horizontal and vertical directions so that the current face is at the center of the current picture; When the current face is at the center of the current picture, the current face is recognized based on preset face information and the opening and closing of the door lock body is controlled according to the recognition result, or the target object is monitored in real time through the camera module.

4. The PTZ control method of the smart door lock according to claim 3, characterized in that: If the radar module is a target control module, then the control of the pan / tilt module to rotate accordingly in the horizontal and vertical directions based on the target control module and its recognition result of the target object, so that the camera module on the pan / tilt module tracks the target object in real time, includes: Determining a current detection position according to a recognition result of the target object by the radar module; Controlling the pan / tilt module to rotate to the current detection position; Obtaining a current recognition result of the camera module on the pan / tilt module at the current detection position; Determining whether the current recognition result includes the target object; When the current recognition result includes the target object, the target control module is updated to a camera module, and based on the camera module and the current recognition result of the target object, the pan-tilt module is controlled to rotate accordingly in the horizontal and vertical directions, so that the camera module on the pan-tilt module tracks the target object in real time; When the current recognition result does not include the target object, the process returns to the step of determining the current detection position according to the recognition result of the target object by the radar module.

5. The PTZ control method of the smart door lock according to claim 4, characterized in that: If the voice module is a target control module, then the control of the pan / tilt module to rotate accordingly in the horizontal and vertical directions based on the target control module and its recognition result of the target object, so that the camera module on the pan / tilt module tracks the target object in real time, includes: Determine the current sound position according to the recognition result of the target object by the voice module; Controlling the pan / tilt module to rotate to the current sound emission position; Obtaining a current detection result of the radar module at the current sound emission position; Determining whether the current detection result contains the target object; When the current detection result includes the target object, the target control module is updated to a radar module, and based on the radar module and its current detection result of the target object, the pan-tilt module is controlled to rotate accordingly in the horizontal and vertical directions, so that the camera module on the pan-tilt module can track the target object in real time; When the current detection result does not include the target object, the process returns to the step of determining the current sound position according to the recognition result of the target object by the speech module.

6. The PTZ control method of the smart door lock according to claim 3, characterized in that: Before determining whether the current face is located at the center of the current picture, the method further includes: When the number of faces in the current face is equal to 1, executing the step of determining whether the current face is located at the center of the current picture; When the number of faces in the current face group is greater than 1, the pan / tilt module is controlled to rotate in sequence in a preset direction, and the recognition result of the camera module for at least one person at each rotation position is obtained accordingly; the face recognition result of the corresponding person is obtained based on the recognition result of each person corresponding to each rotation position, and the door lock body is controlled to be opened and closed based on the face recognition result; wherein, when the face recognition result is a registered person of the smart door lock, the door lock body is controlled to be opened, and the registered person is a person whose identity has been entered in the smart door lock; when the face recognition result is a non-registered person of the smart door lock, the door lock body is controlled to be closed.

7. The PTZ control method of the smart door lock according to claim 6, characterized in that: When the face recognition result is a non-registered person of the smart door lock, the method further includes: Count the number of identifications of non-registered persons; Determine whether the non-registered person belongs to a preset marked person and whether the number of identifications is greater than a preset number threshold; When the non-registered person does not belong to the preset marked person and the number of identification times is greater than the preset number threshold, the non-registered person is determined to be a risk person of the smart door lock, and the smart door lock is controlled to issue a risk warning.

8. The PTZ control method of the smart door lock according to claim 7, characterized in that: Before respectively obtaining the recognition results of the target object by the radar module, the voice module and the camera module, the method further includes: Detect whether a pan-tilt control signal corresponding to an operation of a preset pan-tilt control interface by a target user is received; wherein the target user is a registered person of the smart door lock; the preset pan-tilt control interface includes an operation interface set on a mobile terminal APP that communicates with the smart door lock, and an operation interface set on a preset screen installed on the indoor part of the door lock body of the smart door lock, and the operation interface is used to control, configure and view the pan-tilt module of the smart door lock; When no pan / tilt control signal generated by the target user operating the preset pan / tilt interface is received, the steps of respectively obtaining recognition results of the target object by the radar module, the voice module and the camera module are performed; When a pan / tilt control signal generated corresponding to the target user operating a preset pan / tilt interface is received, the pan / tilt module of the smart door lock is controlled accordingly based on the pan / tilt control signal.

9. The PTZ control method of the smart door lock according to claim 8, characterized in that: The mobile terminal APP is also provided with function activation buttons corresponding to the package monitoring function and the privacy avoidance function. The target user generates corresponding function control signals by operating different function activation buttons, and executes the corresponding pan / tilt control function of the smart door lock based on the function control signals.

10. The PTZ control method of the smart door lock according to claim 9, characterized in that: When the target user presses the function activation button corresponding to the package monitoring function through the mobile terminal APP, the target user generates a corresponding function control signal by operating different function activation buttons, and executes the corresponding pan / tilt control function of the smart door lock based on the function control signal, including: The pan / tilt module of the smart door lock is controlled to rotate in a preset direction in sequence until the monitoring screen contains a package or other persons carrying a package and stops rotating; Identify the current identity information of the package or other persons carrying the package according to the camera module on the pan / tilt module, and determine whether the current identity information belongs to a preset marked person; When the current identity information belongs to a preset marked person, the radar module is used to identify whether there is anyone in the room, and the smart door lock is controlled to perform corresponding processing according to the result of the personnel identification.

11. The PTZ control method of the smart door lock according to claim 10, characterized in that: The controlling of the intelligent door lock to perform corresponding processing according to the personnel situation identification result includes: When someone is identified as being inside the room, the smart door lock is controlled to perform a first preset voice broadcast process to remind the person inside to open the door and take the package; When it is recognized that there is no one indoors, the smart door lock is controlled to perform a second preset voice broadcast processing to remind other people outside to place the package at the door, and the current location information of the package and the current facial information of other people are identified by the camera module of the smart door lock, and the current location information and the current facial information are sent to the administrator of the smart door lock. At the same time, the pan / tilt module of the smart door lock is controlled to rotate downward to monitor the package in real time.

12. The PTZ control method of the smart door lock according to claim 9, characterized in that: When the target user presses the function activation button corresponding to the privacy avoidance function through the mobile terminal APP, the target user generates a corresponding function control signal by operating different function activation buttons, and executes the corresponding pan / tilt control function of the smart door lock based on the function control signal, including: Control the pan / tilt module of the smart door lock to rotate to the neighbor identification area, where the neighbor identification area refers to the monitoring screen containing the neighbor smart door lock; Identify the current door lock state of the neighbor smart door lock according to the camera module on the pan / tilt module; When it is identified that the neighbor smart door lock is currently in the door lock body open state, the pan-tilt module is controlled to rotate to a direction away from the neighbor identification area, and the pan-tilt module is controlled to rotate back to the neighbor identification area after a preset time; if it is identified that the neighbor smart door lock is still in the door lock body open state, the pan-tilt module is controlled to rotate to a direction away from the neighbor identification area again, and the rotation of the pan-tilt module is stopped when it is identified that the neighbor smart door lock is currently in the door lock body closed state.

13. The PTZ control method of the smart door lock according to claim 12, characterized in that: In the process of controlling the pan / tilt module to rotate toward a direction away from the neighbor identification area, the method further includes: If the radar module recognizes the location information of other people, the privacy avoidance function is exited, and the pan / tilt module is controlled to rotate accordingly in the horizontal and vertical directions based on the location information, so that the pan / tilt module can track the other people in real time.

14. A pan / tilt control device for a smart door lock, the smart door lock comprising a door lock body and a radar module, a voice module, a camera module and a pan / tilt module arranged on the door lock body; wherein: The camera module is mounted on the pan-tilt module, and when the pan-tilt module rotates in the horizontal and vertical directions, the camera module rotates accordingly, and the device is characterized in that: An acquisition unit, used to respectively acquire recognition results of the target object by the radar module, the voice module and the camera module; An identification unit, used to determine a target control module based on the identification results of each module on the target object; The control unit is used to control the pan / tilt module to rotate accordingly in the horizontal and vertical directions based on the target control module and its recognition result of the target object, so that the camera module on the pan / tilt module can track the target object in real time.

15. A computer device, characterized in that: The computer device includes: a memory and a processor, the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the pan / tilt control method of the smart door lock according to any one of claims 1 to 13 by executing the computer instructions.

16. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the pan-tilt control method for the smart door lock according to any one of claims 1 to 13.

Images