Prompt methods and related equipment

Abstract

This application discloses a notification method and related device. The smart door lock can detect knocking vibrations, collect vibration data generated by the knocking vibrations, and then identify the identity of the knocking user based on the collected vibration data to determine whether the doorbell needs to be rung. If it is determined that the doorbell needs to be rung, the smart door lock can also play different doorbell notification sounds based on the frequency and / or amplitude of the collected vibration data. This prevents doorbell ringing caused by vibrations other than those generated by the knocking action, increases the diversity of doorbell notification sounds, and improves the user experience.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to a prompting method and related equipment. Background Technology

[0002] With the development of smart homes, smart door locks are widely used in people's lives. When people visit private residences or office areas, they usually ring the doorbell or knock to alert the people inside. However, in many scenarios, such as children being too short to reach the doorbell button, the doorbell button being installed in a non-fixed location requiring users to actively search for it, and users carrying heavy objects making it inconvenient to press the doorbell, ringing the doorbell can cause many inconveniences. Therefore, in actual visits, users generally prefer to knock to alert others. However, if the knocking sound is too soft, the people inside may not easily notice someone knocking, resulting in a poor user experience. Summary of the Invention

[0003] This application provides a notification method and related equipment that rings the doorbell when there is a knocking action, thereby improving the user experience.

[0004] In a first aspect, embodiments of this application provide a notification method applied to a smart door lock, wherein the smart door lock is installed on a door and includes a doorbell. The method includes: the smart door lock detecting vibration of the door, the duration of which is a first duration; the smart door lock generating first vibration data based on the vibration; the smart door lock determining one or both of knocking frequency and knocking force, and the identity of the knocking user, based on the first vibration data; and the smart door lock notifying the doorbell to ring and playing a doorbell notification sound, wherein the doorbell notification sound indicates the identity of the knocking user, and one or more of the volume, rhythm, and tone of the doorbell notification sound are determined by one or both of the knocking frequency and knocking force.

[0005] By implementing the method of the first aspect in this application embodiment, the smart door lock can detect the user's knocking action, and when it is confirmed that a knocking action has occurred, it notifies the doorbell to ring, thereby improving the user experience.

[0006] In one possible implementation, before the smart lock generates first vibration data based on vibration, the method further includes: the smart lock generating second vibration data based on vibration, the second vibration data being generated in a first portion of a first duration, and the first vibration data being generated in a second portion of the first duration, wherein the first portion precedes the second portion; the smart lock determines that the vibration characteristics indicated by the second vibration data match the vibration characteristics of a knocking action. In this way, the smart lock can initially determine that a knocking action has occurred.

[0007] In one possible implementation, before the smart lock generates second vibration data based on vibration, the method further includes: the smart lock detecting a human body approaching the smart lock. This allows the smart lock to transition from a low-power mode to a normal operating mode and begin detecting the user's knocking action.

[0008] In one possible implementation, the smart lock determines one or both of knocking frequency and knocking force based on first vibration data. Specifically, this includes: the smart lock determining the knocking frequency based on the first vibration frequency, where a higher first vibration frequency results in a higher knocking frequency; and / or, the smart lock determining the knocking force based on a first vibration amplitude, where a larger first vibration amplitude results in a larger knocking force; wherein the first vibration frequency is the frequency of the first vibration data, and the first vibration amplitude is the amplitude of the first vibration data. In this way, the smart lock can determine the knocking frequency and / or knocking force based on the first vibration data.

[0009] In one possible implementation, the smart lock determines the identity of the user knocking on the door based on first vibration data. Specifically, this involves: the smart lock preprocessing the first vibration data to obtain preprocessed first vibration data; and the smart lock matching the preprocessed first vibration data with pre-stored knocking data of a first user. If the match is successful, the user knocking is identified as the first user. In this way, the smart lock can determine the identity of the user knocking on the door based on the first vibration data.

[0010] In one possible implementation, the smart lock determines the doorbell alert tone based on one or both of the knocking frequency and knocking force, as well as the identity of the knocking user. Specifically, if the smart lock determines that the knocking frequency is less than a first preset knocking frequency and / or the knocking force is less than a first preset knocking force, then the smart lock determines the doorbell alert tone as a first doorbell alert tone; or, if the smart lock determines that the knocking frequency is greater than the first preset knocking frequency and / or the knocking force is greater than the first preset knocking force, then the smart lock determines the doorbell alert tone as a second doorbell alert tone. The first and second doorbell alert tones differ in one or more of the volume, rhythm, and tone, and the content of both the first and second doorbell alert tones includes the identity of the knocking user. In this way, the smart lock can comprehensively determine the doorbell alert tone based on one or both of the knocking frequency, knocking force, and the identity of the knocking user.

[0011] In one possible implementation, before the smart lock notifies the doorbell to ring and play a doorbell alert, the method further includes: the smart lock using a camera to capture an image of the area in front of the door; and the smart lock detecting that the image includes a human figure and / or a face. Thus, for vibrations triggered by no one, the smart lock may not notify the doorbell to ring.

[0012] In one possible implementation, after the smart lock notifies the doorbell to ring and plays a doorbell alert tone, the method further includes the smart lock entering a low-power mode. This reduces the power consumption of the smart lock.

[0013] In a second aspect, embodiments of this application provide an electronic device including one or more processors and one or more memories; wherein the one or more memories are coupled to one or more processors, and the one or more memories are used to store computer program code, the computer program code including computer instructions, and when the one or more processors execute the computer instructions, the electronic device performs the method in any possible implementation of the first aspect described above.

[0014] Thirdly, embodiments of this application provide a computer storage medium storing a computer program, the computer program including program instructions, which, when executed on an electronic device, cause the electronic device to perform the method in any possible implementation of the first aspect described above.

[0015] Fourthly, embodiments of this application provide a computer program product that, when run on a computer, causes the computer to execute the method in any possible implementation of the first aspect described above. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the system architecture of a smart door lock provided in an embodiment of this application;

[0017] Figure 2 This is a flowchart illustrating a prompting method provided in an embodiment of this application;

[0018] Figure 3 This is a flowchart illustrating another prompting method provided in an embodiment of this application;

[0019] Figure 4 This is a schematic diagram of the structure of a smart door lock provided in an embodiment of this application;

[0020] Figure 5 This is a schematic diagram of the structure of a chip system provided in an embodiment of this application. Detailed Implementation

[0021] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. In the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B; "and / or" in the text is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, in the description of the embodiments of this application, "multiple" refers to two or more than two.

[0022] It should be understood that the terms "first," "second," etc., in the specification, claims, and drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0023] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application can be combined with other embodiments.

[0024] Currently, when people visit private residences or offices, they usually ring the doorbell or knock to alert those inside. Existing doorbell triggers are based on the user pressing a doorbell button on the outside of the door. However, in many scenarios, such as children being too short to reach the doorbell button, the doorbell button's location being unpredictable and requiring the user to actively search for it, or users carrying heavy objects making it inconvenient to press the doorbell, ringing the doorbell causes numerous inconveniences. Therefore, in actual visits, users generally prefer to knock. However, in situations where the knocking is quiet or the door has good sound insulation, those inside may not easily notice someone knocking, resulting in a poor user experience.

[0025] This application provides a notification method whereby a smart door lock can detect door vibrations (i.e., vibrations triggered by a user's knocking action), collect vibration data generated by the knocking vibrations, identify the knocking user based on the collected vibration data, and determine whether the doorbell needs to be rung. If it is determined that the doorbell needs to be rung, the smart door lock can also play different doorbell notification sounds based on the frequency and / or amplitude of the collected vibration data. This prevents doorbell ringing caused by vibrations other than those generated by the knocking action, increases the diversity of doorbell notification sounds, and improves the user experience.

[0026] The method provided in this application mainly addresses the following three problems:

[0027] Question 1: How does the smart door lock collect and detect vibration data during the vibration triggered by the user knocking on the door?

[0028] Question 2: How to identify the identity of the user knocking on the door based on the data collected from the vibrations triggered by the user knocking on the door?

[0029] Question 3: During the vibration triggered by a user knocking on the door, the doorbell may be accidentally triggered by other vibrations.

[0030] To address the above three issues, the embodiments of this application provide the following solutions:

[0031] To address the first issue, this application embodiment utilizes a vibration module to acquire vibration data collected by a gyroscope. If vibration data exceeding a preset threshold is detected, vibration data is continuously collected for a period of time (e.g., 2 seconds, 3 seconds, etc.). The vibration data is then subjected to noise reduction, envelope extraction, template matching (and / or envelope fitting) / machine learning offline diagnosis, etc., to further determine whether the user has engaged in knocking behavior, i.e., whether a knocking event has occurred.

[0032] To address the second issue, this application's embodiments establish a gyroscope angular velocity vibration data template feature library and an offline diagnostic library. The offline diagnostic library is used to identify the identity of the user knocking on the door. The same user can knock on the door using the same knocking method, and user identification can be completed without an internet connection.

[0033] To address issue 3, this application's embodiments add an image decision module and a ringing decision module. The ringing decision module collects data from the image decision module and the vibration module to comprehensively determine whether the doorbell needs to ring, thereby eliminating doorbell ringing caused by false triggering / false identification. In other words, vibrations without human intervention will not cause the doorbell to ring.

[0034] Furthermore, the embodiments of this application can also determine the urgency and force of knocking based on the frequency and amplitude of the angular velocity vibration model and the frequency and amplitude of the knocking vibration. Further, different doorbell prompts can be played based on different urgency and force of knocking, that is, different doorbell prompts are played when the urgency (vibration frequency) of knocking and the force (vibration amplitude) of knocking are different.

[0035] The following is a schematic diagram of the system architecture of the smart door lock 100 provided in the embodiments of this application.

[0036] like Figure 1 As shown, the system architecture may include a low-power system, a decision-making module, an image acquisition and processing module, a vibration module, an operation and maintenance module, a hardware adaptation module, and a hardware layer (or hardware module).

[0037] The low-power system interacts with the ringing decision module. After detecting and recognizing a valid knocking event, the ringing decision module can report the decision result to the low-power system. This decision result is used to instruct the low-power system to trigger the doorbell to ring.

[0038] The system architecture comprises multiple modules, each of which includes multiple sub-modules (or sub-components). The following section provides a detailed description of the functions of each module and its sub-modules within this system architecture:

[0039] 1. Decision Module:

[0040] The decision-making module is a background resident process in the low-power system. It can be based on a software operating system (OS), such as Huawei's Harmony®, to provide the operation and maintenance of the ringing decision and gyroscope data detection threads. The decision-making module is responsible for summarizing and judging the detection results, thereby deciding whether to initiate a doorbell ringing event.

[0041] The decision-making module can include two sub-modules: the ringing decision module and the image decision module.

[0042] Ringing Decision Module:

[0043] The ringing decision module can receive the knocking matching results sent by the vibration module and judge the knocking matching results to determine whether a knocking event has occurred. At the same time, the ringing decision module can also send a request to the image decision module to obtain human shape and / or face detection results. After receiving the human shape and / or face detection results sent by the image decision module, the ringing decision module can make a comprehensive decision on whether to initiate a doorbell ringing event based on the knocking matching results and the human shape and / or face detection results. If so, it further determines the corresponding doorbell prompt tone to be played based on the knocking matching results.

[0044] Image decision module:

[0045] The image decision module can receive a request from the ringing decision module to obtain human and / or face detection results. Upon receiving this request, the image decision module can send an instruction to the image acquisition and processing module to perform human and / or face detection. The image acquisition and processing module can perform a series of processes on the door-facing image captured by the camera to obtain human and / or face detection results, and then send these results to the image decision module. If the image decision module determines that there is indeed a human and / or face based on the human and / or face detection results, it can then send the human / face detection results sent by the image acquisition and processing module to the ringing decision module.

[0046] 2. Image Acquisition and Processing Module:

[0047] The image acquisition and processing module is responsible for acquiring, preprocessing, detecting, and reporting the detection results of images in front of the gate.

[0048] The image acquisition and processing module can include four sub-modules: image acquisition module, image preprocessing module, image detection module, and result reporting module.

[0049] Image acquisition module:

[0050] The image acquisition module can acquire the door-front image data captured by the camera and send the door-front image data captured by the camera to the image preprocessing module.

[0051] In some embodiments, the door-front image data acquired by the image acquisition module may also be door-front image data acquired by a camera after being processed by an image signal processor.

[0052] Image preprocessing module:

[0053] The image preprocessing module can preprocess the door-front image data sent by the image acquisition module and then send the preprocessed door-front image data to the image detection module.

[0054] Image preprocessing can remove irrelevant information from images, restore useful real information, simplify image data to the greatest extent, and improve the detectability of relevant information.

[0055] Image preprocessing steps may include, but are not limited to, the following: image downsampling, image filtering, image smoothing, and image enhancement.

[0056] The specific execution details of each step in the image preprocessing process are existing technologies, and relevant information in existing technologies can be consulted. They will not be repeated here.

[0057] Image detection module:

[0058] The image detection module can receive the preprocessed door image sent by the image preprocessing module, then perform human and / or face detection on the preprocessed door image data to obtain the human and / or face detection results, and send the human and / or face detection results to the result reporting module.

[0059] The specific execution details of the human figure and / or face detection process are all existing technologies, and relevant information in existing technologies can be referenced. They will not be repeated here.

[0060] Result reporting module:

[0061] The result reporting module can receive human and / or face detection results sent by the image detection module and send the human and / or face detection results to the image decision module.

[0062] 3. Vibration module:

[0063] The vibration module is responsible for acquiring data from the gyroscope and performing noise reduction, envelope extraction, and door knocking event recognition on the data.

[0064] The vibration module can include six sub-modules: data acquisition module, data noise reduction module, envelope extraction module, knock matching module, low-power operation and maintenance module, and calibration module.

[0065] Data acquisition module:

[0066] The data acquisition module can acquire angular velocity data collected by the gyroscope and send the angular velocity data collected by the gyroscope to the data noise reduction module.

[0067] Data noise reduction module:

[0068] The data denoising module can receive angular velocity data collected by the gyroscope from the data acquisition module, perform denoising processing on the angular velocity data collected by the gyroscope to obtain denoised gyroscope data, and send the denoised gyroscope data to the envelope extraction module.

[0069] The noise reduction process may include, but is not limited to, the following steps: removing constant drift, angle random walk, rate random walk, quantization noise, and rate ramp.

[0070] The specific execution details of each step in the noise reduction process are existing technologies, and relevant information in existing technologies can be consulted. They will not be repeated here.

[0071] Envelope Extraction Module:

[0072] The envelope extraction module can receive the denoised gyroscope data sent by the data denoising module, extract the envelope of the denoised gyroscope data, obtain the envelope data, and send the envelope data to the door-knocking matching module.

[0073] Envelope extraction is the process of extracting the envelope of gyroscope data after noise reduction. Envelope extraction methods include, but are not limited to, the following: detection method, Hilbert transform, etc.

[0074] The envelope extraction methods used in the embodiments of this application are all existing technologies, and relevant information in the existing technologies can be referred to, which will not be repeated here.

[0075] Knock matching module:

[0076] The door-knocking matching module can receive the envelope data sent by the envelope extraction module, and then use the established template feature library and offline diagnostic library to perform template matching / neural network offline diagnostics on the envelope data to obtain the door-knocking matching result, and send the door-knocking matching result to the ringing decision module.

[0077] The knocking matching results may include whether a knocking event occurred, the frequency and / or force of the knocking, and the identity of the knocking user (who is knocking).

[0078] The specific execution process of the door-knocking matching module performing template matching / neural network offline diagnosis on the envelope data to obtain the door-knocking matching result will be described in detail in subsequent embodiments, and will not be elaborated here.

[0079] Low-power operation and maintenance module:

[0080] The low-power operation and maintenance module can maintain the gyroscope in low-power mode. After the passive infrared sensor (PIR) detects a living person approaching the smart door lock, the low-power operation and maintenance module can interrupt the gyroscope's low-power mode and allow the gyroscope to enter normal detection mode.

[0081] Calibration module:

[0082] The calibration module can calibrate the gyroscope after it is powered on.

[0083] In some embodiments, the gyroscope may have a built-in calibration function, in which case the calibration module may not be configured in the vibration module.

[0084] 4. Operation and Maintenance Module:

[0085] The operation and maintenance module is responsible for the operation and maintenance of each component in the door-knocking vibration doorbell process (also known as the ringing detection process), namely, component operation status monitoring and configuration. Specifically, the operation and maintenance module can perform configuration management, log collection and storage, system operation status statistics and reporting, etc. during the door-knocking vibration doorbell process, providing dynamic operation parameters to the ringing decision module.

[0086] The operation and maintenance module can include three sub-modules: statistics module, log module, and configuration management module.

[0087] Statistics module:

[0088] The statistics module can summarize and statistically analyze the operating status of each component and report it to the low-power system.

[0089] Log module:

[0090] The logging module can collect logs from various components, organize the logs, and store them locally or send them to a low-power system.

[0091] Configuration Management Module:

[0092] The configuration management module provides configuration capabilities, allowing customization of static runtime parameters for each component, such as component startup method, component running method, and component health status.

[0093] 5. Hardware adaptation module:

[0094] The hardware adaptation module is responsible for shielding hardware differences and providing standard capability interfaces for sub-modules in the vibration module and image acquisition and processing module.

[0095] The hardware adapter module can include four sub-modules: Image Signal Processor (ISP) driver, Gyroscope (GYRO) driver, Passive Infra-Red (PIR) sensor driver, and other sensor drivers.

[0096] Image signal processor driver:

[0097] Image signal processor drivers can mask the hardware differences of image signal processors and provide a standard acquisition interface for image acquisition and processing modules.

[0098] Gyroscope driver:

[0099] The gyroscope driver can shield the hardware differences of the gyroscope and provide a standard data acquisition interface and operation and maintenance interface for the vibration module.

[0100] Passive infrared sensor driver:

[0101] The passive infrared sensor driver can shield the hardware differences of the passive infrared sensor and provide a standard interface to the ringing decision module, so that the passive infrared sensor can instruct the ringing decision module to start the ringing detection process after detecting a living person approaching the smart door lock.

[0102] Other sensor drivers:

[0103] Other sensor drivers may include one or more sensor drivers besides those shown in the figure, which can mask the hardware differences of one or more other sensors and provide a standard interface.

[0104] Doorbell driver:

[0105] Doorbell drivers can mask the hardware differences of doorbells and provide a standard interface, thereby enabling the doorbell to ring when a knocking action occurs.

[0106] 6. Hardware layer:

[0107] The hardware layer is responsible for using multiple different hardware components to implement the hardware steps involved in the knocking, vibrating, and turning doorbell process, such as gyroscope data acquisition, human / face image acquisition, and doorbell ringing.

[0108] The hardware layer may include gyroscopes, passive infrared sensors, doorbells, image sensors, image signal processors, and other sensors.

[0109] Gyroscope:

[0110] A gyroscope can collect vibration data generated during the knocking process, which may include angular velocity data from the gyroscope.

[0111] Passive infrared sensor:

[0112] Passive infrared sensors can detect motion by detecting the energy generated by a moving living body (such as a human). For example, when a human body approaches a smart door lock, a passive infrared sensor can detect the moving human body and instruct the ringing decision module to initiate the ringing detection process.

[0113] doorbell:

[0114] The doorbell can receive commands or actions to trigger the ringing and play a specified doorbell tone.

[0115] Image sensor:

[0116] An image sensor is a component of a camera. A camera can include a lens and an image sensor for capturing images. An object can be projected onto the image sensor through the lens to generate an optical image. The image sensor converts the light signal into an electrical signal, which is then passed to an image signal processor to be converted into a digital image signal, such as a standard RGB or YUV format.

[0117] In this embodiment of the application, the camera can receive an instruction from the image acquisition and processing module to acquire an image in front of the door, then acquire the image data in front of the door, and send the acquired image data in front of the door to the image signal processor.

[0118] Image signal processor:

[0119] An image signal processor (ESP) can receive and process door-view image data captured and transmitted by a camera. For example, the ESP can convert the image data captured by the camera into a visually perceptible image, perform algorithmic optimizations on noise, brightness, skin tone, etc., and optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ESP can also be integrated into the camera.

[0120] Other sensors:

[0121] Other sensors may include one or more sensors in addition to those shown in the figure, such as fingerprint sensors, pressure sensors, touch sensors, etc.

[0122] It should be noted that, Figure 1 The system architecture of the smart door lock 100 (also referred to as electronic device 100) shown is merely an example; this system architecture may also include more... Figure 1 The embodiments of this application do not limit the scope of the invention to include other modules (such as locking modules, unlocking modules, etc.) or other components, or may include fewer modules or components, or may combine two or more modules or components, or may include different module or component configurations.

[0123] The following describes a prompting method provided by an embodiment of this application.

[0124] Figure 2 An exemplary illustration shows a prompting method flow provided by an embodiment of this application.

[0125] like Figure 2 As shown, this prompting method can be applied to smart door lock 100, which can be installed on a door. The specific steps of this prompting method are described in detail below:

[0126] Phase 1: Triggering the ring detection process

[0127] S201-S202, when the passive infrared sensor PIR detects a living person approaching, it sends a command to the ringing decision module to start ringing detection.

[0128] Specifically, a passive infrared sensor (PIR) can sense motion by detecting the energy generated by a moving living body (such as a human). When someone approaches the smart door lock 100 from outside the door, the PIR can detect the moving human body approaching the smart door lock 100. Then, the PIR can send a command to the ringing decision module to initiate ringing detection. This command instructs the ringing decision module to trigger the execution of the ringing detection process.

[0129] S203, The ringing decision module sends a command to the vibration module to start the vibration module.

[0130] Specifically, after receiving the command to start ring detection from the passive infrared sensor PIR, the ringing decision module can send a command to start the vibration module. This command is used to instruct the vibration module to start, that is, the vibration module can enter the normal operation mode from the low power mode.

[0131] S204. The vibration module sends a command to the gyroscope to start the gyroscope.

[0132] Specifically, after receiving the command to start the vibration module from the ringing decision module, the vibration module can send a command to start the gyroscope, which is used to instruct the gyroscope to start.

[0133] S205, the gyroscope exits low-power mode and begins collecting vibration data.

[0134] Specifically, after receiving the start command from the vibration module, the gyroscope can start up, that is, the gyroscope can exit the low-power mode and enter the normal detection mode to start collecting vibration data.

[0135] The vibration data may include, but is not limited to, angular velocity data from a gyroscope.

[0136] Phase Two: Execute the Ringing Detection Process

[0137] S206-S207, the vibration module sends a request to the gyroscope to obtain vibration data 1, and then the gyroscope sends vibration data 1 to the vibration module.

[0138] Specifically, the vibration module can send a request to the gyroscope to acquire vibration data 1, and after receiving the request, the gyroscope can send vibration data 1 to the vibration module.

[0139] The vibration data 1 can be used by the vibration module to determine whether there is vibration data generated during the door-knocking vibration triggered by the user's knocking behavior.

[0140] In some embodiments, the vibration module can directly read the vibration data 1 collected by the gyroscope.

[0141] S208. The vibration module determines, based on vibration data 1, that vibration data 1 contains vibration data generated by knocking behavior.

[0142] For example, after receiving vibration data 1 sent by the gyroscope, the vibration module can store the vibration data 1 sent by the gyroscope in a certain time window (sliding window). Assuming that the certain time window is 1 second, the vibration data 1 sent by the gyroscope is 50 angular velocity data (or angular velocity detection values). Then, the vibration module can determine whether there is an angular velocity value exceeding a preset threshold among the 50 angular velocity data. If there is an angular velocity value exceeding the preset threshold, the vibration module can determine that the vibration data 1 contains vibration data generated during the door-knocking vibration process triggered by the user's door-knocking behavior (or door-knocking action).

[0143] In one possible implementation, the vibration module can determine whether the vibration characteristics indicated by the above 50 angular velocity data match the vibration characteristics of the knocking behavior. If they match, the vibration module initially determines that the knocking behavior exists.

[0144] S209-S210, the vibration module sends a request to the gyroscope to acquire vibration data 2, and then the gyroscope sends vibration data 2 to the vibration module.

[0145] Specifically, the vibration module can send a request to the gyroscope to acquire vibration data 2, and after receiving the request, the gyroscope can send vibration data 2 to the vibration module.

[0146] The vibration data 2 can be used by the vibration module to complete subsequent steps in the ringing detection process.

[0147] In some embodiments, the vibration module can directly read the vibration data 2 collected by the gyroscope.

[0148] S211-S212, the vibration module obtains the knocking matching result based on vibration data 2, and sends the knocking matching result to the ringing decision module.

[0149] Specifically, after receiving vibration data 2 from the gyroscope, the vibration module can preprocess the vibration data 2, and then use the established template feature library and offline diagnostic library to perform template matching / neural network offline diagnosis on the preprocessed vibration data 2 to obtain the knocking matching result. Afterwards, the vibration module can send this knocking matching result to the ringing decision module.

[0150] The knocking match result may include information determining whether a knocking event occurred. If so, the knocking match result may also include information such as knocking frequency and / or knocking force, and the identity of the knocking user (who is knocking).

[0151] The vibration data 2 mentioned above can be vibration data collected by the gyroscope for a period of time (e.g., 2 seconds, 3 seconds, etc.). In the embodiment of this application, the vibration data 2 mentioned above is angular velocity data collected by the gyroscope for 2 seconds.

[0152] The preprocessing of vibration data 2 by the vibration module may include, but is not limited to, the following steps:

[0153] 1. Data noise reduction processing:

[0154] For example, the vibration module can perform data noise reduction processing on vibration data 2 using the wavelet denoising principle.

[0155] The wavelet denoising process mainly includes three basic steps: performing a wavelet transform on the noisy signal; processing the wavelet coefficients obtained by the transform to remove the noise; and performing an inverse wavelet transform on the processed wavelet coefficients to obtain the denoised signal. Wavelet denoising can significantly remove a lot of noise from a noisy signal, and the denoised signal becomes smoother.

[0156] The specific execution process of data denoising using wavelet denoising is all existing technology, and relevant information on existing technology can be referenced. It will not be elaborated here.

[0157] It should be noted that the embodiments of this application only use wavelet denoising as an example to illustrate the data denoising process, and are not limited thereto. Other denoising methods can also be used for data denoising, and the embodiments of this application do not limit them.

[0158] 2. Extract the envelope:

[0159] For example, the vibration module can use detection or Hilbert transform to extract the envelope of the noise-reduced vibration data 2 to obtain the envelope data.

[0160] The specific execution process of envelope extraction using detection or Hilbert transform is existing technology, and relevant information on existing technology can be referenced. It will not be elaborated here.

[0161] It should be noted that the embodiments of this application only use the detection method or Hilbert transform as an example to illustrate the envelope extraction process, and are not limited thereto. Other methods can also be used for the envelope extraction process, and the embodiments of this application do not limit them.

[0162] The vibration module utilizes the established template feature library and offline diagnostic library to perform template matching / neural network offline diagnostics on the preprocessed vibration data2. The specific process is as follows:

[0163] 1. The vibration module detects whether a knocking event has occurred:

[0164] In one possible implementation, the vibration module can perform template matching between the preprocessed vibration data 2 and the pre-stored door-knocking vibration data template. If the matching is successful, the vibration module determines that a door-knocking event has occurred.

[0165] In another possible implementation, the vibration module can input the preprocessed vibration data 2 into a pre-trained neural network model (or offline diagnostic library) for offline neural network diagnosis (i.e., machine learning offline diagnosis). If the recognition accuracy is higher than the preset recognition accuracy threshold, the vibration module determines that a knocking event has occurred.

[0166] 2. The vibration module determines the knocking frequency and / or knocking force:

[0167] The vibration module can obtain the frequency and amplitude of the angular velocity data in the preprocessed vibration data 2. Furthermore, the vibration module can determine the knocking frequency based on the frequency of the angular velocity data. The higher the frequency of the angular velocity data, the higher the knocking frequency. The vibration module can also determine the knocking force based on the amplitude of the angular velocity data. The larger the amplitude of the angular velocity data, the greater the knocking force.

[0168] 3. The vibration module identifies the user knocking on the door (who is knocking):

[0169] The vibration module can identify the user who knocks on the door based on the same knocking habits of the same user. That is, if the same user knocks on the door in the same way, the vibration module can identify the user who knocks on the door.

[0170] In one possible implementation, the vibration module can perform template matching between the pre-processed vibration data 2 and the pre-stored door-knocking vibration data of all door-knocking users. If the pre-processed vibration data 2 successfully matches the door-knocking vibration data of one of the pre-stored door-knocking users 1, the vibration module can determine that the door-knocking user is door-knocking user 1.

[0171] In another possible implementation, the vibration module can input the preprocessed vibration data 2 into a pre-trained neural network model (or offline diagnostic library) of multiple knocking users to perform offline neural network diagnosis (i.e., machine learning offline diagnosis). If the recognition accuracy obtained by using the pre-trained neural network model of knocking user 1 is higher than the preset recognition accuracy threshold, the vibration module can determine that the knocking user is knocking user 1.

[0172] In another possible implementation, the identity of the knocking user identified by the vibration module does not match the pre-stored knocking users, or the recognition accuracy rate is lower than the preset recognition accuracy threshold. For example, the knocking user's knocking vibration data is not pre-stored, or the knocking user with pre-stored knocking vibration data deliberately uses a different knocking method than before. In this case, the vibration module can output that the knocking user's identity is "unknown user".

[0173] In some embodiments, the vibration module may not execute steps S206 and S209, and the gyroscope may not execute steps S207 and S210. Instead, the vibration module may actively read the vibration data collected by the gyroscope as needed for the ringing detection process, and execute steps S208 and S211 based on the read vibration data collected by the gyroscope.

[0174] In other embodiments, the vibration module may not execute steps S206 and S209. Instead, the gyroscope may actively execute steps S207 and S210 to send the collected vibration data to the vibration module in real time.

[0175] Phase Three: Implementing the Ringing Decision Process

[0176] S213, The ringing decision module determines that a knocking event has occurred based on the knocking matching results.

[0177] Specifically, after receiving the knock matching result sent by the vibration module, the ringing decision module can determine whether a knocking event has occurred. If so, the ringing decision module executes step S214. If not, subsequent steps do not need to be executed.

[0178] In one possible implementation, the vibration module can send a knock matching result to the ringing decision module when it is determined that a knocking event has occurred, but does not need to send a knock matching result to the ringing decision module when it is determined that no knocking event has occurred.

[0179] S214, The ringing decision module sends a request to the image decision module to obtain human and / or face detection results.

[0180] Specifically, after determining that a knocking event has occurred, the ringing decision module can send a request to the image decision module to obtain human and / or face detection results, which is used to request the image decision module to send human and / or face detection results.

[0181] S215, The image decision module sends an instruction to the image acquisition and processing module to perform human and / or face detection.

[0182] Specifically, after receiving a request from the ringing decision module to obtain human and / or face detection results, the image decision module can send an instruction to the image acquisition and processing module to perform human and / or face detection, thereby instructing the image acquisition and processing module to perform human and / or face detection.

[0183] S216-S217 The image acquisition and processing module performs human and / or face detection, obtains human and / or face detection results, and sends the human and / or face detection results to the image decision module.

[0184] Specifically, after receiving the instruction from the image decision module to perform human and / or face detection, the image acquisition and processing module can send an instruction to the camera to acquire an image of the area in front of the door. This instruction instructs the camera to acquire the image. Further, the image acquisition and processing module can acquire the image data of the area in front of the door captured by the camera, preprocess the image data, and then perform human and / or face detection on the preprocessed image data to obtain the human and / or face detection results. Afterward, the image acquisition and processing module can send the human and / or face detection results to the image decision module.

[0185] The door-front image data captured by the camera can be a certain number of frames (e.g., 5 frames) or a period of time (e.g., 150ms) continuously captured by the camera. The aforementioned human and / or face detection results may include whether there are human figures and / or faces in the door-front image data.

[0186] Optionally, the camera can continuously capture images of the area in front of the door without needing to wait for an instruction to capture such images before starting to capture them. The image acquisition and processing module can acquire only a certain number of frames or a period of time of the latest images captured by the camera.

[0187] In some embodiments, step S215 is optional. The image acquisition and processing module may periodically perform human and / or face detection and periodically send the human and / or face detection results to the image decision module.

[0188] S218-S219, The image decision module determines that there is a human figure and / or face in front of the door based on the human figure and / or face detection results, and sends the human figure and / or face detection results to the ringing decision module.

[0189] Specifically, after receiving the human figure and / or face detection results sent by the image acquisition and processing module, the image decision module can determine whether there is a human figure and / or face in front of the door. If so, it sends the human figure and / or face detection results to the ringing decision module.

[0190] Optionally, if not, the image decision module may not send the human figure and / or face detection result to the ringing decision module, and subsequent steps will not be executed.

[0191] S220-S221, the ringing decision module comprehensively determines the doorbell ringing event and the corresponding doorbell prompt tone to be played based on the knocking matching result and the human shape and / or face detection result, and sends the request to initiate doorbell ringing and the corresponding doorbell prompt tone to be played to the low power system.

[0192] Specifically, after receiving the human figure and / or face detection results sent by the image decision module, the ringing decision module can comprehensively determine whether to initiate a doorbell ringing event based on the door knocking matching results and the human figure and / or face detection results.

[0193] For example, if the ringing decision module determines that there is a human figure and / or face in front of the door based on the human figure and / or face detection results sent by the image decision module, and the ringing decision module has previously determined that a knocking event has occurred based on the knocking matching results in step S213, then the ringing decision module can comprehensively determine that a doorbell ringing event needs to be initiated.

[0194] Furthermore, the ringing decision module can also determine the appropriate doorbell tone to play based on the knocking frequency and / or knocking force in the knocking matching results, and the identity of the knocking user (who is knocking). The ringing decision module's determination of the appropriate doorbell tone to play includes, but is not limited to, the following three possible implementation methods:

[0195] Possible implementation method 1: The ringing decision module can determine the corresponding doorbell prompt tone to be played based solely on the knocking frequency and / or knocking force in the knocking matching results.

[0196] For example, if the knocking frequency is less than a preset frequency threshold (i.e., the frequency of the angular velocity vibration model), and / or the knocking force is less than a preset force threshold (i.e., the amplitude of the angular velocity vibration model), it may indicate that the user is making a normal knocking action without any destructive behavior. In this case, the doorbell alert tone could be a pre-set piece of music with a relatively gentle rhythm or a simple "ding-dong~ding-dong~ding-dong~" sound, etc. Optionally, the volume of the doorbell alert tone could be lower, and the tone of the doorbell alert tone could be gentler.

[0197] For example, when the knocking frequency exceeds a preset frequency threshold, and / or the knocking force exceeds a preset force threshold, it may indicate that the knocking user is engaging in abnormal knocking behavior, such as having an emergency or engaging in destructive behavior. In this case, the doorbell tone can be more urgent, and the doorbell tone can be an alarm-like tone, such as a rapid "beep beep beep," or "Abnormal behavior detected, please be careful!" etc. Optionally, the doorbell tone can be louder, and the tone can be more forceful.

[0198] In some embodiments, the ringing decision module can also pre-set multiple knocking frequency ranges, each corresponding to one or more different doorbell prompts in terms of rhythm, volume, and tone. For example, within different knocking frequency ranges, the lower the knocking frequency, the gentler the doorbell prompt rhythm, the lower the volume, and the softer the tone; conversely, the higher the knocking frequency, the more urgent the doorbell prompt rhythm, the higher the volume, and the harsher the tone. The ringing decision module can first determine which knocking frequency range the knocking frequency falls into, and then determine the corresponding doorbell prompt for that range.

[0199] In other embodiments, the ringing decision module can also pre-set multiple knocking force ranges, each corresponding to one or more different doorbell prompts in terms of rhythm, volume, and tone. For example, within different knocking force ranges, the weaker the knocking force, the gentler the doorbell prompt's rhythm, the lower the volume, and the softer the tone; conversely, the stronger the knocking force, the more urgent the doorbell prompt's rhythm, the higher the volume, and the harsher the tone. The ringing decision module can first determine which knocking force range the force falls within, and then determine the corresponding doorbell prompt for that range.

[0200] In the above possible implementation method 1, one or more of the rhythm, volume, and tone of the doorbell alert tone are determined by the knocking frequency and / or knocking force. Therefore, the doorbell alert tone can reflect the knocking frequency and / or knocking force.

[0201] Possible implementation method 2: The ringing decision module can determine the corresponding doorbell prompt tone to be played based solely on the identity of the user knocking in the knocking matching results (who is knocking on the door).

[0202] For example, when it is confirmed that the user knocking on the door is a pre-stored user, such as user 1, assuming user 1's name is "Xiao Hong", the doorbell prompt can be "Xiao Hong is here", or "Xiao Hong is knocking on the door", etc.

[0203] For example, if the knocking user is confirmed to be an unknown user, the doorbell message can be "Welcome, guest!" or "Guests are here!" etc.

[0204] In the second possible implementation described above, the doorbell notification sound includes the identity of the user knocking on the door. Therefore, the doorbell notification sound can reflect the identity of the user knocking on the door.

[0205] Possible implementation method 3: The ringing decision module can comprehensively determine the corresponding doorbell prompt tone to be played based on the knocking frequency and / or knocking force and the identity of the knocking user (who is knocking) in the knocking matching results.

[0206] For example, when the knocking frequency is less than a preset frequency threshold, and / or the knocking force is less than a preset force threshold, and the knocking user is confirmed to be a pre-stored user (e.g., user 1, whose name is "Xiao Hong"), the doorbell notification sound can be "Ding-dong~ Xiao Hong is here," or "Ding-dong~ Xiao Hong is knocking," etc. That is, the rhythm of the doorbell notification sound can be relatively gentle, and the content of the doorbell notification sound includes the identity of the knocking user (e.g., the knocking user's name). Optionally, the volume of the doorbell notification sound can be lower, and the tone of the doorbell notification sound can be gentler.

[0207] For example, when the knocking frequency exceeds a preset frequency threshold, and / or the knocking force exceeds a preset force threshold, and it is confirmed that the knocking user is a user pre-registered by the homeowner—for example, when the knocking user is user 1, assuming user 1's name is "Xiao Hong"—the doorbell alert could be "Xiao Hong is here, homeowner, open the door quickly," or "Xiao Hong is initiating abnormal behavior, please be careful!" etc. That is, the doorbell alert can have a relatively urgent rhythm, and the content of the doorbell alert includes the knocking user's identity (e.g., the knocking user's name). Optionally, the doorbell alert volume can be higher, and the tone of the doorbell alert can be more forceful.

[0208] For example, when the knocking frequency is less than a preset frequency threshold, and / or the knocking force is less than a preset force threshold, and the knocking user is confirmed to be an unfamiliar user, the doorbell notification tone can be "Welcome, guest!" or "Guests are here!", etc. That is, the rhythm of the doorbell notification tone can be relatively gentle, and the content of the doorbell notification tone includes the identity of the knocking user, for example, the knocking user's identity can be "guest". Optionally, the volume of the doorbell notification tone can be lower, and the tone of the doorbell notification tone can be gentler.

[0209] For example, when the knocking frequency exceeds a preset frequency threshold, and / or the knocking force exceeds a preset force threshold, and the knocking user is confirmed to be an unknown user, the doorbell alert tone can be "A stranger is here, open the door quickly," or "A stranger is engaging in unusual behavior, please be careful!" etc. That is, the doorbell alert tone can be relatively urgent, and the content of the alert tone includes the identity of the knocking user, for example, the knocking user's identity can be "a stranger." Optionally, the doorbell alert tone volume can be higher, and the tone of the alert tone can be more forceful.

[0210] In the above possible implementation method 3, one or more of the rhythm, volume, and tone of the doorbell notification sound are determined by the knocking frequency and / or knocking force, and the content of the doorbell notification sound includes the identity of the knocking user. Therefore, the doorbell notification sound can reflect both the knocking frequency and / or knocking force, as well as the identity of the knocking user.

[0211] Optionally, when it is confirmed that the user knocking on the door is an unfamiliar user, the doorbell notification sound may not indicate the identity of the user knocking on the door; that is, the content of the doorbell notification sound may not include the identity of the user knocking on the door.

[0212] Furthermore, after determining the doorbell ringing event and the corresponding doorbell prompt tone to be played, the ringing decision module can send a request to the low-power system to initiate doorbell ringing and the corresponding doorbell prompt tone to be played.

[0213] It is understood that the above examples of the corresponding doorbell prompts to be played are merely examples, and the corresponding doorbell prompts to be played can be other than those specified in this application.

[0214] S222, The low-power system sends a command to the doorbell to initiate the doorbell ringing and the corresponding doorbell prompt tone to be played.

[0215] Specifically, after receiving the request to initiate doorbell ringing and the corresponding doorbell prompt tone to be played from the ringing decision module, the low-power system can send the instruction to initiate doorbell ringing and the corresponding doorbell prompt tone to be played to trigger the doorbell to ring.

[0216] In one possible implementation, if the low-power system detects that the smart lock 100 is in an unlocked state, or that the smart lock 100 has just been locked within a certain period of time (e.g., 15 seconds), or that the doorbell has rung within a certain period of time (e.g., 15 seconds), the low-power system may not send a command to the doorbell to initiate the doorbell ringing or the corresponding doorbell prompt tone to be played, i.e., it may not trigger the doorbell to ring.

[0217] S223, The doorbell rings, playing the corresponding doorbell notification tone.

[0218] Specifically, after receiving the instruction to initiate doorbell ringing and the corresponding doorbell prompt tone sent by the low-power system, the doorbell can perform the ringing action and play the corresponding doorbell prompt tone, thereby realizing the conversion of knocking vibration into doorbell ringing.

[0219] S224-S229, The low-power system sends a command to the ringing decision module to enter low-power mode, the ringing decision module enters low-power mode, the ringing decision module sends a command to the vibration module to enter low-power mode, the vibration module enters low-power mode, the vibration module sends a command to the gyroscope to enter low-power mode, and the gyroscope enters low-power mode.

[0220] Specifically, after the doorbell is triggered, the low-power system can send a command to the ringing decision module to enter low-power mode, thereby causing the ringing decision module to enter low-power mode. Furthermore, the ringing decision module can send a command to the vibration module to enter low-power mode, thereby causing the vibration module to enter low-power mode. Furthermore, the vibration module can send a command to the gyroscope to enter low-power mode, thereby causing the gyroscope to enter low-power mode and reducing the sampling frequency of the gyroscope.

[0221] This application embodiment provides the above-described prompting method, enabling the smart door lock to detect knocking vibrations (i.e., vibrations triggered by a user's knocking action), collect vibration data generated by the knocking vibrations, identify the knocking user based on the collected vibration data, and determine whether the doorbell needs to be rung. If it is determined that the doorbell needs to be rung, the smart door lock can also play different doorbell prompt sounds based on the frequency and / or amplitude of the collected vibration data. This prevents doorbell ringing caused by vibrations other than those generated by the knocking action, increases the diversity of doorbell prompt sounds, and improves the user experience.

[0222] The following describes another prompting method provided by an embodiment of this application.

[0223] Figure 3 Another prompting method flow provided by an embodiment of this application is illustrated by way of example.

[0224] like Figure 3As shown, this prompting method can be applied to smart door lock 100, which can be installed on a door. The specific steps of this prompting method are described in detail below:

[0225] S301-S302: When the PIR sensor detects a living object approaching the smart door lock 100, the gyroscope exits low-power mode and enters detection mode to collect vibration data.

[0226] Specifically, when someone approaches the smart door lock 100 from outside the door, the smart door lock 100 can detect the moving human body approaching the smart door lock 100 through the passive infrared sensor PIR. After that, the smart door lock 100 can instruct the gyroscope to exit the low power mode and enter the detection mode to start collecting vibration data.

[0227] The specific execution process of steps S301 to S302 can be referred to Figure 2 The details of steps S201 to S205 will not be repeated here.

[0228] S303, Smart Door Lock 100 determines whether vibration data is abnormal.

[0229] Specifically, after the gyroscope exits low-power mode and enters detection mode, it begins to collect vibration data. The smart door lock 100 can determine whether the vibration data collected by the gyroscope is abnormal; that is, the smart door lock 100 can determine whether the vibration data collected by the gyroscope is the vibration data generated during the knocking vibration triggered by the user's knocking action. If so, the smart door lock 100 determines that the vibration data collected by the gyroscope is normal data; if not, the smart door lock 100 determines that the vibration data collected by the gyroscope is abnormal data.

[0230] For example, taking the vibration data collected by the gyroscope as angular velocity data, the smart door lock 100 can determine whether the amplitude of the angular velocity data collected by the gyroscope exceeds a preset threshold within a certain time window. If so, the smart door lock 100 can continue to acquire the angular velocity data collected by the gyroscope for a period of time and perform subsequent steps to perform relevant processing on the angular velocity data within that period of time.

[0231] S304, Smart Door Lock 100 has determined that the vibration data is interference data.

[0232] Specifically, after executing step S303, if the smart door lock 100 determines that the vibration data collected by the gyroscope is abnormal data, the smart door lock 100 can further determine that the abnormal data is interference data.

[0233] S305-S307 and Smart Door Lock 100 use vibration data for data noise reduction, envelope extraction, and knocking behavior recognition.

[0234] Specifically, after the smart door lock 100 determines that the vibration data collected by the gyroscope is normal data, the smart door lock 100 can perform data noise reduction, envelope extraction, and knocking behavior recognition based on the vibration data. If the knocking behavior recognition is successful, that is, the smart door lock 100 determines that a knocking event has occurred, the smart door lock 100 can continue to execute subsequent steps.

[0235] The specific execution process of steps S305-S307 can be referred to Figure 2 The details of steps S211 to S212 will not be repeated here.

[0236] S308-S310 and smart door lock 100 make ringing and image decisions. If the decision is successful, the doorbell will ring.

[0237] Specifically, after determining that a knocking event has occurred, the smart door lock 100 can make a ringing decision and an image decision, and comprehensively determine whether to initiate a doorbell ringing event. If the decision is successful, the doorbell will be notified to ring.

[0238] The specific execution process of steps S308-S310 can be referred to Figure 2 The details of steps S213 to S223 will not be repeated here.

[0239] In this embodiment, the first duration can be the duration of door vibration caused by the knocking action; the first vibration data can be vibration data 2; the second vibration data can be vibration data 1; the first part of the first duration can be a certain time window corresponding to vibration data 1 as mentioned above; the second part of the second duration can be the duration of continuous acquisition of vibration data 2; the first vibration frequency can be the frequency of the first vibration data; the first vibration amplitude can be the amplitude of the first vibration data; the first user can be user 1; the first preset knocking frequency can be a preset frequency threshold; the first preset knocking force can be a preset force threshold; the first doorbell prompt tone can be a doorbell prompt tone played when the knocking frequency is less than the first preset knocking frequency and / or the knocking force is less than the first preset knocking force; and the second doorbell prompt tone can be a doorbell prompt tone played when the knocking frequency is greater than the first preset knocking frequency and / or the knocking force is greater than the first preset knocking force.

[0240] Figure 4 An exemplary embodiment of the present application illustrates the structure of a smart door lock 100.

[0241] like Figure 4 As shown, the smart door lock 100 may include: a processor 401, a memory 402, a sensor 403, a doorbell 404, a bus 405, a camera 406, and a power supply 407. These components can be connected via the bus 405. Wherein:

[0242] Processor 401 can be used to read and execute computer-readable instructions, including one or more processing cores. Processor 401 executes various application functions and information processing by running software programs and modules. Specifically, processor 401 mainly includes a controller, an arithmetic logic unit (ALU), and registers. The controller is mainly responsible for instruction decoding and issuing control signals for the operations corresponding to the instructions. The ALU is mainly responsible for performing fixed-point or floating-point arithmetic operations, shift operations, and logical operations, and can also perform address operations and translations. Registers are mainly responsible for storing register operands and intermediate operation results temporarily stored during instruction execution. Specifically, the hardware architecture of processor 401 can be an Application Specific Integrated Circuit (ASIC) architecture, MIPS architecture, ARM architecture, or NP architecture, etc.

[0243] The memory 402 is connected to the processor 401 via a bus 405. The memory 402 can be used to store various software programs and / or multiple sets of program instructions. The processor 401 is used to execute at least one program instruction to implement the technical solution of the above embodiments. Its implementation principle and technical effects are similar to those of the related embodiments of the above methods, and will not be repeated here.

[0244] Sensor 403 is connected to processor 401 via bus 405 and can be used to collect various sensor data. Sensor 403 may include one or more sensors, such as a gyroscope sensor, pressure sensor, fingerprint sensor, touch sensor, etc. In this embodiment, sensor 403 may include a gyroscope sensor and a passive infrared sensor. The gyroscope sensor can be used to collect vibration data (e.g., angular velocity data) generated during knocking, and the passive infrared sensor can be used to detect whether a human body is approaching the smart door lock 100.

[0245] The doorbell 404 is connected to the processor 401 via the bus 405 and can be used to ring the doorbell and play the corresponding doorbell prompt tone when a knocking event occurs.

[0246] The camera 406 is connected to the processor 401 via the bus 405 and can be used to capture still images or videos. In this embodiment, the camera 406 can be used to capture human body images or face images of users knocking on the door.

[0247] Power supply 407 can be used to supply power to other internal components such as processor 401, memory 402, sensor 403, doorbell 404, and camera 406.

[0248] In the embodiments of this application, the processor may be a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0249] In the embodiments of this application, the memory can be non-volatile memory, such as a hard disk drive (HDD) or a solid-state drive (SS), or it can be volatile memory, such as random-access memory (RAM). Memory is any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, and is not limited thereto.

[0250] The memory in the embodiments of this application can also be a circuit or any other device capable of performing storage functions, used to store program instructions and / or data. The methods provided in the various embodiments of this application can be implemented entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user equipment, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access, or a data storage device such as a server or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state disks (SSDs)).

[0251] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

[0252] Understandable, Figure 4 The illustrated structure does not constitute a specific limitation on the smart lock 100. In other embodiments of this application, the smart lock 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0253] Figure 5 An exemplary embodiment of the present application provides a chip system.

[0254] like Figure 5 As shown, the chip system includes at least one processor 501 and at least one interface circuit 502. The processor 501 and the interface circuit 502 are interconnected via lines. For example, the interface circuit 502 can be used to receive signals from other devices. As another example, the interface circuit 502 can be used to send signals to other devices (e.g., the processor 501). Exemplarily, the interface circuit 502 can read instructions stored in a memory and send those instructions to the processor 501. When the instructions are executed by the processor 501, the smart lock 100 can perform the various steps performed by the smart lock 100 in the above embodiments. Of course, the chip system may also include other discrete devices, which are not specifically limited in this application embodiment.

[0255] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A prompting method, characterized in that, The method, applied to a smart door lock installed on a door, includes a doorbell, a vibration module, a ringing decision module, and an image decision module, and comprises: The smart door lock detects the vibration of the door, and the duration of the vibration is a first duration; The smart door lock generates first vibration data based on the vibration. The smart door lock obtains the knocking matching result through the vibration module. The knocking matching result includes one or both of the knocking frequency and knocking force determined based on the first vibration data, as well as the identity of the knocking user determined based on the first vibration data through template matching or machine learning. The smart door lock sends the knocking matching result to the ringing decision module through the vibration module; The smart door lock determines that a knocking event has occurred based on the knocking matching result through the ringing decision module. The smart door lock sends a request to the image decision module to obtain human figure and / or face detection results through the ringing decision module; The smart door lock determines that there is a human figure and / or face in front of the door based on the human figure and / or face detection results through the image decision module; The smart door lock sends the human figure and / or face detection results to the ringing decision module through the image decision module; The smart door lock determines, through the ringing decision module, to initiate a doorbell vibration event and the doorbell prompt tone to be played, based on the knocking matching result and the human figure and / or face detection result. The smart door lock notifies the doorbell to ring and play the doorbell alert tone. The doorbell alert tone is determined based on one or two of the knocking frequency and the knocking force, as well as the identity of the knocking user. The content of the doorbell alert tone includes the identity of the knocking user, and one or more of the volume, rhythm, and tone of the doorbell alert tone are determined by one or two of the knocking frequency and the knocking force.

2. The method according to claim 1, characterized in that, Before the smart lock generates first vibration data based on the vibration, the method further includes: The smart door lock generates second vibration data based on the vibration. The second vibration data is generated in the first part of the first duration, and the first vibration data is generated in the second part of the first duration, wherein the first part precedes the second part. The smart door lock determines that the vibration characteristics indicated by the second vibration data match the vibration characteristics of the knocking action.

3. The method according to claim 2, characterized in that, Before the smart lock generates second vibration data based on the vibration, the method further includes: The smart door lock detects that a person is approaching it.

4. The method according to any one of claims 1-3, characterized in that, The smart door lock obtains the knock matching result through the vibration module, specifically including: The smart door lock determines the knocking frequency based on a first vibration frequency using the vibration module; the higher the first vibration frequency, the higher the knocking frequency. And / or, The smart door lock determines the knocking force based on a first vibration amplitude using the vibration module; the greater the first vibration amplitude, the greater the knocking force. Wherein, the first vibration frequency is the frequency of the first vibration data, and the first vibration amplitude is the amplitude of the first vibration data.

5. The method according to any one of claims 1-3, characterized in that, The smart door lock obtains the knock matching result through the vibration module, specifically including: The smart door lock preprocesses the first vibration data through the vibration module to obtain the preprocessed first vibration data; The smart door lock uses the vibration module to match the pre-processed first vibration data with the pre-stored knocking data of the first user. If the match is successful, the knocking user is the first user.

6. The method according to claim 4, characterized in that, The smart door lock obtains the knock matching result through the vibration module, specifically including: The smart door lock preprocesses the first vibration data through the vibration module to obtain the preprocessed first vibration data; The smart door lock uses the vibration module to match the pre-processed first vibration data with the pre-stored knocking data of the first user. If the match is successful, the knocking user is the first user.

7. The method according to claim 1, 2, 3, or 6, characterized in that, The smart door lock determines the doorbell prompt tone to be played based on the knocking matching result and the human figure and / or face detection result through the ringing decision module, specifically including: If the smart door lock determines through the ringing decision module that the knocking frequency is less than the first preset knocking frequency and / or the knocking force is less than the first preset knocking force, then the smart door lock determines through the ringing decision module that the doorbell prompt tone to be played is the first doorbell prompt tone. or, If the smart door lock determines through the ringing decision module that the knocking frequency is greater than the first preset knocking frequency and / or the knocking force is greater than the first preset knocking force, then the smart door lock determines through the ringing decision module that the doorbell prompt tone to be played is the second doorbell prompt tone. The first doorbell alert tone and the second doorbell alert tone differ in one or more of the volume, rhythm, and tone, and the content of both the first doorbell alert tone and the second doorbell alert tone includes the identity of the user knocking on the door.

8. The method according to claim 4, characterized in that, The smart door lock determines the doorbell prompt tone to be played based on the knocking matching result and the human figure and / or face detection result through the ringing decision module, specifically including: If the smart door lock determines through the ringing decision module that the knocking frequency is less than the first preset knocking frequency and / or the knocking force is less than the first preset knocking force, then the smart door lock determines through the ringing decision module that the doorbell prompt tone to be played is the first doorbell prompt tone. or, If the smart door lock determines through the ringing decision module that the knocking frequency is greater than the first preset knocking frequency and / or the knocking force is greater than the first preset knocking force, then the smart door lock determines through the ringing decision module that the doorbell prompt tone to be played is the second doorbell prompt tone. The first doorbell alert tone and the second doorbell alert tone differ in one or more of the volume, rhythm, and tone, and the content of both the first doorbell alert tone and the second doorbell alert tone includes the identity of the user knocking on the door.

9. The method according to claim 5, characterized in that, The smart door lock determines the doorbell prompt tone to be played based on the knocking matching result and the human figure and / or face detection result through the ringing decision module, specifically including: If the smart door lock determines through the ringing decision module that the knocking frequency is less than the first preset knocking frequency and / or the knocking force is less than the first preset knocking force, then the smart door lock determines through the ringing decision module that the doorbell prompt tone to be played is the first doorbell prompt tone. or, If the smart door lock determines through the ringing decision module that the knocking frequency is greater than the first preset knocking frequency and / or the knocking force is greater than the first preset knocking force, then the smart door lock determines through the ringing decision module that the doorbell prompt tone to be played is the second doorbell prompt tone. The first doorbell alert tone and the second doorbell alert tone differ in one or more of the volume, rhythm, and tone, and the content of both the first doorbell alert tone and the second doorbell alert tone includes the identity of the user knocking on the door.

10. The method according to claim 1, 2, 3, 6, 8, or 9, characterized in that, Before the smart lock notifies the doorbell to ring and plays the doorbell alert tone, the method further includes: The smart door lock uses a camera to capture images of the area in front of the door; The smart door lock detects that the image in front of the door includes a human figure and / or a face.

11. The method according to claim 4, characterized in that, Before the smart lock notifies the doorbell to ring and plays the doorbell alert tone, the method further includes: The smart door lock uses a camera to capture images of the area in front of the door; The smart door lock detects that the image in front of the door includes a human figure and / or a face.

12. The method according to claim 5, characterized in that, Before the smart lock notifies the doorbell to ring and plays the doorbell alert tone, the method further includes: The smart door lock uses a camera to capture images of the area in front of the door; The smart door lock detects that the image in front of the door includes a human figure and / or a face.

13. The method according to claim 7, characterized in that, Before the smart lock notifies the doorbell to ring and plays the doorbell alert tone, the method further includes: The smart door lock uses a camera to capture images of the area in front of the door; The smart door lock detects that the image in front of the door includes a human figure and / or a face.

14. The method according to claim 1, 2, 3, 6, 8, 9, 11, 12, or 13, characterized in that, After the smart lock notifies the doorbell to ring and plays the doorbell alert tone, the method further includes: The smart door lock enters a low-power mode.

15. The method according to claim 4, characterized in that, After the smart lock notifies the doorbell to ring and plays the doorbell alert tone, the method further includes: The smart door lock enters a low-power mode.

16. The method according to claim 5, characterized in that, After the smart lock notifies the doorbell to ring and plays the doorbell alert tone, the method further includes: The smart door lock enters a low-power mode.

17. The method according to claim 7, characterized in that, After the smart lock notifies the doorbell to ring and plays the doorbell alert tone, the method further includes: The smart door lock enters a low-power mode.

18. The method according to claim 10, characterized in that, After the smart lock notifies the doorbell to ring and plays the doorbell alert tone, the method further includes: The smart door lock enters a low-power mode.

19. An electronic device, characterized in that, The electronic device includes one or more processors and one or more memories; wherein the one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program code, the computer program code including computer instructions, which, when executed by the one or more processors, cause the electronic device to perform the method as described in any one of claims 1-18.

20. A computer storage medium, characterized in that, The computer storage medium stores a computer program, the computer program including program instructions that, when executed on an electronic device, cause the electronic device to perform the method as described in any one of claims 1-18.

Images