Image acquisition device of visual doorbell, information processing method and visual doorbell system

Abstract

The embodiment of the present disclosure relates to an image acquisition device of a visual doorbell, an information processing method and a visual doorbell system, the image acquisition device of the visual doorbell comprises: an imaging lens; an image acquisition sensor arranged at an image side of the imaging lens; and a field of view magnifier arranged at an object side of the imaging lens and used for expanding a field of view area of the imaging lens below a horizontal field of view surface. The technical scheme provided by the embodiment of the present disclosure is that the field of view magnifier is arranged at the object side of the imaging lens, the image acquisition sensor is arranged at the image side of the imaging lens, and the field of view magnifier is arranged to expand the field of view area of the imaging lens below the horizontal field of view surface, so that the field of view range of the imaging lens below the horizontal field of view surface can be increased, the vertical field of view can be widened, and thus the light rays in a larger angle range below the horizontal field of view surface of the imaging lens of the visual doorbell can be acquired, and the problem of the blind area in front and below the visual doorbell is improved.

Description

Technical Field

[0001] This disclosure relates to the field of smart home technology, and in particular to an image acquisition device, information processing method, and video doorbell system for a video doorbell, as well as electronic devices and computer storage media. Background Technology

[0002] A doorbell is a bell used to alert someone to the presence of someone outside a door, such as ringing to let the homeowner know that a guest has arrived. Currently, doorbell products have evolved from the initial simple button-operated bell that rings from the outside to the inside, letting the homeowner know there is a visitor; to incorporating camera visualization features, allowing a direct view of the doorway from inside the house; and further to integrating various AI technologies to directly identify the visitor. They have evolved from simple ringing tools into increasingly intelligent forms.

[0003] However, existing video doorbells have limited field of view. The field of view of a non-fisheye lens is typically limited to the area above an adult's waist, while a fisheye lens, although offering a slightly larger field of view, still struggles to cover the vertically downward field of view of the video doorbell. Thus, the vertical field of view of a video doorbell is relatively small. Summary of the Invention

[0004] To solve the above-mentioned technical problems, or at least partially solve them, embodiments of this disclosure provide an image acquisition device, an information processing method, and a video doorbell system, as well as an electronic device and a computer storage medium.

[0005] This disclosure provides an image acquisition device for a video doorbell, comprising:

[0006] Imaging lens;

[0007] An image acquisition sensor is disposed on the image side of the imaging lens;

[0008] A field magnifying glass is disposed on the object side of the imaging lens to expand the field of view area of ​​the imaging lens below the horizontal field of view.

[0009] This disclosure also provides an information processing method for a video doorbell, including:

[0010] The video image is acquired based on any of the above-mentioned video doorbell image acquisition devices, and the video image includes the video image after the field of view is enlarged by a field magnifying lens;

[0011] Video analysis is performed based on the video images to determine whether a video notification event has occurred.

[0012] When a video notification event is detected, trigger the event notification action.

[0013] This disclosure also provides a method for retrieving items based on a video doorbell, including:

[0014] Acquire video images captured by an image acquisition device based on a video doorbell, the video images including item codes;

[0015] The video image is analyzed according to the encoding recognition algorithm to obtain the express delivery information of the express item corresponding to the item code;

[0016] Generate a delivery notification message based on the express delivery information;

[0017] Output the received express delivery notification message.

[0018] This disclosure also provides an information processing device for a video doorbell, including:

[0019] A video image acquisition module is used to acquire video images captured by an image acquisition device; the image acquisition device is any of the above-mentioned video doorbell image acquisition devices, and the video image includes a video image after the field of view is enlarged by a field magnifying lens.

[0020] The video notification event determination module is used to perform video analysis based on the video image to determine whether a video notification event has occurred.

[0021] The event notification operation triggering module is used to trigger an event notification operation when a video notification event is determined to have occurred.

[0022] This disclosure also provides an item collection device based on a video doorbell, comprising:

[0023] The video image acquisition module is used to acquire video images captured by the image acquisition device based on the video doorbell, wherein the video images include item codes;

[0024] The video image analysis module is used to analyze the video image according to the encoding recognition algorithm to obtain the express delivery information of the express item corresponding to the item code;

[0025] The express delivery notification message generation module is used to generate an express delivery notification message based on the express delivery information.

[0026] The module for receiving express delivery notification messages is used to output the express delivery notification messages.

[0027] This disclosure also provides a video doorbell system, including: an image acquisition device for any of the above-mentioned video doorbells, and an information processing device;

[0028] The image acquisition device is used to acquire video images, including video images after the field of view is enlarged by a magnifying glass.

[0029] The information processing device is used to perform video analysis based on the video image to determine whether a video notification event has occurred; and when it is determined that a video notification event has occurred, it triggers an event notification operation.

[0030] This disclosure also provides an item retrieval system based on a video doorbell, including:

[0031] The video doorbell includes an image acquisition device and an information processing device, wherein the information processing device includes:

[0032] The video image acquisition module is used to acquire video images captured by the image acquisition device of the video doorbell, wherein the video images include item codes;

[0033] The video image analysis module is used to analyze the video image according to the encoding recognition algorithm to obtain the express delivery information of the express item corresponding to the item code;

[0034] The express delivery notification message generation module is used to generate an express delivery notification message based on the express delivery information.

[0035] The module for receiving express delivery notification messages is used to output the express delivery notification messages.

[0036] This disclosure also provides an electronic device, the electronic device comprising:

[0037] processor;

[0038] Memory used to store the processor's executable instructions;

[0039] The processor is configured to read the executable instructions from the memory and execute the instructions to implement the above-described information processing method for video doorbells, or the above-described item retrieval method based on video doorbells.

[0040] This disclosure also provides a computer storage medium storing a computer program that, when executed by a processor, implements any of the above-mentioned information processing methods for video doorbells, or any of the above-mentioned item retrieval methods based on video doorbells.

[0041] Compared with the prior art, the technical solution provided in this disclosure has at least the following advantages: In this disclosure, the image acquisition device of the video doorbell includes: an imaging lens, an image acquisition sensor, and a field magnifying glass. The image acquisition sensor is disposed on the image side of the imaging lens; the field magnifying glass is disposed on the object side of the imaging lens, and the field magnifying glass is used to expand the field of view area of ​​the imaging lens below the horizontal field of view. This can increase the field of view range of the imaging lens below the horizontal field of view, which is beneficial to widening the vertical field of view. This is beneficial to acquiring light within a larger field of view range below the horizontal plane where the video doorbell is located, and improving the blind spot problem in front of and below the video doorbell.

[0042] In this embodiment, the signal processing method for the video doorbell can be implemented based on the image acquisition device of the video doorbell. Specifically, it includes video analysis based on the video image acquired by the image acquisition device to determine whether a video notification event has occurred; and triggering an event notification operation when a video notification event is determined to have occurred. In this method, the video image for video analysis is a video image after the field of view is expanded by a field-of-view magnifying glass. This widens the vertical field of view, which is beneficial for capturing a larger field of view below the horizontal plane of the video doorbell. This can be applied to identify objects or people that are low in position or at a low height, thereby enabling the identification of children or the tracking of target objects, thus improving the intelligence level of the video doorbell. Attached Figure Description

[0043] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0044] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 This is a schematic diagram of the structure of an image acquisition device for a video doorbell provided in an embodiment of this disclosure;

[0046] Figure 2 This is a schematic diagram of the structure of another video doorbell image acquisition device provided in this embodiment of the present disclosure;

[0047] Figure 3 This is a schematic diagram of the structure of another video doorbell image acquisition device provided in this embodiment of the present disclosure;

[0048] Figure 4 This is a schematic diagram of the structure of another video doorbell image acquisition device provided in this embodiment of the present disclosure;

[0049] Figure 5 This disclosure provides an information processing method for a video doorbell.

[0050] Figure 6 This is another information processing method for video doorbells provided in this embodiment;

[0051] Figure 7 This is a flowchart illustrating a method for applying the information processing method of a video doorbell provided in this embodiment to a notification scenario when a target person arrives.

[0052] Figure 8 This is a schematic diagram illustrating a signal interaction method when the information processing method for a video doorbell provided in this embodiment is applied to a notification scene of the arrival of a target person;

[0053] Figure 9 This is a flowchart illustrating a method for applying the information processing method for a video doorbell provided in this embodiment to a stranger alarm scenario.

[0054] Figure 10 This is a schematic diagram of a signal interaction method when the information processing method for a video doorbell provided in this embodiment is applied to a stranger alarm scenario;

[0055] Figure 11 This is a flowchart illustrating a method for applying the information processing method of a video doorbell provided in this embodiment to an item care notification scenario.

[0056] Figure 12 This is a schematic diagram of a signal interaction method when the information processing method of the video doorbell provided in this embodiment is applied to an item care notification scenario;

[0057] Figure 13 This is a schematic diagram illustrating another signal interaction method when the information processing method for a video doorbell provided in this embodiment is applied to an item care notification scenario;

[0058] Figure 14 This is a schematic flowchart of a method for retrieving items based on a video doorbell, provided in an embodiment of this disclosure.

[0059] Figure 15 This is a schematic diagram of the structure of an information processing device for a video doorbell provided in an embodiment of this disclosure;

[0060] Figure 16 This is a schematic diagram of the structure of a video doorbell system provided in an embodiment of this disclosure;

[0061] Figure 17 This is a schematic diagram of the structure of a video doorbell-based item collection device provided in an embodiment of this disclosure;

[0062] Figure 18 This is a schematic diagram of the structure of a video doorbell-based item retrieval system provided in an embodiment of this disclosure;

[0063] Figure 19 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0064] To better understand the above-mentioned objectives, features, and advantages of this disclosure, the solutions disclosed herein will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0065] Numerous specific details are set forth in the following description in order to provide a full understanding of this disclosure, but this disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some, and not all, of the embodiments of this disclosure.

[0066] With the evolution of doorbell products, doorbells have progressed from simple ringing tools to increasingly intelligent forms. Video doorbells, or smart doorbells, require both hardware structure and intelligent services to function. The hardware structure primarily refers to the optical acquisition system, or simply the optical system, or image acquisition device, whose main component is the lens. Existing video doorbells are mainly categorized by lens selection into fisheye lens video doorbells and non-fisheye wide-angle lens video doorbells.

[0067] Non-fisheye lenses are widely used not only in video doorbells but also in mobile phones, cameras, and other devices with image capture capabilities. Typically, wide-angle lenses capture full-frame rectangular images with an aspect ratio of 16:9 or 4:3. Currently, the diagonal field of view of video doorbells is mainly in the range of 105° to 162°; their maximum horizontal field of view is approximately 120°, and their maximum vertical field of view is approximately 60°; a very small number of wide-angle lenses with square sensors can achieve a vertical field of view of 100°. However, even with a 100° vertical field of view, a non-fisheye lens, considering the lowest installation height of a video doorbell (1.5m), still cannot capture the face of a child standing at the door due to their short stature.

[0068] Fisheye lenses are a special type of ultra-wide-angle lens. Their front lens element has a very short diameter and protrudes parabolically towards the front of the lens, resembling a fish's eye. Their field of view (i.e., angle of view) aims to reach or exceed the range of human vision, offering a wider field of view. However, this also introduces greater distortion across the entire image, resulting in a significant difference from the real world as perceived by humans. While fisheye lens video doorbells can achieve a large field of view, the distortion within that field of view prevents the reproduction of realistic images in core application scenarios such as video intercoms. Furthermore, they still cannot cover the vertically downward visual range of the video doorbell.

[0069] Based on the above, video doorbells based on fisheye lenses suffer from overall image distortion and still struggle to cover the visual range directly below the doorbell. Video doorbells based on non-fisheye wide-angle lenses have limited field of view, too narrow a vertical field of view, and cannot simultaneously capture the head of an adult male standing at the door and the body of a child standing at the door.

[0070] Meanwhile, regardless of whether a video doorbell uses a fisheye lens or a regular wide-angle lens, its current intelligent services, based on facial and human detection visual algorithms, are mainly limited to basic judgments such as stranger loitering and facial recognition. However, it cannot provide specific transactional services such as watching over parcels or other targeted items, identifying returning children or other individuals, or alerting strangers. In other words, the existing intelligent services of video doorbells are primarily focused on monitoring visitors and anomalies, while lacking functionality for handling everyday tasks at the doorstep.

[0071] Based on the above, current video doorbells are still far from being "home security assistants," mainly in the following aspects: (1) In terms of hardware specifications, the field of view is limited. Non-fisheye lenses mostly cover the field of view above the waist of an adult, and even fisheye lenses cannot cover the visual range (which can be understood as the field of view) directly below the doorbell. In addition, the entire visual image has serious distortion. (2) In terms of intelligent services, based on algorithms such as face recognition and human detection, it can support basic judgment of the presence of visitors / personnel, but it cannot provide specific service such as watching over express parcels, notifying children when they arrive at home, and alarming strangers.

[0072] To address the aforementioned issues, the technical solutions provided in this disclosure improve upon existing video doorbells in terms of hardware structure and intelligent services, enabling video doorbells to not only serve as monitoring tools for visitors and abnormal situations, but also as a door-to-door service assistant. The following description provides examples of specific structures and scenarios.

[0073] In terms of hardware structure, the image acquisition device for the video doorbell provided in this embodiment (hereinafter also referred to as "image acquisition device") has a field magnifying glass set on the object side of the imaging lens, so that the field of view below the imaging lens can be imaged in the image acquisition sensor, thereby expanding the vertical field of view of the image acquisition device and improving the problem of blind spot in front of the video doorbell.

[0074] The following is combined Figures 1-4 The image acquisition device for a video doorbell provided in the embodiments of this disclosure will be described by way of example.

[0075] In some embodiments, Figure 1 This is a schematic diagram of the structure of an image acquisition device for a video doorbell according to an embodiment of this disclosure, illustrating the structure of the image acquisition device that can increase the vertical field of view. (Refer to...) Figure 1 The video doorbell's image acquisition device 10 includes an imaging lens 101, an image acquisition sensor 102, and a field-of-view magnifying lens 103. The image acquisition sensor 102 is positioned on the image side of the imaging lens 101; the field-of-view magnifying lens 103 is positioned on the object side of the imaging lens 101, and is used to magnify the horizontal field of view 1010 of the imaging lens 101 (exemplarily, the horizontal field of view 1010 is in...). Figure 1 The field of view below (shown by the dotted line).

[0076] The imaging lens 101 is used to converge light, allowing light from as large a field of view as possible to enter the image acquisition sensor 102. The image acquisition sensor 102 receives the light converged by the imaging lens 101 and performs photoelectric conversion and data processing to obtain the image within the field of view. The field-of-view magnifying lens 103 can expand the field of view area of ​​the imaging lens 101 below the horizontal field of view 1010, that is, adjust the light transmission direction of the imaging lens 101 below the horizontal field of view 1010, so that it enters the imaging lens 101. Thus, the field of view area below the horizontal field of view 1010 of the imaging lens 101 can be imaged in the image acquisition sensor 102, thereby capturing the image directly below the video doorbell and improving the blind spot problem in front of and below the video doorbell.

[0077] For example, the field magnifying lens 103 can be a single lens or a group of lenses, which can change the transmission direction of light. The specific structure of this embodiment is not limited. The following description, in conjunction with... Figure 3 and Figure 4 An example is provided.

[0078] For example, the imaging lens 101 may be a general wide-angle lens that is not a fisheye lens, so as to avoid distortion of the image content of the whole picture; or the imaging lens 101 may be other types of lenses known to those skilled in the art, and this disclosure does not limit it.

[0079] It should be noted that, Figure 1 The imaging lens 101 is shown only as a single convex lens in this embodiment, but this does not constitute a limitation on the image acquisition device 10 for the video doorbell provided in this disclosure. In other embodiments, the imaging lens 101 may be a single optical lens of other structures, or an optical lens group composed of two or more optical lenses, which is not limited in this disclosure. In the following, Figures 2-4 Similarly, this will not be repeated below.

[0080] For example, the image acquisition sensor 102 can be a complementary metal-oxide-semiconductor sensor (CMOS sensor), which can achieve higher light sensitivity, shorter exposure time, higher integration and larger dynamic range of brightness and darkness. This is beneficial to improve the imaging quality of the field of view image and to reduce the size of the hardware structure of the video doorbell, thus facilitating the miniaturization design of the video doorbell.

[0081] In other embodiments, the image acquisition sensor 102 may also employ other types of optoelectronic devices known to those skilled in the art, and this disclosure does not limit this type of device.

[0082] In some embodiments, continue to refer to Figure 1 Furthermore, the field-of-view magnifier 103 also allows light to pass through. That is, light rays above the horizontal field of view 1010 of the imaging lens 101 can pass through the field-of-view magnifier 103 without changing their propagation direction, and then enter the imaging lens 101. The light rays that ultimately enter the image acquisition sensor 102 include: the original light rays above the horizontal field of view 1010 of the imaging lens 101, and the light rays within the magnified field of view below the horizontal field of view 1010 of the imaging lens 101. Therefore, with the horizontal field of view 1010 of the imaging lens 101 as the boundary, while expanding the field of view of its lower half, the image content of its upper half remains unaffected. The field of view of the upper half does not increase simultaneously due to the increase in the lower half's field of view, thus avoiding unnecessary increases in video computation.

[0083] The above Figure 1The image shows that when the field magnifying lens 103 is set in the full field of view, its upper and lower halves are set differently, so as to expand the field of view below the video doorbell without affecting the imaging above the horizontal field of view 1010.

[0084] In other embodiments, the field magnifying lens 103 can be placed only in a local area within the field of view, so that the field magnifying lens 103 does not affect the light above the horizontal field of view 1010. The following is in conjunction with... Figure 2 An example is provided.

[0085] In some embodiments, Figure 2 This is a schematic diagram of the structure of another video doorbell image acquisition device provided in this embodiment, illustrating another relative positional relationship between the field-of-view magnifying lens and the imaging lens in the horizontal field of view. (Refer to...) Figure 2 The field magnifying glass 103 is positioned below the horizontal field of view 1010 of the imaging lens 101.

[0086] The field-of-view magnifying lens 103 occupies only the space below the horizontal field of view 1010 of the imaging lens 101, and not the space above the horizontal field of view 1010. Therefore, light from above the horizontal field of view 1010 can directly enter the imaging lens 101 without being affected by the field-of-view magnifying lens 103. This means the upper part of the image content remains unaffected, and the image of the ceiling above the video doorbell is not captured simply because the field of view below the video doorbell is increased, thus avoiding unnecessary video computation. Because the image acquisition device 10 uses a non-fisheye lens, image distortion across the entire screen is avoided, which is beneficial for restoring the true image and picture.

[0087] The field-of-view magnifying lens 103 expands the field of view below the horizontal field of view 1010 of the imaging lens 101. It allows light from both the original lower half of the field of view and the expanded field of view to pass through simultaneously and enter the imaging lens 101. The expanded field of view covers the vertically lower part of the video doorbell, and the entire visible image is free from severe distortion. While increasing the field of view of the video doorbell, the image reproduction is also high.

[0088] Meanwhile, the field magnifying lens 103 is small in size and occupies a small space. The field magnifying lens 103 does not need to allow light to pass through the upper part, so it does not need to be differentiated. That is, the structure of the field magnifying lens 103 itself can be relatively simple, which helps to reduce the difficulty of design and manufacturing.

[0089] It should be noted that, Figure 1 and Figure 2The example shown is merely illustrative, illustrating that the field-of-view magnifying lens 103 can be used to adjust the transmission direction of the lower half of the optical fiber, thereby expanding the field of view area below the horizontal field of view 1010 of the imaging lens 101; its adjustment methods may include reflecting, refracting, converging, and diverging light, etc., which are not limited in this disclosure embodiment, and are discussed below in conjunction with Figure 3 and Figure 4 An example is provided.

[0090] In some embodiments, Figure 3 This is a schematic diagram of the structure of another video doorbell image acquisition device provided in this disclosure, showing a light transmission path in which light from below the horizontal field of view enters the imaging lens via a field magnifying lens. (Refer to...) Figure 3 The field magnifying glass 103 includes a reflecting surface 1030, which is set at an acute angle to the horizontal field of view 1010.

[0091] The reflecting surface 1030 faces the imaging lens 101. Light from the expanded field of view, beyond the original field of view in the lower half, is reflected by the reflecting surface 1030 of the field magnifying lens 103 before entering the imaging lens 101, thus enabling the acquisition of light from the area below the expanded horizontal field of view 1010. Simultaneously, the field magnifying lens 103 can also adjust the light from the original field of view in the lower half to the imaging lens 101.

[0092] For example, the field magnifying lens 103 may simultaneously have a reflecting surface 1030 and a converging lens (or lens group) to transmit, reflect and converge light in the enlarged field of view area below the horizontal field of view 1010 into the imaging lens 101.

[0093] This design simplifies the principle of adjusting the transmission direction of light in the field of view, which in turn simplifies the structure of the field magnifier 103 and reduces its design and manufacturing difficulty.

[0094] It should be noted that, Figure 3 The image only shows, by way of example, that the reflective surface 1030 is located inside the field-of-view magnifying lens 103, but this does not constitute a limitation on the field-of-view magnifying lens 103 in the image acquisition device 10 provided in this embodiment. In other embodiments, the reflective surface 1030 may also be located at the boundary surface of the field-of-view magnifying lens 103, as described below in conjunction with... Figure 4 An example is provided.

[0095] In some embodiments, Figure 4 This schematic diagram illustrates one implementation of a field-of-view magnifying lens for an image acquisition device for a video doorbell, as provided in another embodiment of this disclosure. (Refer to...) Figure 4The field magnifying lens 103 is a field magnifying prism. The field magnifying prism also includes a light incident surface 1031 and a light exit surface 1032. After the field light is incident on the light incident surface 1031, it is reflected by the reflecting surface 1030 and then exits through the light exit surface 1032 to the imaging lens 101.

[0096] Among them, the light rays (i.e. field rays) in the field of view below the horizontal field of view 1010 enter the field magnifying lens 103 through the incident surface 1031, and after the light transmission direction is adjusted in the field magnifying lens 103, they are emitted to the imaging lens 101 through the exit surface 1032.

[0097] For example, the field-magnifying prism can be a triangular prism (see...). Figure 4 The embodiments disclosed herein do not limit the use of prisms, square prisms or other multi-prisms, or irregularly shaped prisms.

[0098] With this configuration, the boundary surface of the field magnifying prism can be used as the light incident surface 1031, the reflecting surface 1030, and the light exit surface 1032. This helps to avoid designing a functional interface for adjusting the light transmission direction inside the field magnifying prism, thereby making the structure of the field magnifying lens 103 simpler and the overall implementation of the image acquisition device 10 easier.

[0099] In other embodiments, the field magnifying lens 103 may also include other types of optical lenses or lens groups known to those skilled in the art, which are not limited in this disclosure.

[0100] In some embodiments, the incident surface of the light is a plane or an arc surface.

[0101] When the light incident surface is a plane, it invariably alters the transmission direction of the field rays incident on it. When the light incident surface is an arc-shaped surface, it can converge or diverge the field rays incident on it into the field magnifying lens. The specific configuration can be set based on the adjustment requirements of the field ray transmission direction; this disclosure is not limited in its embodiments.

[0102] In some embodiments, the light emitting surface may be a planar or arc-shaped surface, and the light reflecting surface may be a planar or arc-shaped surface, which can be set based on the adjustment requirements of the transmission direction of the light in the field of view. This disclosure does not limit this aspect.

[0103] In the above embodiments, the image acquisition device of the video doorbell is fixed to the door or wall.

[0104] The video doorbell's image acquisition device is used to acquire video images outside the door. It can be directly fixed to the door or installed on a wall near the door. This embodiment of the present disclosure does not limit this.

[0105] For example, refer to Figures 1-4 In any image, along the direction from inside the door to outside the door, the image acquisition sensor 102, imaging lens 101 and field magnifying glass 103 in the image acquisition device 10 are arranged in sequence. This enables the acquisition of field light within the field of view outside the door, which facilitates the subsequent identification of objects 011 (e.g., target object 011) and / or people 012 (including target people and strangers) outside the door based on video images.

[0106] In other embodiments, when the image acquisition device 10 also includes other optical elements, the optical elements (including the image acquisition sensor 102, the imaging lens 101, and the field magnifying glass 103) can be arranged according to the adjustment effect of each optical element on the transmission direction of the field light. This disclosure does not limit this.

[0107] The image acquisition device 10 for a video doorbell provided in this embodiment adds a field-of-view magnifying glass 103 to the object side of the imaging lens 101, so that the field of view below the horizontal field of view 1010 of the imaging lens 101 can be imaged in the image acquisition sensor 102, thereby facilitating the capture of the image content directly below the video doorbell and improving the blind spot problem in front of the video doorbell. At the same time, by placing the field-of-view magnifying glass 103 below the horizontal field of view 1010, the image content in the upper part is not affected. The ceiling above the video doorbell will not be captured due to the increased field of view in front of the video doorbell. This avoids increasing unnecessary video computation and avoids the problem of image content distortion of the whole screen caused by the use of a fisheye lens, resulting in higher image fidelity of the video image.

[0108] Based on the image acquisition devices of various video doorbells provided in the above embodiments, since this image acquisition device can acquire images of a larger field of view in the lower half in addition to acquiring the original image of the upper half, it can realize the image acquisition of objects or people that are low in position or height, which is beneficial for realizing item monitoring and notification of children arriving home, and realizing stranger alarm prompts; or it can be applied to office building management or shop management application scenarios, so as to realize the acquisition of a larger field of view based on the image acquisition device, which is beneficial for realizing face analysis or item monitoring within a larger field of view, thereby realizing the monitoring of the frequency of user or customer visits, or realizing the monitoring of items in the shop. The following describes the intelligent service scenarios provided in this technical solution by way of example.

[0109] The following is combined Figures 5-13 The information processing method of the video doorbell provided in the embodiments of this disclosure will be described by way of example, and the applicable smart service scenarios will be described, such as child home notification, stranger alarm (i.e. child care) or item care.

[0110] In some embodiments, Figure 5 This disclosure provides an information processing method for a video doorbell, illustrating a process for signal processing based on an image acquisition device to achieve event notification operations. (See also...) Figure 5 The information processing method for the video doorbell may include:

[0111] S201. A video image acquisition device based on a video doorbell acquires video images.

[0112] In this step, the image acquisition device of the video doorbell can be any of the image acquisition devices described in the above embodiments. When any of the above image acquisition devices is used to acquire video images, the video images may include video images after the field of view area has been expanded by a field magnifying glass.

[0113] Therefore, the video image can cover a large field of view below the horizontal field of view in a video doorbell, which is beneficial for recognizing objects or people that are low in position or height. In this step, the signal processing device can call upon the video image acquired by the image acquisition device to provide basic data for subsequent video analysis.

[0114] In other embodiments, provided that the field of view requirement is met, the image acquisition device of the video doorbell in this step may also be other types of image acquisition devices known to those skilled in the art, such as an image acquisition device based on a fisheye lens, or an image acquisition device based on a non-fisheye lens, or an image acquisition device based on other types of imaging lenses. This disclosure does not limit this.

[0115] S202. Perform video analysis based on video images to determine whether a video notification event has occurred.

[0116] Among them, video notification events can be understood as events that need to be notified to users based on the analysis of video images. For example, these may include events such as a child arriving home, a guest visiting, a stranger loitering at the door or nearby, the arrival of an item, and the removal of an item. The following text uses the arrival notification event of a target person, the stranger alarm event, and the item care notification event as examples for illustration.

[0117] In this step, video analysis is performed based on the video images obtained in S201. For example, by detecting and recognizing faces of people or objects, and tracking targets of people and objects, it can be determined whether a video notification event has occurred.

[0118] S203. When it is determined that a video notification event has occurred, trigger the event notification operation.

[0119] The trigger event notification operation can be understood as executing a trigger action to send a notification that a video notification event has occurred.

[0120] In this step, based on the video analysis of the video image in S202, when it is determined that a video notification event has occurred, an event notification operation is triggered to issue a notification operation that a video notification event has occurred, thereby realizing the prompt that a video notification event has occurred.

[0121] For example, the event notification operation can be issued locally by the video doorbell or by a terminal device associated with the information processing device; the terminal device can be other terminal devices in the smart service system, and can be a movable or fixed terminal device set inside the door; or it can be a mobile terminal device carried by the user, which is not limited in this embodiment.

[0122] The signal processing method for video doorbells provided in this disclosure can acquire video images captured by the image acquisition device of the video doorbell, perform video analysis on the video images, and trigger a video notification operation when a video notification event occurs. Therefore, based on video analysis of the video images, notification operations can be implemented for events such as children arriving home, guests visiting, stranger alarms, the arrival of items, and the removal of items, thereby improving the intelligence level of the video doorbell and facilitating its use as a "home security assistant."

[0123] In the above embodiments, before acquiring the video image captured by the image acquisition device, the method may further include: triggering the image acquisition device of the video doorbell based on a preset time interval or a preset trigger event.

[0124] In some embodiments, Figure 5 Based on this, prior to S201, the method also includes:

[0125] When a human body is detected by the human body detection sensor, the image acquisition device of the video doorbell is activated.

[0126] The human detection sensor is used to detect whether a human body is present within its working range or the field of view of the image acquisition device. When a human body is detected, the image acquisition device is activated to acquire video images. The working range of the human detection sensor can cover the field of view of the image acquisition device, or the two can be the same or approximately the same, ensuring that the image acquisition device is activated when a person enters the field of view of the image acquisition device. This embodiment of the present disclosure is not limited in this respect.

[0127] For example, the human body detection sensor can be a passive infrared (PIR) human body sensor, also known as a PIR detector or PIR sensor. When someone approaches the video doorbell, the PIR human body sensor can detect the presence of a human body, thereby triggering the video acquisition of the video doorbell, that is, activating the video acquisition device of the video doorbell.

[0128] It should be noted that in this step, the image acquisition device is activated based on the human body detection sensor. This can be a direct communication between the two or coordinated control by the control center of the intelligent service system. This embodiment of the disclosure is not limited to this.

[0129] When a PIR sensor is used for human body detection, its power consumption is very low and it is easy to install because the PIR sensor does not actively emit infrared light; and because it is a passive sensor, it will not cause system interference problems, resulting in high accuracy in human body detection.

[0130] In other embodiments, the human body detection sensor may also be an active infrared sensor, or a sensor that can detect human bodies based on other principles, which is not limited in this disclosure.

[0131] In some embodiments, Figure 5 Based on this, prior to S201, the method also includes:

[0132] The video doorbell's image acquisition device is activated periodically based on a pre-set time interval.

[0133] The preset time interval is the time interval between two consecutive activations of the image acquisition device. For example, this time interval can be 1 second, 2 seconds, 3.5 seconds, or other time intervals, and can be set based on the monitoring needs of the video doorbell for the scene outside the door.

[0134] In this step, by setting the image acquisition device to start periodically based on a preset time interval, video images can be acquired periodically. This ensures that the video images have the required continuity while the image acquisition device is not always in operation, thus helping to reduce the energy consumption of the image acquisition device.

[0135] In some embodiments, the image acquisition device may be activated periodically at preset time intervals when the human body detection sensor does not detect a human body; and the image acquisition device may be activated immediately when the human body detection sensor detects a human body.

[0136] In other embodiments, the start-up method and timing of the image acquisition device can be determined based on other conditions or by combining multiple conditions of different dimensions. This disclosure does not limit this aspect.

[0137] In some embodiments, video notification events include a target person arrival notification event, a stranger alert event, or an item care notification event.

[0138] The target person can include family or friends, such as the user themselves, their children, elderly family members, visiting friends, or other people the user knows. When the target person appears in the video image, it is determined that a target person arrival event has occurred, including a child arriving at home, other family members arriving at home, or a guest visiting.

[0139] In this context, a stranger is someone the user does not know, as opposed to the target person. When a person who cannot be matched with the target person appears in the video image, it is determined that a stranger has appeared in the video image, thus triggering a stranger alarm event.

[0140] When the video image includes the target item, it is determined that an item care notification event has occurred, which may include notifying the user that the target item has arrived or that the target item has been moved.

[0141] In other embodiments, when the signal processing method of the video doorbell is applied to other scenarios, the video notification event may also include other video notification events for people, objects or events, which is not limited in this disclosure.

[0142] Based on the above implementation methods, the following text will combine... Figures 6-13 The signal processing flow of this signal processing method is illustrated by example in the scenarios of children arriving at home, child care, and object care.

[0143] In some embodiments, Figure 5 Based on this, for scenarios involving children coming to the home or childcare, S202 may include:

[0144] Step 1: Perform face recognition on the video image based on the face recognition algorithm and the pre-set facial image of the target person.

[0145] Among them, the face recognition algorithm is used to identify face images in video images. It may include calculating and analyzing the position of facial features, face shape and angle of the face image of a person in the video image, and then comparing it with the face image of a preset target person to determine the identity of the person in the video image.

[0146] For example, the face recognition algorithm may be a recognition algorithm based on facial feature points, a recognition algorithm based on the whole face image, a recognition algorithm based on templates, an algorithm that uses a neural network for recognition, an algorithm that uses a support vector machine for recognition, or other face recognition algorithms known to those skilled in the art. This disclosure does not limit the specific face recognition algorithm.

[0147] For example, the preset target person's facial image may include the user's own facial image, the user's child's facial image, or the facial image of the user's other relatives or friends, and this disclosure embodiment is not limited to this.

[0148] Step 2: When a face image that matches the face image of the preset target person is matched, determine that a target person arrival notification event has occurred; or, when a face image that does not match the face image of the preset target person is matched, determine that a stranger alarm event has occurred.

[0149] Specifically, when the target person arrives, they appear in the video image, triggering a target person arrival notification event. Based on this, facial recognition is performed on the video image, and a facial image matching the preset target person's face can be found, at which point it can be determined that a target person arrival notification event has occurred.

[0150] For example, when a face image matching a child's face image is found in a video image, it can be determined that the child has arrived home, thus enabling a child arrival notification.

[0151] Specifically, a stranger alarm event is triggered when a stranger appears in the video image. Based on this, facial recognition of the video image will match a facial image that does not match the preset target person's facial image, at which point a stranger alarm event can be determined.

[0152] For example, when a face image is matched in a video image that does not match the face image of a child, the user, the user's relatives or friends, it is determined that a stranger has appeared in the video image, thus enabling a child care reminder.

[0153] In this embodiment of the invention, based on the acquired video images, facial recognition is performed on the video images using a facial recognition algorithm and a preset target person's facial image. When a facial image matching the preset target person's facial image is found, a notification of the target person's arrival can be triggered. When a facial image not matching the preset target person's facial image is found, a stranger alarm can be triggered, i.e., a child supervision reminder can be implemented. This enriches the functionality of the video doorbell and improves its level of intelligence.

[0154] In some embodiments, when the method is applied to a target person arriving in the scene, Figure 5 Based on this, S203 may include:

[0155] When a video notification event includes a notification of the arrival of a target person, a notification message indicating the arrival of the target person is output through a third terminal device, which may include at least one of a smartphone, a smart speaker, or a video doorbell.

[0156] When the arrival of a target person is determined based on video images, a notification message can be sent via at least one of a smartphone, smart speaker, or video doorbell.

[0157] That is, when the arrival of a target person is determined based on video images, a notification message can be sent through any one, two or three of the following devices: smartphone, smart speaker or video doorbell.

[0158] For example, taking a child arriving home as an example, when the arrival of a child is confirmed, a notification message can be received and output in the mobile app, such as "Baby is home"; the indoor speaker (and smart speaker) or the indoor unit of the video doorbell can be triggered to issue a voice notification. For example, the indoor speaker or the indoor unit of the video doorbell can receive the event notification and broadcast the audio message: "Baby is home, remember to wash your hands when you come in." It should be noted that the above notification message can be output on any third-party terminal device, or on at least two of the above third-party terminal devices. For example, the notification message "Baby is home" can be output in the mobile app, and the notification message "Baby is home, remember to wash your hands when you come in" can also be broadcast on the indoor speaker.

[0159] In some embodiments, Figure 6 This disclosure provides another information processing method for a video doorbell, illustrating a process for determining whether a person in a video image is a preset target based on a face recognition algorithm. (Refer to...) Figure 6 The method may include:

[0160] S301, A video image acquisition device based on a video doorbell acquires video images.

[0161] S302. Based on the face recognition algorithm and the face image of the preset target person, perform face recognition on the video image.

[0162] S303. Based on the facial image of a person identified by facial recognition of the video image, determine whether a facial image that matches the facial image of a preset target person can be matched.

[0163] In this step, the facial images of people identified based on video images are matched with the facial images of preset target people to determine whether a match can be made.

[0164] If it is (Y), then the target person has arrived, and S304 is executed; if not, then only a stranger has been identified in the video image, and S305 is executed.

[0165] S304. When a face image that matches the face image of the preset target person is matched, it is determined that a target person arrival notification event has occurred.

[0166] S305. When a face image that does not match the face image of the preset target person is matched, a stranger alarm event is determined to have occurred.

[0167] Following S304 and S305, S306 is also included.

[0168] S306. When it is determined that a video notification event has occurred, trigger the event notification operation.

[0169] In the signal processing method of the video doorbell provided in this embodiment, based on the face recognition algorithm and the face image of the preset target person, face recognition is performed on the video image acquired by the image acquisition device, and the recognized face image is matched with the face image of the preset target person. Based on the matching result, it is determined whether a target person arrival notification event or a stranger alarm event has occurred.

[0170] In the above embodiments, when it is determined that a face image matching the face image of the preset target person can be found in the video image, it can be further determined whether a target person arrival notification event has occurred based on the movement of the target person, thereby improving the accuracy of the target person arrival notification event determination.

[0171] In some embodiments, Figure 6 Based on this, S304 may include:

[0172] Step 1: When a face image that matches the face image of the preset target person is found, motion analysis is performed based on the video image.

[0173] In addition, when identifying a target person based on video images, motion analysis can also be performed on the video images to determine the movement of the target person.

[0174] For example, motion analysis may be implemented based on target tracking algorithms, target person posture or other principles, and this disclosure does not limit this.

[0175] Step 2: When the motion analysis results indicate that the target person has entered the set location, determine that a target person arrival notification event has occurred.

[0176] The set location can be the position where the target person arrives at the door, and can be a position 20 cm, 30 cm or other preset distances from the door. This embodiment of the present disclosure is not limited to this. When the target person arrives at the set location, a target person arrival notification event can be determined to have occurred, thereby avoiding misjudging an event where the target person passes by a distance from the door as a target person arrival notification event, thus improving the accuracy of the method.

[0177] In other embodiments, the results of motion analysis may also include the target person moving toward the door, or the target person moving from far to near the door relative to the door, or other static or dynamic states that determine whether the target person has arrived at or is about to arrive at the door. This disclosure does not limit these aspects.

[0178] Following steps one and two, when a notification event indicating the arrival of a target person is detected, the process further includes: unlocking the door to allow the target person to enter. After entering, the target person can manually close the door; or, the door can automatically close after a preset time has elapsed since entering, or after the target person has reached a preset distance from the door.

[0179] For example, the preset distance can be a vertical distance of 30 cm, 45 cm, 50 cm or other optional distance values ​​between the target person and the door; the preset time can be 1 minute, 2 minutes or other optional duration values ​​after the target person enters the door, which can be set according to the collaborative control requirements of the intelligent service system, and this disclosure embodiment does not limit it.

[0180] In some embodiments, the target person is a child or a visitor.

[0181] Specifically, when the target person is a child, the signal processing device can send a notification to the user that the child has arrived home, to remind the user that the child has arrived home; when the target person is a visitor, the signal user device can send a notification to the user that a visitor has arrived, to remind the user that the visitor has arrived.

[0182] In the above embodiments, the methods for obtaining the facial image of the preset target person may include uploading it through a terminal device or obtaining it through user intervention.

[0183] In some embodiments, when a user-interventional method is used to obtain a facial image of a preset target person, facial recognition can be performed based on the video image, and the recognized face can be presented to the user for identification.

[0184] In some embodiments, the method may further include:

[0185] Obtain the facial image of a preset target person uploaded by the user through the first terminal device.

[0186] The first terminal device can be a fixed terminal device or a mobile terminal device. The first terminal device captures a facial image of a preset target person and transmits it to a signal processing device.

[0187] For example, the first terminal device may be an image acquisition device for a video doorbell, or other terminal devices with image acquisition function in an intelligent service system, and this disclosure does not limit this.

[0188] For example, the preset target person's facial image may include at least one of the preset target person's frontal image, left side image, side view image, bottom view image, and top view image, which is convenient for recognizing the target person.

[0189] When only a frontal image of the face of the target person is uploaded, the facial features in the image are clear and unobstructed, so as to achieve accurate identification of the target person.

[0190] In some embodiments, the first terminal device includes a smartphone or a smart speaker.

[0191] One method involves using image acquisition software on a smartphone, such as a camera, to take a picture and capture an image of the face of a pre-defined target person, which is then uploaded to an image processing device. Alternatively, an image acquisition software on a smart speaker, such as a smart speaker with a screen, can also be used to take a picture and capture an image of the face of a pre-defined target person, which is then uploaded to an image processing device.

[0192] The above embodiments illustrate the process steps for applying this signal processing method to scenarios where a child arrives at home or during childcare. The following will describe the steps in conjunction with... Figure 7 and Figure 8 This paper provides an exemplary description of signal processing methods and signal interaction modes in the scenario of children arriving at home, combined with... Figure 9 and Figure 10 This paper provides an exemplary description of signal processing methods and signal interaction modes in child care scenarios.

[0193] In some embodiments, Figure 7 This is a flowchart illustrating a method for applying the video doorbell information processing method provided in this embodiment to a notification scenario when a target person arrives. Figure 8 This is a schematic diagram illustrating a signal interaction method when the information processing method for a video doorbell provided in this embodiment is applied to a notification scenario of the arrival of a target person. (Refer to...) Figure 8 Where 0121 represents children, 21 represents indoor terminal devices, and 22 represents mobile terminal devices for personal use. Based on this, refer to... Figure 7 The signal processing method of the video doorbell may include:

[0194] S211. Obtain the facial image of the preset target person uploaded by the user through the first terminal device.

[0195] Users can upload a child's front-facing photo and nickname via mobile terminal device 22 or indoor terminal device 21, with the child's facial features clearly visible and unobstructed.

[0196] For example, the mobile terminal device 22 may be a user's mobile phone, and this step may include uploading the child's lighting and nickname via an application (APP) on the mobile phone. Alternatively, the indoor terminal device 21 may be a speaker with a screen, and this step may include uploading the child's photo and nickname via the speaker with a screen.

[0197] S212. When a human body is detected by the human body detection sensor, the image acquisition device of the video doorbell is activated.

[0198] When a child approaches the video doorbell, the PIR human detection sensor of the video doorbell senses the approaching person, thereby triggering the video acquisition of the video doorbell, that is, activating the image acquisition device of the video doorbell.

[0199] S213. Video is acquired using an image acquisition device based on a video doorbell.

[0200] Once the image acquisition device is activated, it records the scene within the field of view at the entrance to acquire video images, thus achieving video image acquisition.

[0201] S214. Perform face recognition on video images based on face recognition algorithms and the face image of a preset target person.

[0202] Among them, facial recognition algorithms are used to identify the content of captured video frames in video images to determine whether the person in front of the doorbell is a registered child.

[0203] In this step, based on the facial recognition algorithm and the photos uploaded in the previous steps, it is determined whether the person in the video image is a child whose facial image has been recorded. That is, through the visual algorithm processing, including the facial detection and facial recognition process, it is determined whether the person in the video image is a child whose facial image has been recorded.

[0204] Optionally, the visual algorithm processing in this step can be completed entirely on the video doorbell; the face recognition process can be completed on a cloud server, while only face detection is performed on the video doorbell; or the entire process can be deployed on a cloud server. The specific implementation method can be combined with the hardware structure and intelligent service requirements of the video doorbell, such as hardware specifications, costs, and actual implementation settings of software technology development. This disclosure does not limit this aspect.

[0205] S215. When a face image that matches the face image of the preset target person is matched, motion analysis is performed based on the video image.

[0206] If a facial image in the video matches that of a pre-defined target person, it indicates that the target person is present in the video. At this point, the target person's movements are analyzed to determine whether they intend to enter the house.

[0207] For example, if a recognized face image appears in a video image, it indicates that the recorded child is at the doorway, and motion analysis is needed to determine whether the child is going home or just passing by the door.

[0208] Optionally, analyzing the video stream, i.e. performing motion analysis on the video images, may include: determining whether a child is about to enter the house based on at least one of the following factors: whether the background of the image is displaced, the direction of the child's movement, or the direction the child is facing.

[0209] Optionally, the judgment result and the current time information can be sent to the cloud server for event logging to facilitate subsequent querying.

[0210] Optionally, the visual algorithm processing in this step can be completed entirely on the video doorbell; alternatively, only face detection can be performed on the video doorbell, while the motion recognition process is deployed on a cloud server; or the entire process can be deployed on a cloud server. The specific implementation method can be combined with the hardware structure and intelligent service requirements of the video doorbell, that is, the hardware specifications, costs, and actual implementation status of software technology development. This disclosure does not limit this aspect.

[0211] S216. When the motion analysis result indicates that the target person has entered the set location, determine that a target person arrival notification event has occurred.

[0212] For example, when a child arrives at a designated location at the doorway, it indicates that he / she wants to enter the house.

[0213] S217. When it is determined that a target person has arrived, output a prompt message for the target person's main road.

[0214] According to the settings of the video doorbell user (e.g., a parent), when the video doorbell detects that a child has returned home according to the aforementioned steps, it triggers an event notification operation and outputs a notification message indicating that the target person has arrived.

[0215] For example, if the user-side mobile terminal device 022, such as a mobile APP, has an open notification function set, the user can receive a notification event of the arrival of the target person in the mobile APP. For example, the user can receive a "Baby has come home" message notification in the mobile APP.

[0216] Optionally, users can pre-edit the prompts for the indoor speaker or the built-in indoor unit of the video doorbell in the mobile app or the speaker with a screen. For example, the prompt could be: "Baby's home, remember to wash your hands when you come in." The prompt can be entered as text or recorded directly as voice.

[0217] Based on the above settings, when the video doorbell detects a child returning home according to the aforementioned steps, the indoor speaker or the indoor unit built into the video doorbell will be triggered, emitting a corresponding voice prompt, corresponding to the pre-edited settings. For example, the indoor speaker or the indoor unit built into the video doorbell will broadcast the audio: "Baby's home, remember to wash your hands when you come in."

[0218] S218. Save the video images, the target person arrival notification event, and the target person arrival prompt information to the personal cloud storage space.

[0219] The video images and the information obtained after the aforementioned steps can then be uploaded to the cloud server for viewing and playback.

[0220] For example, the cloud server receives the child's daily home arrival time information uploaded by the video doorbell and saves it in an encrypted personal cloud storage space so that the video doorbell user can view and play it back on a mobile APP or a speaker with a screen.

[0221] In other embodiments, the method may further include: when the video image does not include the recorded child, recording and saving the video to an encrypted personal cloud storage space to allow the video doorbell user to view and play it back on a smart speaker with a screen or a mobile app. This step may be performed after S214.

[0222] Thus, after the above steps, the information processing method of the video doorbell can be applied to the scenario of children returning home, realizing intelligent notification and reminders for children returning home.

[0223] For example, Figure 8 This is a schematic diagram illustrating a signal interaction method when the information processing method for a video doorbell provided in this disclosure is applied to a stranger alarm scenario. It shows the signal interaction method in a child or elderly care scenario, specifically a do-not-disturb care scenario where a child or elderly person is at home. (Refer to...) Figure 8 In this diagram, 0122 represents a stranger, and the meanings of the other reference numerals are the same as those in the attached diagram. Figure 7 The same applies here, so it will not be repeated. Based on this, when children or elderly people are at home, users can activate the do-not-disturb monitoring function of the video doorbell. When this function is implemented by the signal processing method of the video doorbell, the signal processing steps may include:

[0224] Step 1: Upload front-facing photos of family members and frequently visiting friends via mobile app or a speaker with a screen. The facial features should be clear and unobstructed in the photos.

[0225] Step 2: When someone approaches the video doorbell, the PIR human detection sensor of the video doorbell senses the approaching person, thereby triggering the video acquisition of the video doorbell, that is, activating the image acquisition device of the video doorbell to acquire video images.

[0226] Step 3: Based on the facial recognition algorithm and the facial image of the preset target person, identify the video frame content in the acquired video image to determine whether the person in front of the doorbell is a family member or friend who has been registered.

[0227] If so, it means that the person in the video is a family member or friend who has been registered, and no stranger alarm is needed. Proceed to step five. Otherwise, if the video image includes a stranger, proceed to step four.

[0228] For example, the visual algorithm processing in this step can be completed entirely on the video doorbell; or only face detection can be performed on the video doorbell while the face recognition process is deployed on a cloud server; or the entire visual algorithm processing can be deployed on a cloud server. The specific implementation method can be combined with the hardware structure and intelligent service requirements of the video doorbell, such as combining hardware specifications and costs with the actual implementation status of software technology development. This disclosure embodiment is not limited.

[0229] Step 4: If a stranger, identified through video image recognition, presses the ring button on the video doorbell, different response actions can be triggered based on the user's settings, including:

[0230] Action 1: If the user has enabled the "Do Not Disturb" function for the video doorbell in the mobile app or the smart speaker with a screen, the video doorbell enters "Do Not Disturb" mode. In this mode, if a stranger presses the ring button, the action will not trigger the guest ring on the indoor speaker or the video doorbell's built-in unit. Instead, it will trigger the playback of a prompt message set by the user through the mobile app or the smart speaker with a screen. For example, this prompt message could be: "Please don't open the door for strangers, dear." This prompt message can be entered as text or directly recorded as voice; the method of setting this message is not limited in this embodiment.

[0231] Action 2: Send the image of the stranger at the door, along with their facial image or related video footage, to the user's mobile device, such as a mobile app, to alert the user that a stranger is loitering at the door.

[0232] In other implementations, if the user does not enable the Do Not Disturb function of the video doorbell, if a stranger presses the ring button, the act of pressing the ring button will normally trigger the guest ringing function of the indoor speaker or the indoor unit built into the video doorbell.

[0233] Step 5: Record the video and upload it to an encrypted personal cloud storage space so that the video doorbell user can view and play it back on the mobile app or a speaker with a screen.

[0234] Step five is performed after steps three and four.

[0235] Thus, after completing steps one through five above, the information processing method of the video doorbell can be applied to the scenario of child or elderly care at home without disturbance, realizing intelligent care and stranger alarm prompts when children or the elderly are at home, thereby improving the safety of children and the elderly.

[0236] In some embodiments, Figure 9 This is another information processing method for a video doorbell provided in this disclosure, illustrating the process of applying this method to an item care notification scenario. Figure 5 Based on, refer to Figure 9 S202 may include:

[0237] S402. Analyze the video image according to the item tracking algorithm, and determine that an item monitoring notification event has occurred after the target item moves out of the preset area.

[0238] Among them, the object tracking algorithm can track identified target objects based on video images. It can determine the position of the target object based on time series or determine the movement path of the target object, that is, to achieve target object tracking. The process of visual target object tracking can be understood as the process of finding the object of interest (i.e., the target object) defined in the current video frame in subsequent video frames of a video frame.

[0239] When a target item is moved out of the preset area, it indicates that the target item may have been moved and lost. At this time, an item care notification event is determined to occur so that the user can be notified that the target item has been moved. This helps to realize intelligent care of the target item and improve its easy loss problem.

[0240] It should be noted that, Figure 9 S401 and S403 shown in the figure are related to Figure 5 S201 and S203 shown in the figure correspond to each other, and can be understood by referring to the explanation above, which will not be repeated here.

[0241] In some embodiments, the target item may be a delivery item, a takeout item, other target items delivered to the door and temporarily remaining within the field of view of the video doorbell, or other target items placed within the field of view of the video doorbell. This disclosure does not limit this.

[0242] Based on this, the identification of a target item can be achieved through the identification code (i.e., item code) of the express delivery item or takeout item, and the shape, location, size, and other features of the target item (including express delivery items, takeout items, or other target items). This disclosure does not limit this aspect.

[0243] In some embodiments, the preset area is all or part of the field of view of the image acquisition device.

[0244] When the preset area is the entire field of view of the image acquisition device, the removal of the target item is determined once it is moved out of the field of view, thus triggering an item monitoring notification event. In this case, the preset area is relatively large, which helps avoid misjudgments caused by minor displacement of the target item due to other reasons.

[0245] Alternatively, the preset area can be a portion of the image acquisition device's field of view, specifically a portion of the field of view near the door. This portion can also be referred to as the designated area. When the target item is no longer within the designated area of ​​the image acquisition device's field of view, it is determined that the target item has been moved, thus triggering an item monitoring notification event. In this case, the preset and selected areas are relatively small. Even after the target item is moved out of the preset area, it may still be located outside the image acquisition device's field of view, allowing for continued monitoring and thus improving the item's security.

[0246] Based on the above implementation method, where the target item is a courier item, the method may further include determining the courier information of the courier item based on video images in order to identify the courier item.

[0247] In some embodiments, Figure 9 Based on this, before S402, it may also include:

[0248] The algorithm analyzes video images containing item codes (i.e., identification codes) to obtain express delivery information for the items.

[0249] The item code corresponds to a unique express delivery information. For example, the item code can be a numeric string, barcode, QR code, or other encoding format, as long as it can identify the express delivery information of the target item. This embodiment of the disclosure is not limited in this respect.

[0250] In this step, the express delivery information of the express item can be obtained by analyzing the item code in the video image. For example, this may include decoding the item code to obtain the associated express delivery information; or it may include identifying the item code and retrieving pre-set, uniquely associated express delivery information; or it may employ other methods known to those skilled in the art to obtain the express delivery information of the express item based on the item code in the video image. This disclosure does not limit this approach.

[0251] In other embodiments, when the target item is a takeout item, the step may also be: analyzing the video image including the item code based on the encoding recognition algorithm to obtain the takeout information of the takeout item.

[0252] Based on the above implementation method, when the target item is a courier item or a takeout item, the method may further include outputting information related to the item care notification event, including a successful courier (or takeout) identification prompt and / or receiving a courier (or takeout) notification message. The following is an example of courier care.

[0253] In some embodiments, the method may further include at least one of the following:

[0254] Output a message indicating successful package recognition on the video doorbell;

[0255] The second terminal device outputs and receives express delivery notification messages.

[0256] The package can be delivered by a courier or a delivery robot. Once the package information is successfully received, a notification message indicating successful package identification will be displayed outside the video doorbell to inform the courier or delivery robot that the package has been successfully identified and can be placed in a preset area.

[0257] The second terminal device can output a notification message to remind the user or other people in the space where the second terminal device is located to receive the package.

[0258] In some embodiments, the second terminal device includes at least one of a smartphone, a smart speaker, or a video doorbell.

[0259] Among them, smartphones are a type of mobile terminal device that users can carry with them. By receiving delivery notification messages on their smartphones, users can be notified that a package has been delivered to their door and needs to be received. Users can then use these notification messages to sign for the package and specify its location.

[0260] The smart speaker can be placed indoors and receive delivery notifications via its output. It can alert users or others indoors that a package has been delivered to their door and needs to be received. Users or others can then open the door to sign for the package, or sign for it through voice or text interaction with the smart speaker, and specify the package's location.

[0261] Among them, the video doorbell includes an indoor unit, which is used in the same way as the indoor smart speaker, and will not be described in detail here.

[0262] In other embodiments, the video doorbell may also include an outdoor unit that can interact with a courier (or a delivery robot) to receive packages, thereby facilitating the automatic receipt of packages when the user is unable to receive them.

[0263] In other implementations, when the target item is a takeout item, the information output method is similar to the information output method of the above-mentioned express delivery monitoring, and will not be described again.

[0264] In some embodiments, the courier information includes at least one of the following: place of shipment, sender, time of shipment, or name of the courier item.

[0265] Therefore, when outputting information related to item care notification events, information can be output based on the express delivery information and the corresponding express delivery item.

[0266] For example, the output information may include "The package sent by XX0 has been delivered to the door" or "The package sent by XX0, XX1 has been delivered to the door" or "The package sent by XX0 from XX3 has been delivered to the door".

[0267] In other implementations, when the number of express items is more than one, the output message may include "XX11 Express, ..., and XX12 Express have been delivered to the door".

[0268] In some embodiments, when there is identical information in the express delivery information of more than one express item, they can be merged and output to simplify the output information. This disclosure does not limit this aspect.

[0269] For example, Figure 10 This is a schematic diagram illustrating a signal interaction method when the information processing method for a video doorbell provided in this embodiment is applied to an item monitoring notification scenario. (Refer to...) Figure 10 In this context, 011 represents the target item, such as a package. The meanings of the other reference numerals are the same as those in the corresponding reference numerals above, and will not be repeated here. Based on this, taking a courier delivering a package as an example, the signal processing method of this video doorbell includes:

[0270] Step 1: When someone approaches the video doorbell, the PIR human detection sensor of the video doorbell senses the approaching person, thereby triggering the video acquisition function and encoding recognition algorithm of the video doorbell.

[0271] For example, corresponding to different forms of identification codes for express delivery items, the encoding and identification algorithm may include barcode recognition algorithm, QR code recognition algorithm or other encoding and identification algorithm, and this disclosure embodiment is not limited to this.

[0272] Step Two: If the courier holds the package's identification code against the video doorbell's image capture device for a preset time, the encoding and recognition algorithm will identify the package's identification code and its corresponding text information. Subsequently, the video doorbell will emit a message indicating successful package identification, such as a preset notification tone, to inform the courier that the package's information has been successfully identified.

[0273] The preset time can be 2s, 3s, or other time lengths, which are not limited here.

[0274] Optionally, the visual algorithm processing in this step can be completed entirely on the video doorbell itself to speed up the response; alternatively, a collaborative algorithm processing approach between the cloud server and the video doorbell can be used to reduce the cost of the video doorbell. The specific implementation method can be determined based on the hardware structure of the video doorbell and the needs of intelligent services, such as hardware specifications, costs, and the actual implementation status of software technology development, and is not limited here.

[0275] Step 3: Send the text information identified in Step 2 to the cloud server, and obtain more detailed information about the express delivery by calling the access interface of the corresponding logistics system, including the place of origin, sender, delivery time, and name of the express item.

[0276] In other implementations, the identification code of the express item can be directly associated with the express information, so that after the identification code of the express item is identified, the associated express information can be retrieved.

[0277] Step 4: The cloud service will organize the express delivery information determined in Step 3 into natural language and send it to at least one of the user's bound mobile APP, indoor speaker with screen, or video doorbell indoor unit.

[0278] As a result, users can receive corresponding information prompts from at least one of the following: a mobile app, an indoor speaker with a screen, or a video doorbell indoor unit.

[0279] Thus, by following steps one through four above, we can achieve notification of the arrival and receipt of express items, and provide a receipt confirmation for the express items.

[0280] For example, Figure 11 This is a schematic diagram illustrating another signal interaction method when the video doorbell information processing method provided in this embodiment is applied to an item monitoring notification scenario. The curve with the arrow indicates the direction and path of the moved item. (Refer to...) Figure 11 ,exist Figure 10 In addition to the aforementioned steps that can be implemented, the signal processing method may further include:

[0281] Step 5: After scanning the identification code of the package, the video doorbell can start the service of continuing to monitor the package according to the user's settings.

[0282] Once the "Continue to Monitor Packages" function is enabled, the video doorbell's target tracking algorithm (i.e., item tracking algorithm) will continuously track the package.

[0283] Therefore, if the package is moved out of the preset area, an alarm will be triggered on the mobile app's indoor smart speaker or the indoor unit of the video doorbell. The alarm message may read, "The package sent by XX0 has just been moved from the door."

[0284] This allows for the monitoring of packages delivered to the door, thereby improving their security.

[0285] In other embodiments, when the target item is a takeout item, the method of receiving or guarding it is the same as the method of receiving and guarding the express delivery item described above, and will not be repeated here.

[0286] In other embodiments, when the target item is another item placed or located at the doorway, it can be monitored based on its characteristics, which will not be elaborated here.

[0287] Based on the above implementation, when the video doorbell is applied to different scenarios, that is, when it is enabled with different functions, the triggered event notification operation may include outputting information associated with the video notification event on at least one terminal device.

[0288] In some embodiments, when the method is applied to a stranger alert scenario, Figure 5 Based on this, S203 may include:

[0289] When a video notification event includes a stranger alarm event, a stranger alarm message is output through a fourth terminal device, which includes at least one of a smartphone, a smart speaker, or a video doorbell.

[0290] When a stranger alarm event is determined to have occurred based on video images, a stranger alarm message can be sent through at least one of a smartphone, smart speaker, or video doorbell.

[0291] That is, when a stranger is identified as loitering based on video images, a stranger alarm message can be sent simultaneously by any one, two or three of the following: a smartphone, a smart speaker, or a video doorbell.

[0292] For example, a stranger alarm message can be received and output in a mobile app; for instance, a user can receive a message saying "A stranger is loitering at the door" in the mobile app, and the app will output a facial image or related video of the stranger. An indoor speaker (and smart speaker) or the indoor unit of a video doorbell can be triggered to issue a voice prompt; for example, the indoor unit of an indoor speaker or video doorbell can receive an event notification and broadcast an audio prompt saying "Don't open the door to strangers, baby." It should be noted that the aforementioned stranger alarm message can be output on any third-party terminal device, or on at least two of the aforementioned third-party terminal devices. For example, a message saying "A stranger is loitering at the door" can be output in the mobile app, and the prompt "Don't open the door to strangers, baby" can also be broadcast on the indoor speaker.

[0293] In some embodiments, when the method is applied to an item care scenario, Figure 5 Based on this, S203 may include:

[0294] When a video notification event includes an item care notification event, an item care notification message is output through a fifth terminal device, which includes at least one of a smartphone, a smart speaker, or a video doorbell.

[0295] When an item care notification event is determined to have occurred based on video images, an item care notification message can be sent via at least one of a smartphone, smart speaker, or video doorbell.

[0296] That is, when a stranger is identified as lingering based on video images, a notification message for item monitoring can be sent jointly by any one, two or three of the following: a smartphone, a smart speaker, or a video doorbell.

[0297] For example, when a target item is delivered to the door, a notification message can be received and output to the mobile app; for instance, the user may receive a message saying "Item XX 1 has been delivered to the door" in the mobile app. The indoor speaker or the indoor unit of the video doorbell can be triggered to issue a voice prompt; for instance, the indoor speaker or the indoor unit of the video doorbell may receive an event notification and broadcast an audio prompt saying "Item XX 1 has been delivered to the door". When a target item is moved out of a preset area, a notification message can be received and output to the mobile app; for instance, the user may receive a message saying "Item XX 1 has been moved from the door" in the mobile app. The indoor speaker or the indoor unit of the video doorbell can be triggered to issue a voice prompt; for instance, the indoor speaker or the indoor unit of the video doorbell may receive an event notification and broadcast an audio prompt saying "Item XX 1 has been moved from the door".

[0298] It should be noted that the item care notification messages in both scenarios can be output from any third-party terminal device, or from at least two of the aforementioned third-party terminal devices. For example, a message stating "A stranger is loitering at the door" can be output to a mobile app, while a reminder message stating "Baby, don't open the door for strangers" can be broadcast on an indoor speaker.

[0299] In any of the methods shown in the above embodiments, the step of performing video analysis based on video images to determine whether a video notification event has occurred is performed in at least one of the video doorbell local, cloud server, or sixth terminal device.

[0300] Video analysis based on video images includes the detection of faces in the video images (i.e., face detection) and the recognition of faces (i.e., face recognition). For example, the face detection and face recognition processes can be performed simultaneously without involving signal transmission interactions. For instance, the complete process including face detection and face recognition can be performed locally on the video doorbell, on a cloud server, or on a sixth terminal device. Alternatively, face detection and face recognition can be performed at different locations, involving signal transmission interactions. For example, face detection can be performed at one of the video doorbell's local location, a cloud server, or a sixth terminal device; after detecting a face image, it can be transmitted to another of the video doorbell's local location, a cloud server, or a sixth terminal device to perform the face recognition process.

[0301] The sixth terminal device is a terminal device in an intelligent service system (also known as a smart home network), including a video doorbell. This means it is essentially a device within the smart home network. The smart home network may also include other terminal devices such as smart speakers, smart TVs, and smart washing machines; this embodiment does not limit this. The aforementioned face detection and / or face recognition processes can be executed on the sixth terminal device, i.e., on other devices within the smart home network, effectively sharing some of the data processing workload from the cloud server.

[0302] The signal processing method for video doorbells provided in this disclosure improves the intelligent services of video doorbells. In this method, based on the face recognition algorithm, it combines multiple visual processing algorithms such as image background displacement detection algorithm, child movement direction detection algorithm, QR code recognition algorithm, and target object tracking algorithm. Furthermore, it applies the visual acquisition data (i.e., video images) of the video doorbell to scenarios such as children returning home / childcare and item care (e.g., delivery item supervision), effectively expanding the existing application scenarios of video doorbells and making the video doorbell more intelligent and closer to a home assistant.

[0303] Specifically, when applied to scenarios involving children returning home and childcare, this method combines facial recognition algorithms, image background displacement detection algorithms, and child movement direction detection algorithms. Through linkages between the video doorbell and indoor smart speakers, and between the video doorbell and the user's mobile device, it enables intelligent child safety monitoring, giving parents greater peace of mind and easing their worries about their children's safety. When applied to item monitoring, this method incorporates QR code recognition and target object tracking algorithms into the video doorbell's intelligent processing. Through linkages between the video doorbell and indoor smart speakers, between the video doorbell and the user's mobile device, and with cloud-based logistics systems (such as cloud servers), it enables intelligent monitoring of packages. This transforms the video doorbell from merely a tool into a home assistant at the doorstep.

[0304] In other implementations, besides applying this method to smart home management, it can also be applied to other commercial scenarios. For example, the method can be applied to the management of companies in office buildings. By detecting and recognizing facial images of people in video images captured by an image acquisition device, it can identify whether a major client is present in the video image, thus identifying a major client visit event. When a major client visit event occurs, a message notification can be sent to remind relevant personnel to prepare for the reception in advance. Alternatively, facial analysis can be performed based on the video images to determine whether the people appearing are new or old clients, and an event notification can be sent to remind relevant personnel to make appropriate reception preparations in advance.

[0305] Alternatively, this method can be applied to shop-related scenarios, analyzing video images to notify shop owners of relevant events. For example, based on facial analysis of video images, it can identify whether the people appearing are new or returning customers. Furthermore, if they are returning customers, information about their preferences and purchasing habits can be sent to the shop owner, allowing the owner to communicate with them in a manner consistent with those habits. Alternatively, based on video image analysis, items within a certain range inside or near the shop entrance can be monitored, and a notification message can be sent when items are moved, serving as a theft prevention measure and helping to ensure the safety of the shop owner's property.

[0306] In other embodiments, the method provided in this disclosure can also be applied to other scenarios where it is necessary to provide prompts for events related to a target person or an item, and this disclosure does not limit this application.

[0307] Based on the above embodiments, this disclosure also provides a method for collecting items based on a video doorbell, so as to realize intelligent collection and notification of items based on the video doorbell, and improve the convenience of item collection. The following description uses a courier item as an example. When the item is a takeout item or other item delivered to the door, the item collection method is similar to the courier collection method, the only difference being that a notification message corresponding to the item is generated and output for the delivered item. This disclosure will not elaborate further.

[0308] In some embodiments, Figure 13 This is a flowchart illustrating a method for retrieving items based on a video doorbell, as provided in an embodiment of this disclosure. (Refer to...) Figure 13 The method for collecting items based on a video doorbell may include:

[0309] S501. Acquire video images captured by the image acquisition device based on the video doorbell.

[0310] The video image includes an item code. Each item code uniquely corresponds to an item and can be associated with item information. For example, the item code can be a numeric string, barcode, QR code, or other encoding format, as long as it can identify the item information of the target item. This disclosure does not limit this.

[0311] In this step, video images including item codes are acquired to prepare for obtaining item information based on item codes in subsequent steps.

[0312] For example, the image acquisition device in this step can be any of the image acquisition devices in the above embodiments, or other types of image acquisition devices known to those skilled in the art. This disclosure does not limit the specific type of image acquisition device.

[0313] S502. Analyze the video image according to the coding recognition algorithm to obtain the express delivery information of the express item corresponding to the item code.

[0314] In this step, the video image can be analyzed using an encoding recognition algorithm based on the item code in the video image to obtain the item information corresponding to the item code, such as the express delivery information of express delivery items and the takeaway information of takeaway items.

[0315] For example, this step may include decoding the item code to obtain the item information associated with it; or it may include identifying the item code and retrieving the item information that is pre-set and uniquely associated with it; or it may use other methods known to those skilled in the art to obtain item information based on the item code in the video image, which is not limited in this embodiment.

[0316] In some embodiments, the courier information may include at least one of the following: place of origin, sender, time of shipment, or name of the courier item.

[0317] The aforementioned express delivery information can be used to identify express delivery items and generate corresponding express delivery notification messages.

[0318] S503. Generate a notification message for receiving express delivery based on express delivery information.

[0319] Among them, receiving express delivery notification messages includes all or part of the information in the express delivery information.

[0320] For example, the received express delivery notification message may only include the sender or the name of the item, or the express delivery notification message may include two, three or more parameters from the express delivery information mentioned above, which can be set according to the needs of receiving the item. This disclosure embodiment does not limit this.

[0321] Optionally, receiving delivery notification information may include delivery information that uniquely corresponds to the current delivery item, thereby improving the accuracy of delivery receipt.

[0322] This step may include generating a delivery notification message that includes at least some of the delivery messages, based on at least some of the delivery messages and in combination with a preset sentence structure.

[0323] For example, when the courier message is sender XX0, the generated courier receipt notification message can be "The courier item sent by sender XX0 has been delivered to the door"; when the courier message is sender location XX3, the generated courier receipt notification message can be "The courier item sent by sender location XX3 has been delivered to the door"; when the courier message includes sender XX1 and courier item name XX3, the generated courier receipt notification message can be "The courier item XX3 sent by sender XX0 has been delivered to the door".

[0324] In other implementations, when other express delivery information is used, the received express delivery notification message may also be generated in combination with other preset sentence patterns, which is uncertain in this embodiment.

[0325] S504, Outputs a message indicating receipt of a package delivery notification.

[0326] In this step, the delivery notification message generated in S503 is output to inform the user that the package or other items have been successfully received, thereby realizing intelligent item collection and improving the intelligence and convenience of item collection.

[0327] For example, the delivery notification message can be output at the item receiving device, or it can be sent to other terminal devices and output on those devices. This disclosure does not limit the scope of the invention.

[0328] The doorbell-based item retrieval method provided in this disclosure can acquire video images captured by an image acquisition device based on a video doorbell; analyze the video images according to an encoding recognition algorithm to obtain the express delivery information of the express delivery item corresponding to the item code; generate and output a receiving express delivery notification message based on the express delivery information. Therefore, intelligent item retrieval can be achieved based on a video doorbell, thereby improving the intelligence and convenience of item retrieval.

[0329] In the above embodiments, the analysis of video images can be performed locally on the video doorbell, or on a cloud server or terminal device.

[0330] In some embodiments, Figure 13 Based on this, S502 may include:

[0331] The video image is analyzed locally by the video doorbell using an encoding recognition algorithm.

[0332] The video image analysis program can be set locally on the video doorbell, which reduces the signal transmission and interaction process, thereby reducing the time spent on signal transmission and interaction and improving the response speed of the video doorbell.

[0333] In some embodiments, Figure 13 Based on this, S502 may include:

[0334] The video images are sent to a cloud server or terminal device, which then analyzes the video images based on the encoding and recognition algorithm.

[0335] One approach is to use a collaborative algorithm processing method between a cloud server and the video doorbell terminal, or a collaborative algorithm processing method between a terminal device and the video doorbell terminal. This involves setting the program for analyzing video images on a cloud server or a terminal device, allowing video analysis to be performed elsewhere outside the video doorbell. This simplifies the structure of the video doorbell and reduces its cost.

[0336] In the above implementation, the output of the received express delivery notification message can be performed locally on the video doorbell or on the terminal device.

[0337] In some embodiments, Figure 13 Based on this, S504 may include:

[0338] The video doorbell displays and receives delivery notification messages locally.

[0339] The delivery notification message can be output locally on the video doorbell. For example, the video doorbell includes an indoor unit and an outdoor unit, and the delivery notification message can be received based on the output of at least one of the indoor and outdoor units, thereby completing the output of the delivery notification message and realizing the intelligent receipt of delivery items.

[0340] For example, when it can be confirmed that someone is inside the room, the indoor unit of the video doorbell outputs a delivery notification message to inform the person inside that a package is waiting to be received. When it can be confirmed that no one is inside the room, the indoor unit of the video doorbell does not output a delivery notification message; in this case, the delivery notification message can be output on other terminal devices.

[0341] For example, the outdoor unit of a video doorbell can output a notification message for receiving a package, such as "Item XX1 has been successfully received," so that the courier or delivery robot can confirm that the package has been successfully delivered.

[0342] In other embodiments, receiving express delivery notification messages can also be output on a terminal device, which may include an indoor smart speaker or a mobile smart terminal.

[0343] In some embodiments, Figure 13 Based on this, S504 may include:

[0344] The terminal device receives a delivery notification message from the video doorbell or cloud server and outputs the delivery notification message.

[0345] When a video analysis program based on video images is deployed on a video doorbell or cloud server, the video doorbell or cloud server can send the generated package delivery notification message to the terminal device, which will then output the message to notify the user that a package is waiting to be received.

[0346] For example, the terminal device may be other indoor terminal devices in the smart service system, such as a smart speaker, or a mobile terminal device carried by the user, such as a smartphone.

[0347] It should be noted that when a video analysis program based on video images is deployed on a terminal device, the terminal device can directly output a notification message for receiving express delivery after analyzing and processing the video images.

[0348] In other embodiments, the received express delivery notification message can be output either locally on the video doorbell or on the terminal device, or both; this disclosure does not limit this.

[0349] Based on the above implementation method, when it is determined that the video image includes express delivery items, the method may further include guarding the items placed at the door.

[0350] In some embodiments, after determining that the video image includes a delivery item, the method may further include:

[0351] Step 1: Analyze the video images using the item tracking algorithm. Once the target item moves out of the preset area, determine that an item monitoring notification event has occurred.

[0352] Among them, the object tracking algorithm can track identified target objects based on video images. It can determine the position of the target object based on time series or determine the movement path of the target object, that is, to achieve target object tracking. The process of visual target object tracking can be understood as the process of finding the object of interest (i.e., the target object) defined in the current video frame in subsequent video frames of a video frame.

[0353] When a target item is moved out of the preset area, it indicates that the target item may have been moved and lost. At this time, an item care notification event is determined to occur so that the user can be notified that the target item has been moved. This helps to realize intelligent care of the target item and improve its easy loss problem.

[0354] Step 2: Output the item care notification message.

[0355] Based on step one above, when an item monitoring notification event is detected, it indicates that the item may be lost. At this time, an item monitoring notification message is output to promptly remind the user or other associated personnel to check the item status, thereby reducing the probability of item loss and improving item security.

[0356] In the above embodiments, the preset area is all or part of the field of view of the image acquisition device.

[0357] When the preset area is the entire field of view of the image acquisition device, the removal of the target item is determined once it is moved out of the field of view, thus triggering an item monitoring notification event. In this case, the preset area is relatively large, which helps avoid misjudgments caused by minor displacement of the target item due to other reasons.

[0358] Alternatively, the preset area can be a portion of the image acquisition device's field of view, specifically a portion of the field of view near the door. This portion can also be referred to as the designated area. When the target item is no longer within the designated area of ​​the image acquisition device's field of view, it is determined that the target item has been moved, thus triggering an item monitoring notification event. In this case, the preset and selected areas are relatively small. Even after the target item is moved out of the preset area, it may still be located outside the image acquisition device's field of view, allowing for continued monitoring and thus improving the item's security.

[0359] Based on the above implementation method, the video images are analyzed using an object tracking algorithm, which can be done locally on the video doorbell or on a cloud server or terminal device.

[0360] In some embodiments, analyzing video images according to an object tracking algorithm may include:

[0361] The video images are analyzed locally by the video doorbell using an item tracking algorithm.

[0362] This reduces the time spent transmitting video images to other locations, thus helping to speed up the response of the video doorbell.

[0363] In some embodiments, analyzing video images according to an object tracking algorithm may include:

[0364] The video images are sent to a cloud server or terminal device, which then analyzes the video images based on the item tracking algorithm.

[0365] In this way, the relevant programs for analyzing video images based on item tracking algorithms can be deployed on cloud servers or terminal devices, without the need for local deployment on the video doorbell, which helps to simplify the structure of the local end of the video doorbell and reduce its cost.

[0366] Based on the above implementation method, the output of item care notification messages can be performed locally on the video doorbell, or on a cloud server or terminal device.

[0367] In some embodiments, the output item care notification message may include:

[0368] Output item monitoring notification messages locally on the video doorbell.

[0369] The item care notification message can be output locally on the video doorbell. For example, when the video doorbell includes an indoor unit and an outdoor unit, the indoor unit can be used to output the item care notification message to notify people indoors to take care of their items and prevent items from being lost; or the outdoor unit can be used to output the item care notification message to scare away people or small animals (such as puppies) and other creatures that move items. This disclosure is not limited in this respect.

[0370] It should be noted that when the indoor unit outputs an item care notification message, the item care notification message may be an alarm sound, such as a ringing bell, or it may be a pre-set audio, such as "Put down item XX1", etc. This disclosure embodiment does not limit this.

[0371] In some embodiments, the output item care notification message may include:

[0372] The terminal device receives item monitoring notification messages from the video doorbell or cloud server and outputs item monitoring notification messages.

[0373] When the video analytics program is deployed on a video doorbell or cloud server, the video doorbell or cloud server can send the generated item monitoring notification message after analysis and processing to the terminal device, which will then output the message to notify the user that the item has been moved.

[0374] For example, the terminal device may be other indoor terminal devices in the smart service system, such as a smart speaker, or a mobile terminal device carried by the user, such as a smartphone.

[0375] It should be noted that when the video analysis program is deployed on the terminal device, the terminal device can directly output a product monitoring notification message after analyzing and processing the video images.

[0376] In other embodiments, the item monitoring notification message may be output either locally on the video doorbell or on the terminal device, or both; this disclosure does not limit this.

[0377] The item retrieval method based on a video doorbell provided in this disclosure can acquire video images captured by an image acquisition device based on the video doorbell; analyze the video images according to an encoding recognition algorithm to obtain the express delivery information of the express delivery item corresponding to the item code; generate and output a receiving express delivery notification message based on the express delivery information. Thus, intelligent item retrieval can be achieved based on the video doorbell, thereby improving the intelligence and convenience of item retrieval. Furthermore, the video images can be analyzed according to an item tracking algorithm, and an item monitoring notification event can be determined after the target item moves out of a preset area, thereby achieving intelligent item monitoring. The video analysis can be performed at least at one of the following: locally on the video doorbell, on a terminal device, or on a smart server; the receiving express delivery notification message can be output locally on the video doorbell or on the terminal device; and the item monitoring notification message can be output locally on the video doorbell or on the terminal device. The terminal device includes a smartphone, an indoor smart speaker, or other terminal devices in a smart service system.

[0378] Based on the above embodiments, this disclosure also provides an information processing device for a video doorbell, which can be used to execute the information processing method for a video doorbell in the above embodiments. Therefore, this information processing device for a video doorbell also has the beneficial effects of the information processing method for a video doorbell that it can execute. The similarities can be understood with reference to the above text, and will not be repeated here.

[0379] In some embodiments, Figure 15 This is a schematic diagram of the structure of an information processing device for a video doorbell provided in an embodiment of this disclosure. (Refer to...) Figure 15 The information processing device 600 of the video doorbell may include: a video image acquisition module 601, used to acquire video images acquired by an image acquisition device; the image acquisition device adopts any of the video doorbell image acquisition devices provided in the above embodiments, and the video image includes a video image after the field of view is enlarged by a field magnifying glass; a video notification event judgment module 602, used to perform video analysis based on the video image to determine whether a video notification event has occurred; and an event notification operation triggering module 603, used to trigger an event notification operation when it is determined that a video notification event has occurred.

[0380] In the signal processing device 600 for a video doorbell provided in this embodiment, the video image acquisition module 601 can acquire video images captured by the image acquisition device; the video notification event judgment module 602 can perform video analysis based on the video images to determine whether a video notification event has occurred; and the event notification operation triggering module 603 can trigger an event notification operation when a video notification event is determined to have occurred. The image acquisition device can acquire images within a large vertical field of view, thereby enabling notification operations for events such as children arriving home, guests visiting, stranger alarms, item arrivals, and item removals based on video image analysis. This improves the intelligence level of the video doorbell and facilitates its use as a "home security assistant."

[0381] In some embodiments, the device may further include: an image acquisition device activation module, used to activate the image acquisition device of the video doorbell when a human body is detected by the human body detection sensor.

[0382] This makes the start-up timing of the image acquisition device controllable. On the one hand, the image acquisition device is not always in working condition, which helps to reduce its energy consumption and extend its service life. On the other hand, the image acquisition device starts when the human body detection sensor detects a human body, which can effectively monitor people passing by or lingering at the door, without collecting a large number of video images without human bodies. This reduces the amount of computation in the video image analysis process, improves the efficiency of video image analysis, and helps to improve the response speed of the video doorbell and the intelligent service system including the video doorbell.

[0383] In some embodiments, video notification events include a target person arrival notification event, a stranger alert event, or an item care notification event.

[0384] Based on this, the video event notification judgment module can include different sub-modules to be applied to different scenarios.

[0385] In some embodiments, the video notification event determination module may include: a face recognition submodule, used to perform face recognition on the video image based on a face recognition algorithm and a face image of a preset target person; a first event determination submodule, used to determine that a target person arrival notification event has occurred when a face image that matches the face image of the preset target person is matched; or, a second event determination submodule, used to determine that a stranger alarm event has occurred when a face image that does not match the face image of the preset target person is matched.

[0386] Therefore, based on the detection and recognition of pre-set target people or strangers, it is possible to determine the people in video images, which can be applied to scenarios such as children coming home, guests visiting, or strangers lingering, to provide information prompts.

[0387] In some embodiments, the first event determination submodule may include: a motion analysis unit, configured to perform motion analysis based on the video image when a face image that matches the face image of a preset target person is found; and an event determination unit, configured to determine that a target person arrival notification event has occurred when the motion analysis result indicates that the target person has entered a set location.

[0388] In this way, based on the target person's movement and location, and in conjunction with the set location, it is possible to determine whether a target person's arrival notification event has occurred, thereby improving the accuracy of the judgment and contributing to the improvement of the accuracy of the device and the intelligent service system including the device.

[0389] In some embodiments, the video notification event determination module may further include: a third event determination submodule, used to analyze the video image according to the item tracking algorithm, and determine that an item care notification event has occurred after the target item moves out of the preset area.

[0390] In this way, intelligent monitoring of target items can be achieved based on the detection, identification, and tracking of the target items, thereby improving the safety of the items.

[0391] In some embodiments, when the target item is a courier item, the device may further include: a courier information acquisition module, used to analyze video images including item codes based on an encoding recognition algorithm to obtain courier information of the courier item.

[0392] In this way, express delivery information can be obtained based on the item codes in video images, so as to achieve the identification of target items.

[0393] The above description exemplifies the signal processing device by illustrating the signal processing method of a video doorbell within the same signal processing device. In other embodiments, the signal processing method of the video doorbell can also be executed interactively in multiple different devices. This disclosure also provides a video doorbell system.

[0394] The video doorbell system provided in this disclosure can be used to execute the signal processing method of the video doorbell in the above embodiments. Therefore, this video doorbell system also has the beneficial effects of the signal processing method of the video doorbell that it can execute, and the similarities can be understood with reference to the above description, which will not be repeated here.

[0395] In some embodiments, Figure 16 This is a schematic diagram of the structure of a video doorbell system provided in an embodiment of this disclosure. (Refer to...) Figure 16 The video doorbell system 610 includes: an image acquisition device 611 for the video doorbell and an information processing device 612.

[0396] The image acquisition device 611 of the video doorbell can be any of the image acquisition devices described in the above embodiments. This image acquisition device is used to acquire video images, including video images after the field of view has been magnified by a magnifying glass. The information processing device 612 is used to perform video analysis based on the video images to determine whether a video notification event has occurred. Upon determining that a video notification event has occurred, it triggers an event notification operation. Therefore, based on video analysis of the video images, notification operations can be implemented for events such as children arriving home, guests visiting, stranger alarms, the arrival of items, and items being moved, thereby improving the intelligence level of the video doorbell and facilitating its use as a "home security assistant."

[0397] In some embodiments, the system may further include: a human detection sensor for activating the image acquisition device of the video doorbell when a human body is detected.

[0398] This makes the start-up timing of the image acquisition device controllable. On the one hand, the image acquisition device is not always in working condition, which helps to reduce its energy consumption and extend its service life. On the other hand, the image acquisition device starts when the human body detection sensor detects a human body, which can effectively monitor people passing by or lingering at the door, without collecting a large number of video images without human bodies. This reduces the amount of computation in the video image analysis process, improves the efficiency of video image analysis, and helps to improve the response speed of the video doorbell and the intelligent service system including the video doorbell.

[0399] In some embodiments, video notification events include a target person arrival notification event, a stranger alert event, or an item care notification event.

[0400] Based on this, in order to be applicable to different scenarios, the information processing device can be deployed with corresponding software programs to realize intelligent services in each scenario.

[0401] In some embodiments, when the video doorbell system is applied to a target person arrival scenario or a stranger alarm scenario, the system may further include: a first terminal device for uploading a preset target person's facial image; and an information processing device for performing facial recognition on the video image based on a facial recognition algorithm and the preset target person's facial image, determining that a target person arrival notification event has occurred when a facial image that matches the preset target person's facial image is matched; or, determining that a stranger alarm event has occurred when a facial image that does not match the preset target person's facial image is matched.

[0402] In this way, the first terminal device can preset a target person, and the information processing device can determine whether the facial image detected in the video image is the target person or a stranger based on the preset target person, thereby realizing the determination of target person arrival notification events and stranger alarm events. Therefore, users can upload facial images of preset target persons as needed, which can meet the personalized needs of different users.

[0403] In some embodiments, the first terminal device includes a smartphone or a smart speaker.

[0404] Both smartphones and smart speakers can include image acquisition capabilities to capture facial images of preset target individuals and upload them to an information processing device. This enables collaborative work between different terminal devices in a video doorbell system. While achieving multiple functions, the various functions and their corresponding execution steps can be distributed, simplifying the structure of each terminal device, reducing their individual costs, and providing flexibility in application.

[0405] In some embodiments, the device further includes a video doorbell and / or a second terminal device; the information processing device is also configured to analyze video images including item codes based on an encoding recognition algorithm to obtain express delivery information of the express items; and to analyze video images according to an item tracking algorithm to determine that an item monitoring notification event has occurred after the express items have moved out of a preset area; the video doorbell is configured to output a successful express delivery identification prompt message; and the second terminal device is configured to output an express delivery notification message.

[0406] In this way, based on the coding recognition algorithm and the item tracking algorithm, express delivery items or other target items in video images can be identified and located and tracked. When the target item is successfully identified, the video doorbell can output a message indicating that the express delivery has been successfully identified, so that the courier or the delivery robot can place the express delivery item in a preset area; and / or, the second terminal device can output a message indicating that the user or other people in the space where the second terminal device is located can receive the express delivery or that the express delivery has been successfully received, thereby realizing the intelligent reception and care of express delivery items or other target items.

[0407] In some embodiments, the second terminal device includes at least one of a smartphone, a smart speaker, or a video doorbell.

[0408] In this way, smartphones can be used to notify users, such as those carrying them, of receiving delivery notifications; smart speakers can be used to notify people indoors of receiving delivery notifications; and video doorbells can be used to notify people indoors of receiving delivery notifications.

[0409] It should be noted that the above three message prompt methods can be selected to be executed, or two of them can be executed, or all of them can be executed. This embodiment of the disclosure does not limit this.

[0410] In some embodiments, the system may further include: a third terminal device, configured to output a target person arrival notification message when the video notification event includes a target person arrival notification event; the third terminal device includes at least one of a smartphone, a smart speaker, or a video doorbell;

[0411] In this way, at least one of a smartphone, smart speaker, or video doorbell can be used to output a notification message indicating the arrival of the target person, thereby achieving intelligent reminders of the target person's arrival.

[0412] In some embodiments, the system may further include: a fourth terminal device for outputting a stranger alarm message when the video notification event includes a stranger alarm event; the fourth terminal device includes at least one of a smartphone, a smart speaker, or a video doorbell.

[0413] In this way, at least one of a smartphone, smart speaker, or video doorbell can be used to output a stranger alarm message, thereby realizing intelligent alarm for strangers loitering.

[0414] In some embodiments, the system may further include: a fifth terminal device, configured to output an item care notification message when the video notification event includes an item care notification event; the fifth terminal device includes at least one of a smartphone, a smart speaker, or a video doorbell.

[0415] In this way, at least one of a smartphone, smart speaker, or video doorbell can be used to output item monitoring notification messages, thereby realizing intelligent monitoring of items at the door.

[0416] In the above embodiments, the signal processing device is located in at least one of the video doorbell local unit, cloud server, or sixth terminal device.

[0417] The signal processing device can be entirely located in the local video doorbell, a cloud server, or a sixth terminal device; alternatively, the functional modules of the signal processing device can be distributed among at least one of the local video doorbell, a cloud server, or a sixth terminal device. When the functional modules of the signal processing device are deployed in different locations, the signal processing also includes a signal interaction process, which can be understood above and will not be elaborated here.

[0418] In some embodiments, when the video doorbell system is applied in an office building or shop setting, the video doorbell's image acquisition device can acquire video images and send them to an information processing device. The information processing device analyzes the video images and, upon determining that a major customer visit, a new customer visit, a returning customer visit, a returning customer visiting the shop, a new customer visiting the shop, goods being moved, or goods awaiting signature have occurred, triggers an event notification operation.

[0419] The image processing device can be located locally on the video doorbell, on a cloud server, or on any terminal in the doorbell system, such as the terminal that issues the event notification operation. This embodiment of the present disclosure is not limited in this regard. The event notification operation can be issued locally by the video doorbell or by any terminal in the video doorbell system; that is, it can realize on-site alarm prompts or alarm prompts from a client installed on the terminal.

[0420] For example, when this video doorbell system is applied to company management in an office building, it can receive and output notification messages about visits by major clients in a mobile app. For instance, a user might receive a message in the app stating, "Client XXX has entered the building and is expected to arrive at our office in 5 minutes. Please prepare to receive them." The indoor speaker or the indoor unit built into the video doorbell can be triggered to issue a voice prompt. For example, the indoor speaker or the indoor unit built into the video doorbell might receive the event notification and broadcast the audio prompt, "Client XXX has entered the building and is expected to arrive at our office in 5 minutes. Please prepare to receive them."

[0421] Alternatively, when this video doorbell system is applied to shop-related scenarios, it can provide notifications for goods receipt and monitoring. For example, when goods are delivered to the door, a notification message can be received and output to the mobile app; at this time, the user can receive a message in the mobile app saying, "Item XXXX has been delivered to the door, please arrange for timely receipt." The indoor speaker or the indoor unit built into the video doorbell can be triggered to issue a voice prompt; for example, the indoor speaker or the indoor unit built into the video doorbell can receive the event notification and broadcast the audio prompt message, "Item XXXX has been delivered to the door, please arrange for timely receipt." After the item is received, if it is still within the field of view of the image acquisition device, the item monitoring function of the video doorbell system can be activated. When the target item is moved out of the preset area, an item monitoring notification message can be received and output to the mobile app; for example, the user can receive a message in the mobile app saying, "Item XXXX has been moved from the door." The indoor speaker or the indoor unit built into the video doorbell can be triggered to issue a voice prompt; for example, the indoor speaker or the indoor unit built into the video doorbell will receive an event notification and broadcast an audio prompt message such as "Item XXXX has been moved from the door".

[0422] Similarly, when video analysis and event notification operations are both performed locally on the video doorbell or on the same terminal, there is less signal interaction and the event response speed is faster. When video analysis and event notification operations involve signal interaction between multiple terminals, please refer to the above for understanding, which will not be repeated here.

[0423] Based on the above embodiments, this disclosure also provides a video doorbell-based item collection device, which can be used to execute the video doorbell-based item collection method described above. Therefore, this video doorbell-based item collection device also possesses the beneficial effects of its executable item collection method; the similarities can be understood with reference to the above explanation and will not be repeated here.

[0424] In some embodiments, Figure 17 This is an embodiment of a video doorbell-based item collection device provided in this disclosure. (See also...) Figure 17 The item receiving device 700 may include: a video image acquisition module 701, used to acquire video images captured by an image acquisition device based on a video doorbell, the video images including item codes; a video image analysis module 702, used to analyze the video images according to an encoding recognition algorithm to obtain the express delivery information of the express item corresponding to the item code; an express delivery notification message generation module 703, used to generate an express delivery notification message based on the express delivery information; and an express delivery notification message output module 704, used to output the express delivery notification message.

[0425] In the doorbell-based item collection device 700 provided in this embodiment, the video image acquisition module 701 can acquire video images captured by the image acquisition device based on the video doorbell, and the video images include item codes; the video image analysis module 702 can analyze the video images according to the code recognition algorithm to obtain the express delivery information of the express item corresponding to the item code; the express delivery notification message generation module 703 can generate an express delivery notification message based on the express delivery information; and the express delivery notification message output module 704 can output the express delivery notification message. Therefore, intelligent item collection can be achieved based on the video doorbell, thereby improving the intelligence and convenience of item collection.

[0426] In some embodiments, the item receiving device may further include: an item care notification event determination module, configured to analyze video images according to an item tracking algorithm and determine that an item care notification event has occurred after the target item moves out of a preset area; and an item care notification message output module, configured to output an item care notification message.

[0427] This enables intelligent monitoring of collected items, which helps reduce the possibility of loss and improves item security.

[0428] The above description only illustrates the item collection device by using the video doorbell-based item collection method executed within the same item collection device as an example. In other embodiments, the video doorbell-based item collection method can also be executed interactively in multiple different devices. This disclosure also provides a video doorbell-based item collection system.

[0429] The item collection system based on a video doorbell provided in this disclosure can be used to execute the item collection method based on a video doorbell described in the above embodiments. Therefore, this item collection system based on a video doorbell also has the beneficial effects of the item collection method based on a video doorbell that it can execute; the similarities can be understood with reference to the above description, and will not be repeated here.

[0430] In some embodiments, Figure 18 This is a schematic diagram of a video doorbell-based item retrieval system provided in an embodiment of this disclosure. (Refer to...) Figure 18 The item collection system 710 based on a video doorbell may include: an image acquisition device 711 and an information processing device 712 for the video doorbell.

[0431] The information processing device 712 includes: a video image acquisition module 7121, used to acquire video images captured by the image acquisition device based on the video doorbell, the video images including item codes; a video image analysis module 7122, used to analyze the video images according to the code recognition algorithm to obtain the express delivery information of the express delivery items corresponding to the item codes; a receiving express delivery notification message generation module 7123, used to generate receiving express delivery notification messages based on the express delivery information; and a receiving express delivery notification message output module 7124, used to output the receiving express delivery notification message.

[0432] Therefore, this item collection system based on a video doorbell can realize intelligent item collection, thereby improving the intelligence and convenience of item collection.

[0433] In some embodiments, the video image analysis module is located locally on the video doorbell, or the video image analysis module is located on a cloud server or terminal device.

[0434] Thus, video analytics programs can be deployed locally on the video doorbell, which helps reduce the time spent on signal interaction and improves the response speed of the video doorbell and the item retrieval system based on it; or, video analytics programs can be deployed on a cloud server or terminal device, which helps simplify the hardware structure and signal processing steps of the video doorbell and reduces its cost.

[0435] In some embodiments, the module for receiving express delivery notification messages is located locally on the video doorbell, or the module for receiving express delivery notification messages is located in the terminal device.

[0436] In this way, the received delivery notification message can be output locally on the video doorbell; or, the received delivery notification message can be transmitted to the terminal device and output by the terminal device.

[0437] In some embodiments, the terminal device includes a smartphone or a smart speaker.

[0438] In this way, smartphones can be used to notify users of the receipt of packages, or smart speakers can be used to notify people indoors of the receipt of packages.

[0439] In some embodiments, the information processing device may further include: an item care notification event determination module, configured to analyze video images according to an item tracking algorithm and determine that an item care notification event has occurred after the target item moves out of a preset area; and an item care notification message output module, configured to output an item care notification message.

[0440] In this way, based on the item tracking algorithm, intelligent care of items can be achieved, improving the intelligence and convenience of item care.

[0441] In some embodiments, the item care notification event determination module is located locally on the video doorbell, or the item care notification event determination module is located on a cloud server or terminal device.

[0442] In this way, a module for determining the time of item monitoring notification can be set locally on the video doorbell to improve the response speed of item monitoring notification events; or a module for determining the time of item monitoring notification events can be set in the cloud server or terminal device to simplify the hardware structure and software program of the video doorbell and realize the coordinated control of various terminal devices in the item collection system.

[0443] In some embodiments, the item care notification message output module is located locally on the video doorbell, or the item care notification message output module is located on a cloud server or terminal device.

[0444] In this way, the item care notification message can be output locally on the video doorbell; or, the item care notification message can be transmitted to the terminal device and output by the terminal device.

[0445] The item care system based on a video doorbell provided in this disclosure, based on the coordinated control of the video doorbell and the information processing device, can realize the intelligent collection and care of items, thereby improving the intelligence and convenience of item collection, as well as the intelligence and convenience of item care.

[0446] Based on the above embodiments, this disclosure also provides a computer storage medium storing a computer program, which, when executed by a processor, implements any of the above-described information processing methods for video doorbells, or implements any of the above-described item retrieval methods based on video doorbells.

[0447] Based on the above description of the implementation methods, those skilled in the art can clearly understand that the methods described in the embodiments of this disclosure can be implemented using software and necessary general-purpose hardware, and of course, they can also be implemented using hardware, but in many cases the former is a better implementation method. Based on this understanding, the essence of the technical solutions related to the methods of the embodiments of this disclosure, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the various methods of the embodiments of this disclosure.

[0448] Based on the above embodiments, this disclosure also provides an electronic device. For example, Figure 19 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. (Refer to...) Figure 19 The electronic device includes:

[0449] One or more processors 801, Figure 19 Take the 801 processor as an example;

[0450] Memory 802;

[0451] The electronic device may also include an input device 803 and an output device 804.

[0452] The processor 801, memory 802, input device 803, and output device 804 in this electronic device can be connected via a bus or other means. Figure 19 The connection method is illustrated by example using a bus connection.

[0453] Among them, memory 802, as a non-transitory computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as program instructions / modules corresponding to any of the above-described methods of the application program in the embodiments of this disclosure (e.g., appendix). Figure 15 The video image acquisition module 601, video notification event judgment module 602, and event notification operation triggering module 603 shown are described; or as attached Figure 17 The video image acquisition module 701, video image analysis module 702, express delivery notification message generation module 703, and express delivery notification message output module 704 are shown. The processor 801 executes various server functions and data processing by running software programs, instructions, and modules stored in the memory 802, thereby realizing the hybrid vehicle cabin heating control method of the above method embodiment.

[0454] The memory 802 may include a program storage area and a data storage area, wherein the program storage area may store the operating system and application programs required for at least one function; and the data storage area may store data created based on the use of the electronic device.

[0455] In addition, memory 802 may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device.

[0456] In some embodiments, memory 802 may optionally include memory remotely located relative to processor 801, and this remote memory may be connected to the terminal device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0457] The input device 803 can be used to receive input digital or character information, and to generate key signal inputs related to user settings and function control of the electronic device.

[0458] The output device 804 may include a display screen or other display device.

[0459] For example, the electronic device provided in this disclosure embodiment may be at least any electronic device in an intelligent service system, and this disclosure embodiment is not limited thereto.

[0460] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0461] The above are merely specific embodiments of this disclosure, enabling those skilled in the art to understand or implement this disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to these embodiments, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An image acquisition device for a video doorbell, suitable for video doorbells, characterized in that, include: Imaging lens; An image acquisition sensor is disposed on the image side of the imaging lens; A field magnifying lens is disposed on the object side of the imaging lens. The field magnifying lens is positioned below the horizontal field of view. The field magnifying lens is used to adjust the light transmission direction of the imaging lens below the horizontal field of view so that light below the horizontal field of view enters the imaging lens, while the light transmission direction of the imaging lens above the horizontal field of view remains unchanged, thereby expanding the field of view area of ​​the imaging lens below the horizontal field of view. The field magnifying lens includes a light incident surface and a light exit surface, wherein the light incident surface is a plane or an arc surface, and / or the light exit surface is a plane or an arc surface.

2. The apparatus according to claim 1, characterized in that, The field magnifying lens includes a reflective surface, which is set at an acute angle to the horizontal field of view.

3. The apparatus according to claim 2, characterized in that, The field magnifying lens is a field magnifying prism, which also includes a light incident surface and a light exit surface. After the field light is incident on the light incident surface, it is reflected by the reflective surface and then exits through the light exit surface to the imaging lens.

4. The apparatus according to any one of claims 1-3, characterized in that, The image acquisition device of the video doorbell is fixed to the door or wall.

5. An information processing method for a video doorbell, characterized in that, include: The video image is acquired by the image acquisition device of the video doorbell according to any one of claims 1-4, and the video image includes the video image after the field of view is enlarged by the field magnifying lens; Video analysis is performed based on the video images to determine whether a video notification event has occurred. When a video notification event is detected, trigger the event notification action; The video notification events include notifications of the arrival of a target person, alarms from strangers, or notifications of the care of items. The step of performing video analysis based on the video image to determine whether a video notification event has occurred includes: Face recognition is performed on video images based on face recognition algorithms and pre-set facial images of target individuals. When a face image matching the preset target person's face image is found, a target person arrival notification event is determined to occur; or, When a facial image that does not match the preset target person's facial image is matched, a stranger alarm event is determined to have occurred; The step of determining the occurrence of a target person arrival notification event when a face image matching the face image of the preset target person is found includes: When a face image that matches the face image of the preset target person is matched, motion analysis is performed based on the video image; When the motion analysis result indicates that the target person has entered the set position, a target person arrival notification event is determined to occur. The motion analysis result includes the target person moving towards the door, or the target person moving from far to near the door relative to the door, or other static or dynamic states that can determine that the target person has arrived or is about to arrive at the door. The motion analysis based on the video images includes: Based on at least one of the following factors in the video image: whether the background is shifted, the direction the child is moving, or the direction the child is facing, determine whether the child is about to enter the house. The step of triggering an event notification operation when a video notification event is determined to have occurred includes: When the video notification event is a stranger alarm event and the video doorbell is in Do Not Disturb mode, control the video doorbell not to trigger the regular visitor ringing, but to trigger the playback of preset prompt content; The step of performing video analysis based on the video image to determine whether a video notification event has occurred includes: The video image is analyzed based on the item tracking algorithm, and an item monitoring notification event is determined to have occurred after the target item moves out of the preset area. The target item is a packaged item, and the step of analyzing the video image using an item tracking algorithm includes, prior to: The video image containing the item code is analyzed based on the code recognition algorithm to obtain the express delivery information of the express item; It also includes at least one of the following: Output a message indicating successful package recognition on the video doorbell; The second terminal device outputs and receives express delivery notification messages.

6. The method according to claim 5, characterized in that, Before the video image acquisition device based on the video doorbell acquires the video image, it also includes: When a human body is detected by the human body detection sensor, the image acquisition device of the video doorbell is activated.

7. The method according to claim 5, characterized in that, The target individuals are children or visitors.

8. The method according to claim 5, characterized in that, Also includes: Obtain the facial image of a preset target person uploaded by the user through the first terminal device.

9. The method according to claim 8, characterized in that, The first terminal device includes a smartphone or a smart speaker.

10. The method according to claim 5, characterized in that, The preset area is the entire or part of the field of view of the image acquisition device.

11. The method according to claim 5, characterized in that, The express delivery information includes at least one of the following: place of shipment, sender, shipment time, or name of the express item; the second terminal device includes at least one of the following: smartphone, smart speaker, or video doorbell.

12. The method according to claim 5, characterized in that, The step of triggering an event notification operation when a video notification event is determined to occur includes: when the video notification event includes a target person arrival notification event, outputting a target person arrival prompt message through a third terminal device, wherein the third terminal device includes at least one of a smartphone, a smart speaker, or a video doorbell. When the video notification event includes a stranger alarm event, a stranger alarm message is output through a fourth terminal device, which includes at least one of a smartphone, a smart speaker, or a video doorbell. When the video notification event includes an item care notification event, an item care notification message is output through a fifth terminal device, which includes at least one of a smartphone, a smart speaker, or a video doorbell.

13. The method according to any one of claims 5-12, characterized in that, The video analysis is performed based on the video image to determine whether a video notification event has occurred in at least one of the video doorbell's local, cloud server, or sixth terminal device.

14. A method for retrieving items based on a video doorbell, characterized in that, include: In response to the detection of someone approaching the video doorbell, the video capture function and encoding recognition algorithm of the video doorbell are triggered; Acquire a video image captured by the image acquisition device of the video doorbell according to any one of claims 1-4, wherein the video image includes an item code; If the item code remains within the field of view of the image acquisition device for a preset time, the video image is analyzed according to the code recognition algorithm to obtain the express delivery information of the express delivery item corresponding to the item code. The express delivery information includes the place of shipment, the sender, the shipment time, and the name of the express delivery item. The express delivery information is obtained by calling the access interface of the corresponding logistics system based on the identification code of the express delivery item and its corresponding text information. Generate a delivery notification message based on the aforementioned delivery information; Output the received express delivery notification message; The video image is analyzed based on the item tracking algorithm, and an item monitoring notification event is determined to have occurred after the target item moves out of the preset area. Output item care notification message.

15. The method according to claim 14, characterized in that, The analysis of the video image based on the encoding recognition algorithm includes: The video image is analyzed locally on the video doorbell using an encoding recognition algorithm; or... The video image is sent to a cloud server or terminal device, which then analyzes the video image based on an encoding recognition algorithm.

16. The method according to claim 14, characterized in that, The output of the received express delivery notification message includes: The video doorbell can be used to locally output and receive delivery notification messages; or... The terminal device receives a delivery notification message from the video doorbell or cloud server and outputs the delivery notification message.

17. The method according to claim 15 or 16, characterized in that, The terminal device includes a smartphone or a smart speaker.

18. The method according to claim 14, characterized in that, The preset area is the entire or part of the field of view of the image acquisition device.

19. The method according to claim 14, characterized in that, The analysis of the video image based on the item tracking algorithm includes: The video images are analyzed locally on the video doorbell using an item tracking algorithm; or... The video image is sent to a cloud server or terminal device, which then analyzes the video image based on an item tracking algorithm.

20. The method according to claim 19, characterized in that, The output item care notification message includes: Output a notification message about item monitoring locally via the video doorbell; or... The terminal device receives item monitoring notification messages from the video doorbell or cloud server and outputs item monitoring notification messages.

21. An information processing device for a video doorbell, characterized in that, include: The video image acquisition module is used to acquire video images captured by the image acquisition device. The image acquisition device is the image acquisition device of the video doorbell according to any one of claims 1-4, and the video image includes the video image after the field of view is enlarged by the field magnifying lens; The video notification event determination module is used to perform video analysis based on the video image to determine whether a video notification event has occurred. The event notification operation triggering module is used to trigger an event notification operation when a video notification event is determined to have occurred. The video notification events include notifications of the arrival of a target person, alarms from strangers, or notifications of the care of items. The video notification event judgment module includes: The face recognition submodule is used to perform face recognition on video images based on face recognition algorithms and the face image of a preset target person; The first event determination submodule is used to determine that a target person arrival notification event has occurred when a face image matching the face image of the preset target person is found; or, The second event determination submodule is used to determine that a stranger alarm event has occurred when a face image that does not match the face image of the preset target person is found. The first event determination submodule includes: The motion analysis unit is used to perform motion analysis based on the video image when a face image that matches the face image of the preset target person is found. The event determination unit is used to determine that a target person arrival notification event has occurred when the motion analysis result indicates that the target person has entered the set position. The motion analysis result includes the target person moving towards the door, or the target person moving from far to near the door relative to the door, or other static or dynamic states that can determine that the target person has arrived or is about to arrive at the door. The motion analysis based on the video images includes: Based on at least one of the following factors in the video image: whether the background is shifted, the direction the child is moving, or the direction the child is facing, determine whether the child is about to enter the house. The event notification operation triggering module is used for: When the video notification event is a stranger alarm event and the video doorbell is in Do Not Disturb mode, control the video doorbell not to trigger the regular visitor ringing, but to trigger the playback of preset prompt content; The video notification event judgment module is used for: The video image is analyzed based on the item tracking algorithm, and an item monitoring notification event is determined to have occurred after the target item moves out of the preset area. The target item is a packaged item, and the step of analyzing the video image using an item tracking algorithm includes, prior to: The video image containing the item code is analyzed based on the code recognition algorithm to obtain the express delivery information of the express item; It also includes at least one of the following: Output a message indicating successful package recognition on the video doorbell; The second terminal device outputs and receives express delivery notification messages.

22. The apparatus according to claim 21, characterized in that, Also includes: The image acquisition device activation module is used to activate the image acquisition device of the video doorbell when a human body is detected by the human body detection sensor.

23. A device for collecting items based on a video doorbell, characterized in that, include: The trigger module is used to trigger the video capture function and encoding recognition algorithm of the video doorbell in response to the detection of someone approaching the video doorbell; A video image acquisition module is used to acquire video images captured by the image acquisition device of the video doorbell according to any one of claims 1-4, wherein the video images include item codes; The video image analysis module is used to analyze the video image according to the code recognition algorithm if the item code stays in the field of view of the image acquisition device for a preset time, so as to obtain the express delivery information of the express item corresponding to the item code. The express delivery information includes the place of shipment, the sender, the shipment time and the name of the express item. The express delivery information is obtained by calling the access interface of the corresponding logistics system based on the identification code of the express item and its corresponding text information. The express delivery notification message generation module is used to generate an express delivery notification message based on the express delivery information. The express delivery notification message output module is used to output the express delivery notification message. The item care notification event determination module is used to analyze the video image based on the item tracking algorithm and determine that an item care notification event has occurred after the target item moves out of the preset area. The item care notification message output module is used to output item care notification messages.

24. A video doorbell system, characterized in that, The video doorbell includes the image acquisition device according to any one of claims 1-4, and the information processing device; The image acquisition device of the video doorbell is used to acquire video images, including video images after the field of view is enlarged by a magnifying glass. The information processing device is used to perform video analysis based on the video image to determine whether a video notification event has occurred. And when a video notification event is detected, an event notification operation is triggered; The first terminal device is used to upload facial images of a preset target person; The information processing device is also used to perform face recognition on the video image based on the face recognition algorithm and the face image of the preset target person, and determine that a target person arrival notification event has occurred when a face image that matches the face image of the preset target person is matched. Alternatively, when a face image that does not match the face image of the preset target person is matched, a stranger alarm event is determined to have occurred. When a face image that matches the face image of the preset target person is matched, motion analysis is performed based on the video image; When the motion analysis result indicates that the target person has entered the set position, a target person arrival notification event is determined to occur. The motion analysis result includes the target person moving towards the door, or the target person moving from far to near the door relative to the door, or other static or dynamic states that can determine that the target person has arrived or is about to arrive at the door. The motion analysis based on the video images includes: Based on at least one of the following factors in the video image: whether the background has shifted, the child's direction of movement, or the child's orientation, determine whether the child intends to enter the house. The step of triggering an event notification operation when a video notification event is determined to have occurred includes: When the video notification event is a stranger alarm event and the video doorbell is in Do Not Disturb mode, control the video doorbell not to trigger the regular visitor ringing, but to trigger the playback of preset prompt content; The system also includes a video doorbell and / or a second terminal device; The information processing device is also used to analyze video images including item codes based on an encoding recognition algorithm to obtain express delivery information of the express items; and to analyze the video images according to an item tracking algorithm to determine that an item monitoring notification event has occurred after the express items have moved out of a preset area; The video doorbell is used to output a notification message indicating successful package recognition. The second terminal device is used to output a message indicating receipt of a delivery notification.

25. The system according to claim 24, characterized in that, Also includes: A human body detection sensor is used to activate the image acquisition device of the video doorbell when a human body is detected.

26. The system according to claim 24, characterized in that, The video notification events include notifications of the arrival of a target person, alerts to strangers, or notifications of the care of items.

27. The system according to claim 24, characterized in that, The first terminal device includes a smartphone or a smart speaker.

28. The system according to claim 24, characterized in that, The second terminal device includes at least one of a smartphone, a smart speaker, or a video doorbell.

29. The system according to claim 26, characterized in that, Also includes: The third terminal device is used to output a target person arrival prompt message when the video notification event includes a target person arrival notification event; The third terminal device includes at least one of a smartphone, a smart speaker, or a video doorbell; A fourth terminal device is configured to output a stranger alarm message when the video notification event includes a stranger alarm event; the fourth terminal device includes at least one of a smartphone, a smart speaker, or a video doorbell. A fifth terminal device is configured to output an item care notification message when the video notification event includes an item care notification event; the fifth terminal device includes at least one of a smartphone, a smart speaker, or a video doorbell.

30. The system according to any one of claims 24-29, characterized in that, The information processing device is located in at least one of the following: local video doorbell, cloud server, or sixth terminal device.

31. A product retrieval system based on a video doorbell, characterized in that, The video doorbell includes an image acquisition device and an information processing device, wherein the information processing device includes: The trigger module is used to trigger the video capture function and encoding recognition algorithm of the video doorbell in response to the detection of someone approaching the video doorbell; A video image acquisition module is used to acquire video images captured by the image acquisition device of the video doorbell according to any one of claims 1-4, wherein the video images include item codes; The video image analysis module is used to analyze the video image according to the code recognition algorithm if the item code stays in the field of view of the image acquisition device for a preset time, so as to obtain the express delivery information of the express item corresponding to the item code. The express delivery information includes the place of shipment, the sender, the shipment time and the name of the express item. The express delivery information is obtained by calling the access interface of the corresponding logistics system based on the identification code of the express item and its corresponding text information. The express delivery notification message generation module is used to generate an express delivery notification message based on the express delivery information. The express delivery notification message output module is used to output the express delivery notification message. The item care notification event determination module is used to analyze the video image based on the item tracking algorithm and determine that an item care notification event has occurred after the target item moves out of the preset area. The item care notification message output module is used to output item care notification messages.

32. The system according to claim 31, characterized in that, The video image analysis module is set locally on the video doorbell, or... The video image analysis module is located on a cloud server or terminal device.

33. The system according to claim 31, characterized in that, The module for receiving and outputting express delivery notification messages is set locally on the video doorbell, or... The module for receiving express delivery notification messages is set in a cloud server or terminal device.

34. The system according to claim 32 or 33, characterized in that, The terminal device includes a smartphone or a smart speaker.

35. The system according to claim 31, characterized in that, The item monitoring notification event determination module is set locally on the video doorbell, or... The item care notification event determination module is set in a cloud server or terminal device.

36. The system according to claim 31, characterized in that, The item monitoring notification message output module is set locally on the video doorbell, or... The item care notification message output module is set in a cloud server or terminal device.

37. An electronic device, characterized in that, The electronic device includes: processor; Memory used to store the processor's executable instructions; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the information processing method of the video doorbell as described in any one of claims 5-13, or the item retrieval method based on the video doorbell as described in any one of claims 14-20.

38. A computer storage medium, characterized in that, It stores a computer program, which, when executed by a processor, implements the information processing method of the video doorbell as described in any one of claims 5-13, or the item collection method based on the video doorbell as described in any one of claims 14-20.

Images