Monitoring method and device of intelligent door lock, computer equipment and storage medium

Abstract

The invention relates to the technical field of intelligent locks, and discloses a monitoring method and device of an intelligent door lock, computer equipment and a storage medium, the method comprises the following steps: obtaining an original image collected by an image collection device on the intelligent door lock, the original image comprising a reflection image in a reflection device on the intelligent door lock and a natural image outside the reflection device; identifying a reflection area in the original image, and rotating a reflection image in the reflection area to obtain a rotated reflection image; splicing the rotated reflection image with a natural image outside the reflection area to obtain a spliced image; and detecting whether a target object exists in the spliced image, and if the target object exists, executing a corresponding reminding operation. According to the intelligent door lock, the problem that a blind area exists when an existing intelligent door lock monitors a low area of a doorway is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of smart locks, and in particular to a monitoring method, device, computer equipment and storage medium for smart door locks. Background Art

[0002] The popularity of smart home technology has made smart door locks and doorbell products a common choice for modern families due to their convenience, safety and intelligence. These products not only enhance home security, but also provide remote monitoring and visitor identification functions through built-in cameras. However, since these devices are often installed on doors and perpendicular to the ground, they have blind spots in monitoring low areas at the door. When couriers place packages low at the door and users are not at home, these packages may be out of monitoring, increasing security risks.

[0003] However, there are two main solutions for existing door locks or doorbells to monitor doorstep delivery, but both have obvious shortcomings: Although dual cameras solve some blind spot problems by adding cameras, the system is highly complex, the failure rate increases, and the dual camera image stitching is not perfect, affecting the user experience. Although the wide-angle fisheye lens expands the monitoring range, the edge distortion is serious and difficult to correct. It may still form a monitoring blind spot for small packages or packages placed at a specific angle. At the same time, it requires high-performance hardware support, which increases cost and complexity. Summary of the invention

[0004] In view of this, the embodiments of the present invention provide a monitoring method, device, computer equipment and storage medium for a smart door lock to solve the problem of blind spots that exist in existing smart door locks when monitoring low areas at doorways.

[0005] In a first aspect, an embodiment of the present invention provides a monitoring method for a smart door lock, the method comprising:

[0006] Acquire an original image captured by an image acquisition device on a smart door lock, wherein the original image includes a reflected image in a reflection device on the smart door lock and a natural image outside the reflection device;

[0007] Identifying a reflection area in the original image, and rotating the reflection image in the reflection area to obtain a rotated reflection image;

[0008] splicing the rotated reflected image with the natural image outside the reflection area to obtain a spliced ​​image;

[0009] It is detected whether a target object exists in the stitched image, and if the target object exists, a corresponding reminder operation is performed.

[0010] Furthermore, the identifying the reflection area in the original image and rotating the reflection image in the reflection area to obtain the rotated reflection image includes:

[0011] Identifying edge information and texture features in the original image;

[0012] Determining a reflection area in the original image according to the edge information and the texture feature;

[0013] The reflected image in the reflective area is rotated to obtain a rotated reflected image.

[0014] Further, determining the reflection area in the original image according to the edge information and the texture feature includes:

[0015] Determine a candidate edge line of the reflection area according to the edge information;

[0016] Acquire texture features within a preset range of the candidate edge line, and compare the texture features with preset texture features to obtain a comparison result;

[0017] Determine the target pixel point belonging to the reflection area based on the comparison result;

[0018] Adjacent target pixel points are connected to obtain a reflection area in the original image.

[0019] Furthermore, rotating the reflected image in the reflective area to obtain a rotated reflected image includes:

[0020] identifying directional features in the reflected image;

[0021] Matching the directional feature with preset features in a plurality of preset directional templates to obtain a hit target directional template;

[0022] A rotation angle in the target direction template is obtained, and a rotation operation is performed on the reflected image according to the rotation angle to obtain a rotated reflected image.

[0023] Furthermore, the step of splicing the rotated reflected image with the natural image outside the reflection area to obtain a spliced ​​image includes:

[0024] Detecting position information of the rotated reflected image in the original image;

[0025] Cropping the original image based on the position information to obtain a cropped natural image;

[0026] Analyze the rotated reflected image and the cropped natural image to obtain a splicing position;

[0027] According to the stitching position, the rotated reflected image and the cropped natural image are fused to obtain a stitching image.

[0028] Furthermore, before detecting whether a target object exists in the stitched image, the method further includes:

[0029] Detecting whether there is a candidate object in the natural image outside the reflection area;

[0030] If there is a candidate object in the non-edge area of ​​the natural image, a reminder operation is directly performed; if there is a candidate object in the edge area of ​​the natural image, or there is no candidate object in the natural image, a detection operation is performed on the spliced ​​image.

[0031] Furthermore, the detecting whether a target object exists in the stitched image includes:

[0032] Obtain multiple standard features in a preset object feature library;

[0033] Extracting key features from the stitched image, and calculating the matching degree between the key features and the standard features;

[0034] If the matching degree exceeds a preset threshold, it is determined that the target object exists in the stitched image, or, if the matching degree does not exceed the preset threshold, it is determined that the target object does not exist in the stitched image.

[0035] Furthermore, the method further comprises:

[0036] Detecting whether key features in the stitched image are missing;

[0037] If there is a missing situation, analyzing the position distribution of the key features with the missing situation in the stitched image to obtain the area to be supplemented in the stitched image;

[0038] Determining an adjustment direction of the reflecting device according to the area to be supplemented, and controlling the reflecting device to perform a moving operation according to the adjustment direction;

[0039] After the movement operation is completed, the image acquisition device on the smart door lock is restarted to perform the acquisition operation.

[0040] In a second aspect, an embodiment of the present invention provides a monitoring system for a smart door lock, the system comprising: an image acquisition device, a processing device, and a reflection device;

[0041] The image acquisition device is used to acquire an original image within the monitoring range of the smart door lock, wherein the original image includes a reflected image within the reflection device and a natural image outside the reflection device;

[0042] The processing device is used to stitch the reflected image and the natural image in the original image to obtain a stitched image, determine the adjustment angle of the reflecting device according to the stitched image, and send the adjustment angle to the reflecting device;

[0043] The reflecting device is used to receive the adjustment angle sent by the processing device and perform the moving operation according to the adjustment angle.

[0044] In a third aspect, an embodiment of the present invention provides a monitoring device for a smart door lock, the device comprising:

[0045] An acquisition module, used to acquire an original image acquired by an image acquisition device on a smart door lock, wherein the original image includes a reflected image in a reflection device on the smart door lock and a natural image outside the reflection device;

[0046] A rotation module, used for identifying a reflection area in the original image, and rotating the reflection image in the reflection area to obtain a rotated reflection image;

[0047] A stitching module, used for stitching the rotated reflected image with the natural image outside the reflection area to obtain a stitched image;

[0048] The execution module is used to identify whether there is a target object in the spliced ​​image, and if the target object exists, execute a corresponding reminder operation.

[0049] In a fourth aspect, an embodiment of the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, computer instructions being stored in the memory, and the processor executing the method of the above-mentioned first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0050] In a fifth aspect, an embodiment of the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the method of the first aspect or any corresponding embodiment thereof.

[0051] The method provided in the embodiment of the present application has the following beneficial effects:

[0052] The method provided in the embodiment of the present application provides a rich data source for subsequent image processing and monitoring blind spot compensation by simultaneously collecting the reflected image inside the reflective device and the natural image outside the device. The image in the reflection area is accurately identified and rotated, so that the perspective distortion originally caused by the installation angle of the equipment is corrected, laying the foundation for subsequent image stitching. The rotated reflected image is spliced ​​with the natural image outside the reflection area to achieve full coverage of the low area at the door, eliminate the blind spots in the traditional monitoring method, and improve the integrity and reliability of the monitoring. Through intelligent recognition technology, it is possible to quickly and accurately detect packages or other important objects placed at the door, provide users with timely reminders, and enhance home safety.

[0053] The method provided in the embodiment of the present application utilizes image analysis technology to improve the accuracy and efficiency of reflection area identification, and provides a reliable foundation for subsequent image processing steps. By matching the preset direction template, automatic rotation correction of the reflection image is achieved, making the spliced ​​image more natural and distortion-free. The position information of the reflected image after rotation is detected and cropped and spliced, so as to achieve accurate splicing of the reflected image and the natural image after rotation, and improve the visual quality of the final monitoring image. Through efficient feature extraction and matching algorithms, the target object can be identified quickly and accurately, and the response speed and accuracy of the monitoring system are improved. The monitoring blind area in the spliced ​​image is intelligently analyzed, and the monitoring blind area is further compensated by dynamically adjusting the position of the reflective device to ensure comprehensive monitoring without blind spots. By re-capturing the image, the real-time and continuity of the monitoring system are ensured, providing users with continuous monitoring guarantees. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0055] Figure 1 is a flow chart of a monitoring method for a smart door lock according to an embodiment of the present invention;

[0056] Figure 2 is a schematic diagram of a method for installing a camera lens of a smart door lock according to an embodiment of the present invention;

[0057] Figure 3 is a schematic diagram of the viewing angle and monitoring blind area of ​​a smart door lock camera according to an embodiment of the present invention;

[0058] Figure 4 is a schematic diagram of a viewing angle after a reflective device is installed in a smart door lock according to an embodiment of the present invention;

[0059] Figure 5 is a schematic diagram of the rotation of the reflected image of the smart door lock according to an embodiment of the present invention;

[0060] Figure 6 is a structural block diagram of a monitoring device for a smart door lock according to an embodiment of the present invention;

[0061] Figure 7 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0063] According to an embodiment of the present invention, a monitoring method, apparatus, computer device and storage medium for a smart door lock are provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in an order different from that shown here.

[0064] In this embodiment, a monitoring method for a smart door lock is provided. Figure 1 is a flow chart of a monitoring method for a smart door lock according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0065] Step S11, obtaining an original image captured by an image acquisition device on the smart door lock, wherein the original image includes a reflected image inside a reflection device on the smart door lock and a natural image outside the reflection device.

[0066] In the embodiment of the present application, the image acquisition device may refer to a high-definition camera embedded in the smart door lock panel. This camera is used to capture image information in front of the door and then convert this information into digital signals for subsequent image processing and analysis. Figure 2 As shown, the camera lens installation method can include but is not limited to two methods: one is to embed the lens under the smart lock panel, and the panel is made of glass or other high-light-transmittance materials; the other is to open a hole in the panel so that the camera lens is placed exactly in the hole and is almost flush with the panel surface.

[0067] It should be noted that in order to expand the monitoring range, the camera often uses a fisheye lens with a large field of view, but when it is installed flush with the smart lock panel, Figure 3 As shown in the figure, a monitoring blind spot will be formed at angle 1, and the field of view angle is angle 2. A large field of view angle can easily cause image edge distortion, increasing processing and correction costs. Traditional wide-angle design will waste the field of view on the ceiling, which is not an effective monitoring area. To solve these problems, Figure 4 As shown, a reflective device such as a plane mirror can be installed directly above the camera lens, extending the eaves structure out of the lock panel plane, and by adjusting the extension length and angle, the lens field of view can be adjusted from angle 2 to angle 3 using the field of view of angle 4 to make up for the blind spot in front of the door. The reflective device supports dynamic adjustment and can be flexibly adjusted through the smart lock control system to achieve the best monitoring effect.

[0068] Step S12, identifying the reflection area in the original image, and rotating the reflection image in the reflection area to obtain a rotated reflection image.

[0069] In the embodiment of the present application, step S12 specifically includes the following steps A1-A3:

[0070] Step A1, identifying edge information and texture features in the original image.

[0071] Specifically, edge information can be identified through edge detection algorithms such as Sobel operator, Canny algorithm, etc. Taking the Canny algorithm as an example, the original image is first subjected to Gaussian filtering for denoising, and then the gradient is calculated using the first-order partial derivative finite difference to obtain the amplitude and direction of each pixel point, followed by non-maximum suppression, retaining the local gradient maximum point, removing non-maximum points, and finally using dual threshold detection to determine the edge, setting high and low thresholds, and points greater than the high threshold are strong edge points, points less than the low threshold are non-edge points, points between the two and connected to strong edge points are edge points, otherwise they are non-edge points.

[0072] Specifically, texture features can be identified by the following methods: Gray-Level Co-occurrence Matrix (GLCM), which calculates the frequency of the combination of gray values ​​of two pixels at a certain distance and direction, and extracts energy, contrast, correlation and other features to describe the texture; Local Binary Patterns (LBP), which compares the central pixel with the surrounding neighborhood pixels to generate a binary code, and counts the frequencies of different codes as texture features; Wavelet transform, which decomposes the image by wavelet to obtain sub-images of different scales and directions, and extracts texture features from them.

[0073] Step A2: determining the reflection area in the original image according to the edge information and texture features.

[0074] In the embodiment of the present application, step A2 includes the following steps A21-A24:

[0075] Step A21, determining the candidate edge lines of the reflection area according to the edge information.

[0076] Specifically, the shape, length, direction and other features of the detected edge information are analyzed. The edge of the reflective area usually has a regular shape or has a certain continuity in a specific direction. At the same time, the edge strength, that is, the gradient amplitude of the edge pixel point, is considered. The stronger the edge, the more likely it is to belong to the boundary of the reflective area. The edge lines that may belong to the reflective area are screened out based on the edge features. Threshold conditions such as edge length greater than a certain value and edge direction within a specific range can be set. The edge lines that meet the conditions are marked as candidate edge lines. These candidate edge lines may be part of the boundary of the reflective area and need further verification and processing.

[0077] Step A22, obtaining texture features within a preset range of the candidate edge line, and comparing the texture features with preset texture features to obtain a comparison result.

[0078] Specifically, first determine the preset range centered on the candidate edge line, which can be determined based on experience or analysis of a large number of image samples, such as setting a rectangular or circular area of ​​a specific size around the candidate edge line, in order to obtain relevant texture information of the reflection area without introducing too many irrelevant areas. Then, use texture feature extraction methods (such as gray level co-occurrence matrix, local binary pattern, wavelet transform, etc.) to obtain texture features within the range, including energy, contrast, correlation and other features of the gray level co-occurrence matrix and binary code frequency distribution features of the local binary pattern. Then, compare the obtained texture features with the preset texture features obtained by analyzing the known reflection area image, and use distance measurement methods such as Euclidean distance and cosine similarity to determine the similarity between the texture within the range and the texture of the typical reflection area, so as to provide a basis for determining the reflection area.

[0079] Step A23, determining the target pixel points belonging to the reflection area based on the comparison result.

[0080] Specifically, if the comparison result shows that the texture features within the preset range of the candidate edge line are similar to the preset texture features, it can be considered that the pixels within this range may belong to the reflection area. A similarity threshold can be set, and when the comparison result exceeds this threshold, the pixels within the range are determined as target pixels belonging to the reflection area. For example, if the Euclidean distance is used as the comparison method, a distance threshold can be set, and when the distance is less than this threshold, the corresponding pixel is determined as the target pixel.

[0081] Step A24, connecting adjacent target pixel points to obtain the reflection area in the original image.

[0082] Specifically, after determining the target pixel points belonging to the reflection area, the image connected region analysis method (such as four-neighborhood connection or eight-neighborhood connection) can be used to start from a target pixel point and gradually expand outward to connect adjacent target pixel points to form a continuous area. This area is the reflection area in the original image. The final reflection area can be used for subsequent processing such as rotating the reflection image to correct the image deformation problem caused by plane mirror reflection and improve the image quality.

[0083] Step A3: rotating the reflected image in the reflective area to obtain a rotated reflected image.

[0084] In the embodiment of the present application, step A3 includes the following steps A31-A33:

[0085] Step A31, identifying directional features in the reflected image.

[0086] Specifically, directional features can be identified by analyzing the shape of the object, the direction of the texture, the direction of the edge, and other information in the reflected image. For example, if there is an obvious straight object in the image, the direction can be determined based on its tilt angle; if the texture shows a specific directionality, it can also be used as a basis for directional features. The gradient information of the image, that is, the rate of change of the pixel value in different directions, can also be used to determine the main direction of the image. The direction with a larger gradient usually represents the important structural direction in the image.

[0087] Step A32, matching the directional feature with preset features in a plurality of preset directional templates to obtain a hit target directional template.

[0088] Specifically, multiple directional templates are established in advance, and each template contains a specific directional feature description. These templates can be obtained by analyzing and summarizing a large number of reflection images with known directions. The directional features identified from the reflection image are compared with the preset features of each preset directional template, and a similarity measurement method can be used, such as calculating the distance between feature vectors. If the similarity exceeds a certain threshold, it is considered that the directional features of the reflection image match this preset directional template. Finally, the target directional template that best matches the directional features of the reflection image is determined, and this template will provide a directional reference for subsequent rotation operations.

[0089] Step A33, obtaining the rotation angle in the target direction template, and performing a rotation operation on the reflected image according to the rotation angle to obtain a rotated reflected image.

[0090] Specifically, after determining the target direction template, it is necessary to obtain the corresponding rotation angle from the template. This rotation angle is determined based on the difference between the direction represented by the template and the expected standard direction. According to the obtained rotation angle, the reflected image is rotated using an image rotation algorithm. Common image rotation algorithms can be implemented through matrix transformation to ensure that the image maintains pixel integrity and clarity during the rotation process. After the rotation operation, a rotated reflected image is obtained, which corrects the directional problems such as upper and lower mirror images caused by plane mirror reflection, making the image more consistent with the direction of the actual scene and improving the quality and usability of the image.

[0091] As an example, Figure 5 As shown in the figure, due to the physical characteristics of plane mirror reflection, the object in the blind spot in front of the door is captured by the camera through the reflection device, forming the original picture on the left, the upper part is the image in the reflection device, and the lower part is the normal image. Due to the mirror principle, the upper image is a mirror image up and down. First, identify the directional features of the reflected image, which can be judged according to the shape and texture of the object. Then match the preset features of the contour feature templates of multiple objects at different angles to obtain the target direction template. Finally, rotate the reflected image according to the rotation angle of the target template to obtain the rotated image on the right, correct the upper and lower mirror images, and make the image more consistent with the actual scene direction.

[0092] Step S13, stitching the rotated reflected image with the natural image outside the reflection area to obtain a stitched image.

[0093] In the embodiment of the present application, step S13 specifically includes the following steps B1-B4:

[0094] Step B1, detecting the position information of the rotated reflected image in the original image.

[0095] Specifically, the position of the rotated reflected image in the original image is determined by analyzing the pixel coordinates of the image. For example, each pixel point of the original image is traversed, and when a pixel area matching the features of the rotated reflected image is found, the coordinate position of the area is recorded. An image recognition algorithm can also be used to identify specific feature points or markers in the reflected image, and then the position of the reflected image is determined based on the coordinates of these feature points in the original image.

[0096] Step B2, cropping the original image based on the position information to obtain a cropped natural image.

[0097] Specifically, according to the determined position information of the rotated reflected image, the area to be cropped is determined in the original image. This area is usually the part except the position of the reflected image, that is, the area of ​​the natural image. The original image is cropped according to the determined cropping area using an image cropping tool or algorithm to obtain a cropped natural image. The cropped natural image does not contain the part of the reflected image, and is ready for the subsequent stitching operation.

[0098] Step B3, analyzing the rotated reflected image and the cropped natural image to obtain a splicing position.

[0099] Specifically, the specific process of analyzing the reflected image and the natural image may include: first, determining the specific object in the reflected image and the corresponding object in the natural image, or finding an area with similar texture. Secondly, for the edges of the reflected image and the natural image, determine whether they are relatively smooth to avoid obvious jagged or discontinuous edges. Subsequently, analyze the color distribution of the two images, including the mean, variance, color saturation, etc. of the color, find the area with natural color transition, and avoid obvious color differences at the splicing position. Then, use image feature matching algorithms such as Scale-Invariant Feature Transform (SIFT) and Speeded Up Robust Features (SURF) to query similar feature point pairs in the reflected image and the natural image. Finally, determine the best splicing position based on the positional relationship of the feature points and factors such as the image content, edge features, and color distribution analyzed previously. Usually, the area with smooth edges, natural color transition, and good feature point matching is selected for splicing.

[0100] Step B4: according to the stitching position, the rotated reflected image and the cropped natural image are merged to obtain a stitched image.

[0101] Specifically, the rotated reflected image and the cropped natural image are fused according to the stitching position. Linear blending, weighted average and other image blending algorithms can be used to make the transition at the stitching natural. The fusion process includes: first, according to the image features around the stitching position (such as contrast, brightness, texture complexity, etc.), the weight coefficient is determined and dynamically adjusted to achieve a gradual transition from the reflected image weight to the natural image weight near the stitching position to achieve a smoothing effect; second, a pixel-level fusion operation is performed, and the color values ​​of the corresponding pixels of the reflected image and the natural image are weighted and summed according to the weight coefficient to obtain the fused pixel color value; finally, the fused image is post-processed to check whether there are obvious seams or unnatural transitions in the fusion area. If so, the defects are eliminated by adjusting the weight coefficient, local smoothing or image restoration technology to make the stitched image more natural and complete.

[0102] Step S14, detecting whether there is a target object in the stitched image, and if there is a target object, executing a corresponding reminder operation.

[0103] In the embodiment of the present application, after a series of image processing operations are performed to obtain a spliced ​​image, it is detected whether there is a specific target object in the image, so as to perform a corresponding reminder operation when necessary. For example, when it is recognized that there are express delivery, take-out and other items in front of the door, the user can be reminded to collect them in time by pushing messages on the mobile phone, making the door lock emit a reminder sound, etc. This function can provide convenience for users and improve the intelligence of life.

[0104] In an embodiment of the present application, before detecting whether the target object exists in the stitched image, the method further includes: detecting whether there is a candidate object in the natural image outside the reflection area; if there is a candidate object in the non-edge area of ​​the natural image, directly performing a reminder operation; if there is a candidate object in the edge area of ​​the natural image, or if there is no candidate object in the natural image, performing a detection operation on the stitched image.

[0105] Specifically, before officially starting to detect the target object in the stitched image, first detect whether there are candidate objects in the natural image outside the reflection area, with the goal of quickly identifying objects that can be confirmed without complex stitched image analysis. The detection process involves detecting non-edge areas and edge areas of natural images. Non-edge areas usually contain the main content of the image, while edge areas may contain image distortion, artifacts, or incomplete information introduced by operations such as image acquisition, cropping, or stitching.

[0106] If candidate objects are detected in the non-edge area of ​​the natural image, the reminder operation can be performed directly. This is because these objects are clear enough in the original image, and their features and shapes are obvious enough, so there is no need to further confirm them through stitching images.

[0107] If the candidate object appears in the edge area of ​​the natural image, since the objects in these areas may be incomplete or have changed shape due to image distortion, a more comprehensive analysis is needed to confirm their identity. At the same time, if there is no candidate object in the natural image, the reflection area in the stitched image may become the only source of useful information, so further detection operations need to be performed on the stitched image.

[0108] Through this pre-detection step, the system can process image data more efficiently, reduce unnecessary calculations and analysis, and thus improve the accuracy and speed of target object detection.

[0109] In the embodiment of the present application, detecting whether there is a target object in the stitched image includes the following steps C1-C3:

[0110] Step C1, obtaining multiple standard features in a preset object feature library.

[0111] Specifically, the preset object feature library is a pre-established database that stores standard feature information of various target objects. These features can be obtained by analyzing and extracting images of a large number of target objects. For example, for express parcels, its shape, color, texture and other features can be extracted as standard features. By obtaining these standard features, a reference basis is provided for subsequent comparison with the features in the spliced ​​image.

[0112] Step C2, extracting key features from the spliced ​​image and calculating the matching degree between the key features and the standard features.

[0113] Specifically, first, extract key features from the stitched image. Image feature extraction algorithms such as SIFT and SURF can be used to extract key points, edges, textures and other features in the image. Then, compare these key features with the standard features in the preset object feature library to calculate the matching degree between them. The matching degree can be calculated in many ways, such as calculating the distance and similarity between feature vectors.

[0114] Step C3: if the matching degree exceeds the preset threshold, it is determined that the target object exists in the stitched image; or, if the matching degree does not exceed the preset threshold, it is determined that the target object does not exist in the stitched image.

[0115] Specifically, the calculated matching degree is compared with a preset threshold. If the matching degree exceeds the threshold, it means that the features in the stitched image are similar to the standard features of the target object, and it can be determined that the target object exists in the stitched image. On the contrary, if the matching degree does not exceed the threshold, it is considered that the target object does not exist in the stitched image. The setting of the threshold can be adjusted according to the actual situation.

[0116] In the embodiment of the present application, the method further includes the following steps D1-D4:

[0117] Step D1, detecting whether there are any missing key features in the stitched image.

[0118] Specifically, after obtaining the stitched image, check whether the key features are complete. The key features cover the shape, color, texture of the target object and important structural elements of the scene. The process of detecting missing conditions is as follows: first, establish a standard model of key features, such as determining the shape outline, color range, texture characteristics, etc. for express parcels; second, compare the stitched image features with the standard model, analyze the extracted features of each area and match them. If the difference is large or cannot be matched, there may be missing features; finally, comprehensively consider factors such as image quality, contrast, brightness, etc. to make judgments, and further analyze and verify whether the blurred or uncertain areas are really missing.

[0119] Step D2: if there is a missing situation, the position distribution of the key features with the missing situation in the stitched image is analyzed to obtain the area to be supplemented in the stitched image.

[0120] Specifically, if a key feature is detected to be missing, it is necessary to determine the specific location of the missing part in the stitched image. The area to be supplemented can be determined by analyzing the type of missing features and the possible location pattern. For example, if one side of the target object is missing, the area to be supplemented is around the corresponding position of the object. Determining the area to be supplemented helps to adjust the reflective device in a targeted manner to obtain more complete image information.

[0121] Step D3, determining the adjustment direction of the reflection device according to the area to be supplemented, and controlling the reflection device to perform a moving operation according to the adjustment direction.

[0122] Specifically, the adjustment direction of the reflective device is determined according to the location of the area to be supplemented. For example, if the area to be supplemented is above the image, the reflective device needs to be adjusted to move upward or adjust the angle so that it can better capture the image information of the area. Through the smart lock control system, the reflective device can be accurately controlled to perform movement operations in the determined adjustment direction to improve the image acquisition effect and obtain more complete key features.

[0123] Step D4, after the moving operation is completed, restart the image acquisition device on the smart door lock to perform the acquisition operation.

[0124] Specifically, when the moving operation of the reflective device is completed, the image acquisition device is restarted to acquire images. In this way, a new image adjusted by the reflective device can be acquired to ensure that the missing key features can be supplemented and the integrity and accuracy of the image can be improved. The newly acquired image can be processed and analyzed again through the previous steps to meet the needs of target object detection and scene monitoring.

[0125] This embodiment provides a monitoring system for a smart door lock, the system comprising: an image acquisition device, a processing device, and a reflection device;

[0126] An image acquisition device, used to acquire an original image within the monitoring range of the smart door lock, wherein the original image includes a reflected image within the reflection device and a natural image outside the reflection device;

[0127] A processing device, used for splicing the reflected image and the natural image in the original image to obtain a spliced ​​image, and determining an adjustment angle of the reflecting device according to the spliced ​​image, and sending the adjustment angle to the reflecting device;

[0128] The reflecting device is used to receive the adjustment angle sent by the processing device and perform the moving operation according to the adjustment angle.

[0129] Specifically, the smart door lock monitoring system of this embodiment, on the basis of acquiring and processing the original image to generate the spliced ​​image, pays special attention to the function of automatically adjusting the reflective device according to the monitoring needs. The system first captures the original data containing the reflected image and the natural image through the image acquisition device, and then uses the processing device to identify, rotate and splice the image. Importantly, after generating the spliced ​​image, the system will further analyze the key features in the image to detect whether there are any missing features. Once the feature is found to be missing, the system will intelligently calculate the adjustment direction of the reflective device according to the position of the missing part in the spliced ​​image, and send instructions to the reflective device to move it at a specified angle to optimize the monitoring field of view. This adjustment mechanism ensures that the smart door lock monitoring system can continuously capture complete and blind-spot monitoring images, thereby greatly improving the accuracy and reliability of monitoring.

[0130] In this embodiment, a monitoring device for a smart door lock is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and will not be repeated hereafter. As used below, the term "module" may be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0131] This embodiment provides a monitoring device for a smart door lock, such as Figure 6 As shown, including:

[0132] An acquisition module 61 is used to acquire an original image acquired by an image acquisition device on a smart door lock, wherein the original image includes a reflected image inside a reflection device on the smart door lock and a natural image outside the reflection device;

[0133] A rotation module 62, used for identifying a reflection area in the original image, and rotating the reflection image in the reflection area to obtain a rotated reflection image;

[0134] A stitching module 63, used for stitching the rotated reflected image with the natural image outside the reflection area to obtain a stitched image;

[0135] The execution module 64 is used to detect whether there is a target object in the stitched image, and if there is a target object, execute a corresponding reminder operation.

[0136] In an optional embodiment of the present application, the rotation module 62 is used to identify edge information and texture features in the original image; determine the reflection area in the original image based on the edge information and texture features; rotate the reflection image in the reflection area to obtain a rotated reflection image.

[0137] In an optional embodiment of the present application, the rotation module 62 is used to determine the candidate edge line of the reflection area based on the edge information; obtain the texture features within a preset range of the candidate edge line, and compare the texture features with the preset texture features to obtain a comparison result; determine the target pixel points belonging to the reflection area based on the comparison result; connect adjacent target pixel points to obtain the reflection area in the original image.

[0138] In an optional embodiment of the present application, the rotation module 62 is used to identify directional features in the reflected image; match the directional features with preset features in multiple preset directional templates to obtain a hit target directional template; obtain the rotation angle in the target directional template, and perform a rotation operation on the reflected image according to the rotation angle to obtain a rotated reflected image.

[0139] In an optional embodiment of the present application, the stitching module 63 is used to detect the position information of the rotated reflected image in the original image; crop the original image based on the position information to obtain a cropped natural image; analyze the rotated reflected image and the cropped natural image to obtain a stitching position; and fuse the rotated reflected image and the cropped natural image according to the stitching position to obtain a stitched image.

[0140] In an optional embodiment of the present application, the device also includes: a detection module, used to detect whether there is a candidate object in the natural image outside the reflection area; if there is a candidate object in the non-edge area of ​​the natural image, directly performing a reminder operation; if there is a candidate object in the edge area of ​​the natural image, or if there is no candidate object in the natural image, performing a detection operation on the stitched image.

[0141] In an optional embodiment of the present application, the execution module 64 is used to obtain multiple standard features in a preset object feature library; extract key features in the stitched image, and calculate the matching degree between the key features and the standard features; if the matching degree exceeds a preset threshold, it is determined that the target object exists in the stitched image, or, if the matching degree does not exceed the preset threshold, it is determined that the target object does not exist in the stitched image.

[0142] In an optional embodiment of the present application, the device also includes: an adjustment module, used to detect whether there is any missing key feature in the stitched image; if there is any missing, analyzing the position distribution of the missing key feature in the stitched image to obtain the area to be supplemented in the stitched image; determining the adjustment direction of the reflection device according to the area to be supplemented, and controlling the reflection device to perform a moving operation according to the adjustment direction; after the moving operation is completed, restarting the image acquisition device on the smart door lock to perform the acquisition operation.

[0143] See also Figure 7 , Figure 7 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 7 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system).

[0144] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0145] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0146] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the use of a computer device based on the presentation of a small program landing page, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0147] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0148] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0149] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0150] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A monitoring method for a smart door lock, characterized in that: The method comprises: Acquire an original image captured by an image acquisition device on a smart door lock, wherein the original image includes a reflected image in a reflection device on the smart door lock and a natural image outside the reflection device; Identifying a reflection area in the original image, and rotating the reflection image in the reflection area to obtain a rotated reflection image; splicing the rotated reflected image with the natural image outside the reflection area to obtain a spliced ​​image; It is detected whether a target object exists in the stitched image, and if the target object exists, a corresponding reminder operation is performed.

2. The method according to claim 1, characterized in that The identifying the reflection area in the original image and rotating the reflection image in the reflection area to obtain the rotated reflection image includes: Identifying edge information and texture features in the original image; Determining a reflection area in the original image according to the edge information and the texture feature; The reflected image in the reflective area is rotated to obtain a rotated reflected image.

3. The method according to claim 2, characterized in that The determining the reflection area in the original image according to the edge information and the texture feature includes: Determine a candidate edge line of the reflection area according to the edge information; Acquire texture features within a preset range of the candidate edge line, and compare the texture features with preset texture features to obtain a comparison result; Determine the target pixel point belonging to the reflection area based on the comparison result; Adjacent target pixel points are connected to obtain a reflection area in the original image.

4. The method according to claim 2, characterized in that: The step of rotating the reflected image in the reflective area to obtain a rotated reflected image includes: identifying directional features in the reflected image; Matching the directional feature with preset features in a plurality of preset directional templates to obtain a hit target directional template; A rotation angle in the target direction template is obtained, and a rotation operation is performed on the reflected image according to the rotation angle to obtain a rotated reflected image.

5. The method according to claim 1, characterized in that: The step of stitching the rotated reflected image with the natural image outside the reflection area to obtain a stitched image comprises: Detecting position information of the rotated reflected image in the original image; Cropping the original image based on the position information to obtain a cropped natural image; Analyze the rotated reflected image and the cropped natural image to obtain a splicing position; According to the stitching position, the rotated reflected image and the cropped natural image are fused to obtain a stitching image.

6. The method according to claim 1, characterized in that Before detecting whether there is a target object in the stitched image, the method further includes: Detecting whether there is a candidate object in the natural image outside the reflection area; If there is a candidate object in the non-edge area of ​​the natural image, a reminder operation is directly performed; if there is a candidate object in the edge area of ​​the natural image, or there is no candidate object in the natural image, a detection operation is performed on the spliced ​​image.

7. The method according to claim 1, characterized in that The detecting whether there is a target object in the stitched image includes: Obtain multiple standard features in a preset object feature library; Extracting key features from the stitched image, and calculating the matching degree between the key features and the standard features; If the matching degree exceeds a preset threshold, it is determined that the target object exists in the stitched image, or, if the matching degree does not exceed the preset threshold, it is determined that the target object does not exist in the stitched image.

8. The method according to claim 1, characterized in that The method further comprises: Detecting whether key features in the stitched image are missing; If there is a missing situation, analyzing the position distribution of the key features with the missing situation in the stitched image to obtain the area to be supplemented in the stitched image; Determining an adjustment direction of the reflecting device according to the area to be supplemented, and controlling the reflecting device to perform a moving operation according to the adjustment direction; After the movement operation is completed, the image acquisition device on the smart door lock is restarted to perform the acquisition operation.

9. A monitoring system for a smart door lock, characterized in that: The system comprises: an image acquisition device, a processing device, and a reflection device; The image acquisition device is used to acquire an original image within the monitoring range of the smart door lock, wherein the original image includes a reflected image within the reflection device and a natural image outside the reflection device; The processing device is used to stitch the reflected image and the natural image in the original image to obtain a stitched image, determine the adjustment angle of the reflecting device according to the stitched image, and send the adjustment angle to the reflecting device; The reflecting device is used to receive the adjustment angle sent by the processing device and perform the moving operation according to the adjustment angle.

10. A monitoring device for a smart door lock, characterized in that: The device comprises: An acquisition module, used to acquire an original image acquired by an image acquisition device on a smart door lock, wherein the original image includes a reflected image in a reflection device on the smart door lock and a natural image outside the reflection device; A rotation module, used for identifying a reflection area in the original image, and rotating the reflection image in the reflection area to obtain a rotated reflection image; A stitching module, used for stitching the rotated reflected image with the natural image outside the reflection area to obtain a stitched image; The execution module is used to detect whether there is a target object in the spliced ​​image, and if the target object exists, execute a corresponding reminder operation.

Images