Target detection method, device, electronic device and readable storage medium

Abstract

The present application discloses a target detection method, device, electronic device and readable storage medium, which is applied to the field of artificial intelligence technology. The method includes: obtaining an image to be detected, performing target detection on a target object in the image to be detected, and obtaining a target position feature corresponding to the target object; obtaining the similarity between the target position feature and at least one position reference feature, wherein the position reference feature is determined based on the position of the target reference object in multiple sample images; and obtaining a detection result based on the similarity and the corresponding similarity threshold. The target detection method proposed in the present application can improve the accuracy of target detection to a certain extent by matching the target position feature with at least one position reference feature.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and more specifically, to a target detection method, device, electronic device and readable storage medium. Background Art

[0002] With the rise and development of smart cities, smart buildings, smart security, and smart homes, intelligent basic data detection has become one of the core keys, especially the detection of target objects is the top priority in the intelligent process. Among them, target objects can include human bodies, objects, or animals. Therefore, how to detect target objects more accurately is an urgent problem to be solved. Summary of the invention

[0003] In view of this, the present application proposes a target detection method, device, electronic device and readable storage medium to improve the above-mentioned defects.

[0004] In a first aspect, an embodiment of the present application provides a target detection method, the method comprising: acquiring an image to be detected, performing target detection on a target object in the image to be detected, and obtaining a target position feature corresponding to the target object; acquiring a similarity between the target position feature and at least one position reference feature, the position reference feature being determined based on the position of the target reference object in multiple sample images; and obtaining a detection result based on the similarity and a corresponding similarity threshold.

[0005] In the second aspect, an embodiment of the present application provides a target detection method, which is applied to a smart door lock and includes: obtaining an image to be detected collected by the smart door lock, performing target detection on a target object in the image to be detected, and obtaining a target position feature corresponding to the target object; obtaining the similarity between the target position feature and at least one position reference feature, wherein the position reference feature is determined based on the position of the target reference object in multiple sample images; if the similarity reaches a corresponding similarity threshold, generating a warning information based on the image to be detected and sending it to the target terminal.

[0006] Optionally, if present, starting the image acquisition device includes: if present, determining whether the human body is continuously located within the preset range within a preset time period; if so, starting the image acquisition device.

[0007] Optionally, the method further includes: if the target object does not belong to a specified type, shutting down the image acquisition device.

[0008] In a third aspect, an embodiment of the present application provides a target detection device, the device comprising: a first image acquisition module, a first similarity acquisition module and a first result determination module. Among them, the first image acquisition module is used to acquire an image to be detected, perform target detection on a target object in the image to be detected, and obtain a target position feature corresponding to the target object. The first similarity acquisition module is used to obtain the similarity between the target position feature and at least one position reference feature, and the position reference feature is determined based on the position of the target reference object in multiple sample images. The first result determination module is used to obtain a detection result based on the similarity and a corresponding similarity threshold.

[0009] Optionally, the target detection device also includes a feature acquisition module, which is used to acquire the multiple sample images, wherein the sample images include labeled target reference objects; extract sample target features corresponding to each target reference object from each sample image; cluster each sample target feature, and generate at least one position reference feature based on the central feature in each clustering result.

[0010] Optionally, the first similarity acquisition module is further used to determine whether the similarity is greater than a similarity threshold; if so, it is determined that a target object exists in the image to be detected.

[0011] Optionally, the first result determination module is further configured to generate warning information based on the image to be detected if the detection result indicates that a target object exists in the image to be detected; and send the warning information to a target terminal.

[0012] In the fourth aspect, the embodiment of the present application provides a target detection device, which is applied to a smart door lock, and the device includes: a second image acquisition module, a second similarity acquisition module and a sending module. Among them, the second image acquisition module is used to acquire the image to be detected collected by the smart door lock, perform target detection on the target object in the image to be detected, and obtain the target position feature corresponding to the target object; the second similarity acquisition module is used to obtain the similarity between the target position feature and at least one position reference feature, and the position reference feature is determined based on the position of the target reference object in multiple sample images; the sending module is used to generate warning information according to the image to be detected and send it to the target terminal if the similarity reaches the corresponding similarity threshold.

[0013] Optionally, the target detection device also includes an image acquisition device for detecting whether there is a human body within a preset range of the smart door lock; if so, the image acquisition device is started, and the image acquisition device is used to acquire the image to be detected.

[0014] Optionally, the image acquisition device is also used to start the image acquisition device if a human body exists, including: if so, determining whether the human body is continuously located within the preset range within a preset time period; if so, starting the image acquisition device.

[0015] Optionally, the second similarity acquisition module is also used to obtain the identity information of the target object corresponding to the target position feature if the similarity reaches a corresponding similarity threshold; determine whether the target object belongs to a specified type based on the identity information of the target object, and the object of the specified type has safety hazards; if the target object belongs to the specified type, generate warning information based on the image to be detected and send it to the target terminal.

[0016] Optionally, the image acquisition device is further configured to shut down the image acquisition device if the target object does not belong to a specified type.

[0017] In a fifth aspect, the present application provides an electronic device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the above-described method when executed by the processor.

[0018] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored, and the computer program implements the above method when executed by a processor.

[0019] Compared with the prior art, the embodiments of the present application propose a target detection method, device, electronic device and readable storage medium. The target detection method can improve the accuracy of target detection to a great extent by matching the target position feature with at least one position reference feature. Specifically, the present application can perform target detection on the target object in the acquired image to be detected to obtain the target position feature corresponding to the target object, and then obtain the similarity between the target position feature and at least one position reference feature, wherein the position reference feature can be determined based on the position of the target reference object in multiple sample images, and the detection result is obtained based on the similarity and the corresponding similarity threshold. The present application can improve the accuracy of target detection by matching the target position feature with at least one position reference feature, thereby avoiding the image to be detected from being misidentified, and effectively improving the accuracy of target detection.

[0020] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the following specifically cites preferred embodiments and describes them in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.

[0022] Figure 1 A schematic diagram showing the structure of a network system proposed in this application is shown;

[0023] Figure 2 A flow chart of a target detection method proposed in one embodiment of the present application is shown;

[0024] Figure 3 A schematic diagram showing a target object in an image to be detected in a target detection method proposed in an embodiment of the present application;

[0025] Figure 4 A schematic diagram showing a target detection method proposed in an embodiment of the present application in which an image to be detected includes multiple target objects;

[0026] Figure 5 A schematic diagram showing a target detection method proposed in an embodiment of the present application in which an image to be detected includes target objects of multiple different categories;

[0027] Figure 6 A schematic diagram of a sample image in a target detection method proposed in an embodiment of the present application is shown;

[0028] Figure 7 A flow chart of a target detection method proposed in another embodiment of the present application is shown;

[0029] Figure 8 An example diagram showing a result obtained by clustering multiple sample target features in a target detection method proposed in another embodiment of the present application;

[0030] Fig. 9 A flow chart of a target detection method proposed in one embodiment of the present application is shown;

[0031] Fig.10 A flowchart of step S330 in a target detection method proposed in another embodiment of the present application is shown;

[0032] Fig.11 A structural block diagram of a target detection device proposed in an embodiment of the present application is shown;

[0033] Fig.12 A structural block diagram of other modules of an object detection device proposed in an embodiment of the present application is shown;

[0034] Fig.13 A structural block diagram of a target detection device proposed in an embodiment of the present application is shown;

[0035] Fig.14 A structural block diagram of an electronic device for executing a target detection method according to an embodiment of the present application is shown;

[0036] Fig.15 A schematic diagram of the hardware structure of a mobile terminal in a target detection method proposed in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0037] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0038] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0039] See also Figure 1 , a network system 10 provided in an embodiment of the present application, the network system 10 includes: a target terminal 11, a server 12, a gateway 13, an electronic device 14 and a router 15. Among them, the target terminal 11 can be any device with communication and storage functions, such as: a smart phone, a desktop computer, a laptop computer, a tablet computer or other intelligent communication devices with network connection functions. The server 12 can be a network access server, a database server, a cloud server, etc. Optionally, the gateway 13 can be built based on the ZigBee protocol, and the electronic device 14 can be a device pre-added to the gateway 13. For example, the electronic device can be a device in the kit to which the gateway belongs when the gateway leaves the factory; it can also be a device subsequently connected to the gateway 13 through user operation.

[0040] Optionally, the electronic device 14 can establish a network connection with the gateway 13 based on the ZigBee protocol, thereby joining the ZigBee network. In addition, the electronic device 14 can be a smart door lock in a smart home device, and the smart door lock can be equipped with an image acquisition device, which can be used to capture scenes outside the smart door lock, such as a human body outside the door lock.

[0041] The gateway 13 and the target terminal 11 can be connected to the router 15 and connected to the Ethernet through the router 15. The router 15 can access the server through a wired or wireless communication connection. For example, the gateway 13 and the target terminal 11 can store the acquired information in the server 12. Optionally, the target terminal 11 can also establish a network connection with the server 12 through 2G / 3G / 4G / 5G, WIFI, etc., so as to obtain the data sent by the server 12.

[0042] Optional, such as Figure 1 The LAN path shown indicates that the target terminal 11 is in the same LAN as the router 15 and the gateway 13, and the WAN path indicates that the target terminal 11 is in the same LAN as the router 15 and the gateway 13. When the target terminal 11 is in the same LAN as the router 15 and the gateway 13, the target terminal 11 can Figure 1 The local area network path shown interacts with the gateway 13 and the electronic device 14 connected to the gateway 13; it can also be communicated through Figure 1 The wide area network path shown interacts with the gateway 13 and the electronic device 14 connected to the gateway 13. When the target terminal 11 is not in the same local area network as the router 15 and the gateway 13, the target terminal 11 can communicate with the gateway 13 through the following methods: Figure 1 The wide area network path shown interacts with a gateway 13 and an electronic device 14 connected to the gateway 13 .

[0043] As an important part of computer vision, target detection has developed rapidly in recent years. The main task of target detection is to determine whether the input image or video contains the target object and accurately find the position of the target object in the image. The target object can be a human body or an animal. Existing target detection technology has been widely used in security, crowd analysis, intelligent transportation, smart home and other fields. However, target detection also has problems such as dark light and complex background during its application, which leads to the problem of false detection when performing target detection tasks, thus affecting the accuracy of target detection.

[0044] Therefore, in order to overcome the above-mentioned defects, Figure 2 The present application embodiment provides a target detection method, which is described by taking the method applied to a computer device (or electronic device) as an example. The computer device may be Figure 1 The electronic device in the embodiment may be a server, a gateway or a smart home device, and the method may include steps S110 to S130.

[0045] Step S110: Acquire an image to be detected, perform target detection on a target object in the image to be detected, and obtain target position features corresponding to the target object.

[0046] The embodiments of the present application can be applied to electronic devices, which may include smart door locks, smart cameras, smart air conditioners, smart robots, or servers and gateways, etc. The common feature of these electronic devices is that they are equipped with image acquisition devices.

[0047] In some embodiments, the image to be detected may include a target object, which refers to a target to be detected from the image to be detected, and may include a human body, a human face, or other objects. For example, the target object may be a human body, or an animal, such as a pet cat or a pet dog, or other objects. The specific target object is not specifically limited here, and may be selected according to actual conditions.

[0048] In other implementations, an image to be detected may include one target object or multiple target objects. For example, when a human body appears outside the smart door lock, there is only one target object in the image to be detected collected by the smart door lock. Figure 3 As shown, when the target object 101 appears outside the smart door lock, the image acquisition device on the smart door lock can acquire the image of the target object 101 to obtain the image to be detected. At this time, the target contained in the image to be detected is only the target object 101.

[0049] In other implementations, a to-be-detected image may contain multiple target objects. For example, when multiple human bodies appear outside the smart door lock, the to-be-detected image collected by the smart door lock contains multiple target objects. Figure 4 As shown, when target object 102 and target object 103 appear outside the smart door lock, the image acquisition device on the smart door lock can acquire images of the target object 102 and target object 103 to obtain the image to be detected. At this time, the targets contained in the image to be detected are target object 102 and target object 103.

[0050] In other embodiments, when the image to be detected contains multiple target objects, the multiple target objects may be objects of the same category, such as Figure 4 In the above figure, target object 102 and target object 103 are both human bodies. In addition, when the image to be detected contains multiple target objects, these multiple target objects can also be objects of different categories, such as Figure 5 The target object 104 and the target object 105 are objects of two categories, that is, the target object 104 is a pet dog, and the target object 105 is a human body. The two target objects can be objects of different categories.

[0051] In some embodiments, the image to be detected may be an image collected by an image acquisition device, or may be a certain video frame in a video to be detected collected by an image acquisition device. In other words, when the video to be detected is collected, the embodiment of the present application may use certain frames of video in the video to be detected as the image to be detected. The specific selection of how many frames, or which frame of image to be used as the image to be detected can be determined according to the detection rate. For example, if it takes 0.25s to detect an image, then a frame of image can be captured every 0.25s in the video to be detected for target detection. Therefore, when acquiring the image to be detected, the present application may first determine the target time required to acquire a target position feature, and then acquire an image to be detected from the video to be processed every target time.

[0052] In the embodiment of the present application, the target position feature may be a feature used to characterize the position information of the target object in the image to be detected, wherein the target position information may include coordinate information and length and width information, wherein the coordinate information may be the coordinates of at least one point on the candidate box corresponding to the target object, and the length and width information is the length and width of the candidate box corresponding to the target object. By combining the coordinate information and the length and width information, the embodiment of the present application can more accurately detect the target object.

[0053] In some embodiments, the target acquisition time may be the total time required for the target detection model to detect the image to be detected and obtain the target position features corresponding to the target object. In addition, the target time may also be the total time required for the target detection model to detect the image to be detected and obtain the target position features corresponding to the target object to send the warning information to the target terminal. The specific determination of the target time is not explicitly limited here and can be selected according to actual conditions.

[0054] In some embodiments, after acquiring the image to be detected, the electronic device may perform target detection on the target object in the image to be detected to obtain the target position feature corresponding to the target object. Specifically, the embodiment of the present application may input the image to be detected into a target detection model to obtain the target position feature corresponding to the target object using the target detection model. As an example, the target detection model may be a human detection model, that is, the target detection model may be used to detect the human body in the image to be detected to obtain the target position feature corresponding to the target object, and the target position feature may be the position of the human body in the image to be detected.

[0055] In an embodiment of the present application, before inputting the image to be detected into the target detection model, the embodiment of the present application may pre-process the image to be detected, such as performing denoising, filtering and other operations on the image to be detected. In addition, when the target detection model is used to perform target detection on the target object in the image to be detected, the embodiment of the present application may also first scale the image to be detected so that the size of the scaled image to be processed is similar to the preset size, that is, the size of the image to be detected is scaled to the preset size, wherein the preset size may be the same as the size of the image for training the target detection model. For example, the preset size may be 300×300. By scaling the size of the image to be detected to the preset size, the rate of target detection can be accelerated to a certain extent.

[0056] In some embodiments, before performing target detection on the target object in the image to be detected, the embodiment of the present application may first obtain a target detection model. Specifically, first obtain a target detection network, which may be an SSD network (Single Shot MultiBox Detector, single-stage target detection algorithm), an R-CNN network, a Fast R-CNN network, and a Faster R-CNN network.

[0057] Optionally, an embodiment of the present application can obtain sample data, and the sample data can be composed of multiple sample images and sample target features corresponding to each sample image, wherein the sample target features can be position features of the target reference object in the sample image, and the sample target features can also be referred to as rectangular annotation box information, and the rectangular box annotation information can include coordinate information and length and width information.

[0058] Optionally, the coordinate information in the sample target feature information may be the coordinates of the rectangular box corresponding to the target reference object in the sample image, and the coordinate information may be the coordinates of the upper left corner of the rectangular box, or the coordinates of the lower right corner of the rectangular box, or the coordinates of the lower left corner or the upper right corner of the rectangular box, or the coordinates of the center point of the rectangular box. The coordinate information specifically refers to the coordinates of which point of the rectangular box is not explicitly limited here, and can be selected according to actual conditions. In addition, the length and width information may be the length and width of the rectangular box.

[0059] As an example, the target detection model is a human detection model. In this case, the data for training the target detection network can be images containing human figures, that is, the sample data includes multiple sample images and rectangular annotation box information corresponding to the target reference object in each sample image. The rectangular annotation box information can be expressed as (x0, y0, w, h), where x0 and y0 are the coordinates of the center point of the annotation box, and w and h are the width and height of the annotation box, respectively. For example, the sample image can be as follows Figure 6As shown in the figure, the sample image includes a target reference object 106, and the rectangular annotation box corresponding to the target reference object 106 is 107. The rectangular annotation box information may include the coordinates (x0, y0) of D1, and the height h and width w of the rectangular box. In addition, the sample images may all contain a human figure, and the annotation box may be the smallest rectangular box that completely surrounds the human figure outline.

[0060] In the embodiment of the present application, the sample data may be a database consisting of annotated images on the Internet, or it may be data obtained by the user manually annotating the image after acquiring the image, that is, the sample data may be data obtained by the user by calibrating the target reference object in the image.

[0061] Step S120: Obtaining the similarity between the target location feature and at least one location reference feature.

[0062] As a method, after obtaining the target position feature, the embodiment of the present application can obtain the similarity between the target position feature and at least one position reference feature, wherein the position reference feature can be determined based on the position of the target reference object in multiple sample images, and the sample image can be an image in the above sample data. The position reference feature can be obtained by clustering the position information of the target reference object in multiple sample images.

[0063] In addition, obtaining the similarity between the target position feature and at least one position reference feature may be obtaining the distance between the target position feature and at least one position reference feature. After obtaining the similarity between the target position feature and at least one position reference feature, the embodiment of the present application may determine whether the similarity is greater than a similarity threshold. If the distance between the target position feature and at least one position reference feature is greater than the similarity threshold, it is determined that the target position feature is similar to the at least one position reference feature.

[0064] In some embodiments, the target position feature may be the position of the target object in the image to be detected, and the target position feature may be the position information of the target candidate frame. In addition, the target position feature may include coordinate information and length and width information, wherein the coordinate information may be the coordinates of a point on the target candidate frame, which may be the coordinate point of the upper left corner of the target candidate frame, the coordinate point of the lower right corner of the target candidate frame, or the coordinate point of the center of the target candidate frame, etc.

[0065] It should be noted that when the target position feature may also include multiple coordinate information, the embodiments of the present application may include description information of the coordinate points in addition to including these coordinate point information. Specifically, it may be explained whether the coordinate point belongs to the upper left corner or the lower left corner of the target candidate frame, or to the center point. In this way, the overall position information of the target candidate frame can be quickly and effectively determined. For example, the coordinate information included in the target position feature is the upper left corner coordinate information and the center point coordinate information. In this way, the width of the target candidate frame can be determined as the horizontal axis coordinate of the center point minus twice the horizontal axis coordinate of the upper left corner, and the width of the target candidate frame can be determined as the vertical axis coordinate of the center point minus twice the vertical axis coordinate of the upper left corner.

[0066] In some embodiments, the information included in the target position feature and the position reference feature may correspond to each other. For example, if the position reference feature includes a coordinate information and length and width information, then the target position feature also includes a coordinate information and length and width information. For another example, if the position reference feature includes two coordinate information, and the two coordinate information are the upper left corner coordinate information and the lower right corner coordinate information, then the target position feature also includes two coordinate information, and the two coordinate information are also the upper left corner coordinate information and the lower right corner coordinate information. Therefore, before obtaining the similarity between the target position feature and at least one position reference feature, the embodiment of the present application may also determine the information included in the position reference feature, and then determine the information included in the target position feature based on the information included in the position reference feature, so as to facilitate the matching of the position information.

[0067] In the embodiment of the present application, the scene of the sample image and the scene of the image to be detected may be the same, wherein the same scene may be that the target reference object contained in the sample image and the target object in the image to be detected are of the same type. For example, the target reference object and the target object are both human bodies, or the target reference object and the target object are both pet cats or pet dogs, etc.

[0068] In addition, the same scene can also mean that the target reference object in the sample image and the target object in the image to be detected are in the same environment, where the same environment can mean that the sample image and the image to be detected are acquired using the same type of image acquisition device, and the same type of image acquisition device means that the electronic devices configured with the image acquisition device are the same. For example, the sample image and the image to be referenced are both images acquired by the image acquisition device on the smart door lock, but the identity of the target object included in the sample image and the identity of the target reference object may be the same or different, and the two may also be images acquired using different smart door locks.

[0069] In some implementations, after obtaining the similarity between the target location feature and at least one location reference feature, the embodiment of the present application may obtain a detection result based on the similarity and a similarity threshold, that is, enter step S130.

[0070] Step S130: Obtain a detection result based on the similarity and the corresponding similarity threshold.

[0071] As a method, after obtaining the similarity between the target location feature and at least one location reference feature, the embodiment of the present application can obtain the similarity between the coordinate information in the target location feature and the coordinate information in the location reference feature, and determine whether the target location feature is similar to at least one location reference feature based on the similarity, and then the determination result can be used as the detection result. Specifically, if the similarity between the coordinate information in the target location feature and the coordinate information in at least one location reference feature is greater than the similarity threshold, it is determined that the target location feature is similar to at least one location reference feature, and the detection result can be that the object corresponding to the target location feature belongs to the target object.

[0072] As another way, after obtaining the similarity between the target position feature and at least one position reference feature, the embodiment of the present application can obtain the similarity between the length and width information in the target position feature and the length and width information in the position reference feature, and then determine whether the target position feature is similar to the at least one position reference feature based on the similarity, and use the determination result as the detection result. Specifically, if the similarity between the length and width information in the target position feature and the length and width information in the at least one position reference feature is greater than the similarity threshold, it is determined that the target position feature is similar to the at least one position reference feature.

[0073] As another way, after obtaining the similarity between the target position feature and at least one position reference feature, the embodiment of the present application can obtain the similarity between the coordinate information in the target position feature and the coordinate information in the position reference feature, and obtain the similarity between the length and width information in the target position feature and the length and width information in the position reference feature, and then determine whether the target position feature is similar to at least one position reference feature based on the similarity, and use the determination result as the detection result. Specifically, if the similarity between the coordinate information in the target position feature and the coordinate information in the position reference feature, and the similarity between the length and width information in the target position feature and the length and width information in at least one position reference feature is greater than the similarity threshold, it is determined that the target position feature is similar to at least one position reference feature, and the detection result can be that the object corresponding to the target position feature belongs to the target object.

[0074] As another way, the target position feature may also include multiple coordinate information, wherein the multiple coordinate information may be the coordinates of at least two points on the target candidate frame. After obtaining the similarity between the target position feature and at least one position reference feature, the embodiment of the present application may obtain the similarity between the coordinate information of at least two points in the target position feature and the coordinate information of at least two points in the position reference feature, and then determine whether the target position feature is similar to at least one position reference feature based on the similarity, and use the determination result as the detection result. Specifically, if the similarity between the coordinate information of at least two points in the target position feature and the coordinate information of at least two points in at least one position reference feature is greater than the similarity threshold, it is determined that the target position feature is similar to at least one position reference feature. In addition, the coordinates of at least two points may be two coordinates in different directions, such as the first coordinate point may be the upper left corner coordinate, and the second coordinate point may be the lower right corner coordinate.

[0075] In other words, the coordinates of at least two points may be diagonal coordinate points on the target candidate box, such as at least two coordinate points may be coordinate points of the lower left corner and the upper right corner of the target candidate box, or at least two coordinate points may be coordinate points of the upper left corner of the target candidate box and the center of the target candidate box. It should be noted that the similarity thresholds in the above embodiments may be the same or different, and there is no explicit restriction on how to set them here, and they may be selected according to actual conditions.

[0076] A target detection method proposed in the present application can improve the accuracy of target detection to a great extent by matching the target position feature with at least one position reference feature. Specifically, the present application can perform target detection on the target object in the acquired image to be detected to obtain the target position feature corresponding to the target object, and then obtain the similarity between the target position feature and at least one position reference feature, wherein the position reference feature can be determined based on the position of the target reference object in multiple sample images, and the detection result is obtained based on the similarity and the corresponding similarity threshold. The present application can improve the accuracy of target detection by matching the target position feature with at least one position reference feature, thereby avoiding the image to be detected from being misidentified, that is, improving the user experience.

[0077] Another embodiment of the present application provides a target detection method, which is described by taking the method applied to a computer device (or electronic device) as an example. The computer device may be Figure 1 The electronic device in the system can be a server, a gateway or a smart home device. Figure 7 ,from Figure 7 It can be seen that the method includes steps S210 to S260.

[0078] Step S210: Acquire multiple sample images.

[0079] In an embodiment of the present application, each sample image may include a marked target reference object, and the sample image has the same scene as the image to be detected, that is, the type of target contained in the sample image is the same as the type of target contained in the image to be detected. In addition, the device for acquiring the sample image and the device for acquiring the image to be detected may be the same, such as the device for acquiring the sample image and the device for acquiring the image to be detected may both be smart door locks. In other words, the image acquisition device for acquiring the sample image and the image acquisition device for acquiring the image to be detected may be configured on the same electronic device.

[0080] As a method, when multiple sample images are acquired, the embodiment of the present application may perform image preprocessing on each sample image, such as filtering and denoising the sample image. In addition, the embodiment of the present application may also perform scaling processing on the sample images, that is, scaling each sample image to a preset size. For example, scaling the size of all sample images to 300×300 (pixels).

[0081] Step S220: extracting sample target features corresponding to each target reference object from each sample image.

[0082] As a method, after acquiring multiple sample images, the embodiment of the present application can determine the sample target features of the target reference object in each sample image. The sample target features of the target reference object can be acquired together with the sample image, that is, the sample target features of the target reference object can be pre-marked. In addition, the sample target features of the target reference object can also be acquired by the user through manual annotation after acquiring the sample image.

[0083] Optionally, when scaling the size of the sample image, the embodiment of the present application can also perform corresponding scaling for the position information of the target reference object, that is, perform geometric scaling. In the embodiment of the present application, the sample target feature of the target reference object can be the position feature of the smallest rectangular frame that completely surrounds the target reference object. It should be noted that a sample image can correspond to one sample target feature or multiple sample target features, that is, a sample image can include one target reference object or multiple target reference objects.

[0084] As an example, multiple sample images are obtained, and the sample target features determined based on the multiple sample images are n sample target features, which can be expressed as (x0, y0, w, h), where n is the number of target reference objects in all sample images, x0 and y0 are the center point coordinates of the candidate box corresponding to the target reference object, and w and h are the width and height of the candidate box corresponding to the target reference object, respectively. The number of sample target features can be the same as the number of sample images, or can be greater than the number of sample images. The n sample target features constitute n four-dimensional vectors.

[0085] Step S230: performing clustering processing on each sample target feature, and generating at least one position reference feature according to the central feature in each clustering result.

[0086] As a method, after obtaining multiple sample target features, the embodiment of the present application can cluster the multiple sample target features to obtain multiple types of sample target features. Specifically, when clustering the multiple sample target features, the embodiment of the present application can use the K-Means clustering method for clustering.

[0087] As an example, clustering multiple sample target features can obtain the following Figure 8 The multiple types of sample target features shown may be first type sample target features 201, second type sample target features 202, and third type sample target features 203. Each type of sample target feature may include multiple sample target features, and the distance between sample target features in each type of sample target feature is small.

[0088] It should be noted that Figure 8 The multi-class sample target features shown are only examples. In the actual clustering process, the number of clusters is much greater than three, that is, the number of clusters in the embodiment of the present application is much greater than three, so as to ensure the accuracy of target detection. For example, the number of clusters can be set between 100 and 10,000 according to empirical values.

[0089] In some embodiments, after clustering multiple sample target features to obtain multiple classes of sample target features, the embodiment of the present application can determine the sample target features under each class of sample target features, and use the central sample target feature as the preset sample target feature. In other words, the embodiment of the present application can obtain multiple central sample target features by using the clustering method, and the multiple central sample target features can be expressed as (x0 1 ,y0 1 , w 1 ,h 1 )、(x0 2 ,y0 2 , w 2 ,h2 ),……、(x0 c ,y0 c , w c ,h c ), where c is the category of the sample target feature, that is, n sample target features are clustered into c categories.

[0090] As an example, Figure 8 The sample target feature 2011 in the first category of sample target features 201 can be used as the first center-based sample target feature; the sample target feature 2021 in the second category of sample target features 202 can be used as the second center-based sample target feature; and the sample target feature 2031 in the third category of sample target features 203 can be used as the third center-based sample target feature.

[0091] It should be noted that, in the embodiment of the present application, the central sample target feature under each category of sample target features can be a sample target feature in each category of sample target features, or it can be the average of all sample target features under each category of sample target features. The specific determination of the central sample target feature is not explicitly restricted here and can be selected according to actual conditions.

[0092] Step S240: Acquire the image to be detected, perform target detection on the target object in the image to be detected, and obtain the target position feature corresponding to the target object.

[0093] Step S250: Obtain the similarity between the target location feature and at least one location reference feature.

[0094] In an embodiment of the present application, the step of acquiring position reference features may be performed after acquiring the image to be detected, or may be performed before acquiring the image to be detected. There is no clear restriction on which step is performed, and it may be selected according to actual circumstances.

[0095] From the above introduction, it can be known that the target position features corresponding to the target object in the embodiment of the present application may include coordinate information and length and width information, wherein the coordinate information may be the coordinate of a coordinate point of the candidate box corresponding to the target object, and the coordinate may include the horizontal axis coordinate and the vertical axis coordinate, and the length and width information may include the length and width of the candidate box corresponding to the target object.

[0096] In an embodiment of the present application, the similarity can be obtained by calculating the Euclidean distance between the target position feature and at least one position reference feature. The similarity between the target position feature and at least one position reference feature can be obtained by calculating the Euclidean distance between the target position feature and at least one position reference feature. In addition, the Euclidean distance is the distance between the coordinate information in the target position feature and the coordinate information in the position reference feature, and the distance between the length and width information in the target position feature and the length and width information in the position reference feature. The Euclidean distance is negatively correlated with the similarity. The larger the Euclidean distance, the smaller the similarity; the smaller the Euclidean distance, the greater the similarity.

[0097] If the Euclidean distance is used, after calculating the Euclidean distance between the target position feature and at least one position reference feature, the Euclidean distance is then converted to obtain the corresponding similarity.

[0098] As an example, the similarity can be calculated in the following way. The i-th position reference feature can be (x0 i ,y0 i , w i ,h i ), the target position feature obtained by target detection can be (x p ,yp,w p ,h p ), the Euclidean distance between the target location feature and at least one location reference feature can be:

[0099] Where i = 1, 2, 3, ..., c. d i Represents the Euclidean distance between the target position feature and the i-th position reference feature.

[0100] The similarity between the target location feature and at least one location reference feature may be:

[0101] S i =1 / d i ; where i = 1, 2, 3, ..., c. S i Represents the similarity between the target position feature and the i-th position reference feature.

[0102] If the calculated c similarities (S1-S c ) are both less than the similarity threshold, it is determined that the object corresponding to the target position feature does not belong to the target object. For example, the object may be a background object and does not belong to the target object.

[0103] On the contrary, if the calculated c similarities (S1-S c) is greater than the similarity threshold, it can be determined that the object corresponding to the target position feature belongs to the target object.

[0104] It is understandable that the similarity calculation method is not limited to the method of calculating the Euclidean distance, and can also be calculated by other methods, for example, the similarity can also be calculated by calculating the cosine similarity, and this application does not limit this.

[0105] Step S260: Obtain a detection result based on the similarity and the corresponding similarity threshold.

[0106] As a method, when the similarity between the target position feature and at least one position reference feature is obtained, the embodiment of the present application can determine whether the similarity is greater than the similarity threshold. If the similarity is greater than the similarity threshold, the detection result at this time can be that the object corresponding to the target position feature in the graphic to be detected belongs to the target object.

[0107] As another way, at least one location reference feature may include multiple candidate location features. Therefore, when obtaining the distance between the target location feature and at least one location reference feature, the embodiment of the present application can also obtain the similarity between the target location feature and each candidate location feature, and then determine whether there is a candidate location feature with a similarity greater than a similarity threshold among the multiple candidate location features. If there is at least one candidate location feature with a similarity greater than the similarity threshold among the multiple candidate location features, it is determined that the similarity is greater than the similarity threshold, that is, it is determined that the target location feature is similar to the at least one location reference feature.

[0108] In some embodiments, when multiple candidate location features are obtained, the embodiments of the present application may also classify the multiple candidate location features, and then sort each category from large to small according to the size of the probability of occurrence, wherein the size of the probability of occurrence may be the number of candidate location features under each category, that is, the larger the number of candidate location features, the larger the corresponding probability of occurrence. When obtaining the similarity between the target location feature and at least one location reference feature, the embodiments of the present application may first obtain a candidate location feature with a larger probability, and search from the candidate location features of this category for a location reference feature similar to the target location feature. If so, it is determined that the target location feature is similar to at least one location reference feature. If not, the next category of candidate location features is obtained, and so on. This can speed up the rate of information similarity and improve the user experience.

[0109] In other embodiments, if there is no candidate position feature similar to the target position feature among the multiple candidate position features, it means that the candidate frame corresponding to the target position feature may be the background, that is, the detected target position feature may be erroneous information, and in this case, the image to be detected does not need to be sent to the target terminal. This can reduce the problem of high misrecognition rate caused by complex environment and dim light.

[0110] In an embodiment of the present application, after obtaining the detection result based on the similarity and the corresponding similarity threshold, the embodiment of the present application can generate warning information according to the graphics to be detected, and send the warning information to the target terminal, wherein the target terminal can be an electronic device that can run applications, such as a smart phone, a tablet computer, an e-book, etc. As a way, before the electronic device sends the warning information to the target terminal, it can also determine whether the network status between the electronic device and the target terminal is normal, and if normal, send the warning information to the target terminal. If the network status between the electronic device and the target terminal is abnormal, it can be detected again after a preset time period whether the network status between the electronic device and the target terminal is normal, and if normal, send the warning information to the target terminal.

[0111] As another way, the electronic device can count the number of times the network status is detected. When the number is greater than a preset number, the embodiment of the present application can stop the detection of the network status. At the same time, it can also determine whether the device connected to the electronic device has an audio-visual device or determine whether the electronic device is equipped with an audio-visual device. If there is an audio-visual device, the electronic device can instruct the audio-visual device to output prompt information. In addition, the electronic device can count the number of times the network status is detected. When the number is greater than a preset number, the embodiment of the present application can stop the detection of the network status. At the same time, it can also use the built-in audio-visual device of the electronic device to output prompt information.

[0112] In some embodiments, when it is determined that the target position feature is similar to at least one position reference feature, the embodiment of the present application may perform motion analysis on the image to be detected, and then determine the danger level of the target object in the image to be detected based on the motion analysis result, and then send different prompt information based on the danger level, that is, the higher the danger level, the stronger the corresponding prompt information. Among them, the danger level can be determined based on the motion analysis result, that is, the more similar the motion of the target object in the image to be detected is to the preset motion, the higher its danger level. Among them, the preset motion can be a sneaky motion, and the sneaky motion can be a motion of looking around, a motion of prying the door, or a motion of deliberately covering the face, etc.

[0113] In the embodiment of the present application, different danger levels correspond to different prompt information. For example, when the danger level is level 1, the corresponding output prompt is only a display reminder, and when the danger level is level 3, the corresponding output prompt is a sound and display reminder. For another example, as the danger level increases, the volume of the corresponding sound reminder increases.

[0114] A target detection method proposed in the present application can improve the accuracy of target detection to a great extent by matching the target position feature with at least one position reference feature. Specifically, the present application can perform target detection on the target object in the acquired image to be detected to obtain the target position feature corresponding to the target object, and then obtain the similarity between the target position feature and at least one position reference feature, wherein the position reference feature can be determined based on the position of the target reference object in multiple sample images, and the detection result is obtained based on the similarity and the corresponding similarity threshold. The present application can improve the accuracy of target detection by matching the target position feature with at least one position reference feature, thereby avoiding the image to be detected from being misidentified, that is, improving the user experience. In addition, the embodiment of the present application uses a clustering operation to select a central sample target feature from multiple sample target features as a position reference feature, which can further improve the accuracy of target detection.

[0115] Another embodiment of the present application proposes a target detection method, which is applied to a smart door lock. Fig. 9 ,from Fig. 9 It can be seen that the method includes steps S310 to S330.

[0116] Step S310: Acquire the image to be detected collected by the smart door lock, perform target detection on the target object in the image to be detected, and obtain the target position feature corresponding to the target object.

[0117] As a method, the embodiment of the present application can be applied to a smart door lock of a smart home device, and the smart door lock can be configured with an image acquisition device, which can be used to obtain an image to be detected, and the image acquisition device can be used to acquire an image outside the door lock. In other words, the image to be detected in the embodiment of the present application can be an image outside the door lock, that is, when the user appears outside the door lock, the image acquisition device on the smart door lock can acquire the image of the user to obtain the image to be detected.

[0118] It can be known from the above introduction that the embodiment of the present application can be applied to a smart door lock, and the smart door lock can be configured with an image acquisition device, which is used to acquire the scene image outside the door. Before acquiring the image to be detected, the embodiment of the present application can detect whether there is a human body within the preset range of the smart door lock. If it is determined that there is a human body within the preset range of the smart door lock, the image acquisition device can be started, and the image to be detected can be acquired using the image acquisition device.

[0119] Optionally, when detecting whether there is a human body within the preset range of the smart door lock, the embodiment of the present application may use an infrared detection device to determine whether there is a human body within the preset range of the smart door lock, or may use an ultrasonic detection device to determine whether there is a human body within the preset range of the smart door lock. As another way, when using an infrared detection device to determine whether there is a human body within the preset range of the smart door lock, the embodiment of the present application may also use an ultrasonic detection device to detect whether there is a human body within the preset range, and if so, it is determined that there is a human body within the preset range of the smart door lock. By combining an infrared detection device and an ultrasonic detection device, the embodiment of the present application can detect the human body more accurately, thereby preventing the image acquisition device from being mistakenly started.

[0120] As a method, when it is determined that there is a human body within the preset range of the smart door lock, the embodiment of the present application can also determine whether the human body is continuously within the preset range within a preset time period. If the human body is continuously within the preset range within the preset time period, the image acquisition device can be started; if the human body is not continuously within the preset range within the preset time period, the image acquisition device can be turned off.

[0121] As another way, if it is detected that there is no human body within the preset range of the smart door lock, the image acquisition device can be kept in a closed state, that is, the image acquisition device is not turned on. This can to a certain extent reduce the unnecessary power consumption of the smart door lock caused by the image acquisition device being in a long-term open state.

[0122] Step S320: Obtain the similarity between the target location feature and at least one location reference feature.

[0123] The position reference feature may be determined based on the position of the target reference object in a plurality of sample images.

[0124] Step S330: If the similarity reaches the corresponding similarity threshold, a warning message is generated according to the image to be detected and sent to the target terminal.

[0125] As a method, when it is determined that the target location feature is similar to at least one location reference feature, that is, when it is determined that the similarity reaches the corresponding similarity threshold, the embodiment of the present application can generate warning information based on the image to be detected, and then send the warning information to the target terminal. In addition, when sending the warning information to the target terminal, the embodiment of the present application can also send the image to be detected to the target terminal at the same time, so that the user can clearly know the actual situation of the door.

[0126] Optionally, when it is determined that the similarity reaches a corresponding similarity threshold, the embodiment of the present application can also obtain the video where the image to be detected is located, and send the video to the target terminal, so that it is convenient for users to view the videos before and after the image to be detected, so as to improve the user's experience. In addition, when it is determined that the similarity reaches a corresponding similarity threshold, the embodiment of the present application can also instruct the image acquisition device to record a video of a preset length, and the video of the preset length can be a video after the image to be detected, and then send the recorded video to the target terminal.

[0127] In some other embodiments, step S330 may include: Fig.10 Steps S331 to S333 are shown.

[0128] Step S331: If the similarity reaches the corresponding similarity threshold, the identity information of the target object corresponding to the target position feature is obtained.

[0129] As a method, when it is determined that the similarity reaches the corresponding similarity threshold, the embodiment of the present application can obtain the identity information of the target object corresponding to the target position feature. Specifically, the smart door lock can identify the clothing of the target object, and then determine the identity information of the target object based on the identification result.

[0130] In the embodiment of the present application, there may be a preset relationship between the clothing and identity information of the target object. For example, when the target object is a property, the corresponding clothing may be the first clothing; when the target object is a takeaway person, the corresponding clothing may be the second clothing. It can be seen that different clothing of the target object will also correspond to different identity information.

[0131] Step S332: Determine whether the target object is of a specified type based on the identity information of the target object. Objects of the specified type may have security risks.

[0132] As a method, after obtaining the identity information of the target object, the embodiment of the present application can determine whether the target object is of a specified type based on the identity information, wherein objects of the specified type have security risks.

[0133] Optionally, since food delivery personnel or property personnel usually wear designated clothes, the present application can determine whether the target object is of a designated type by identifying the target object's clothes, wherein the designated type refers to people who do not include food delivery personnel and property personnel. In addition, since food delivery personnel or property personnel are less dangerous, when determining that the target object is of these types, the present application embodiment may not send the image to be detected to the target terminal, thereby reducing the interference caused to the user by unnecessary information.

[0134] Step S333: If the target object belongs to the specified type, a warning message is generated according to the image to be detected and sent to the target terminal.

[0135] In the embodiment of the present application, when it is determined that the target object belongs to a specified type, the smart door lock can first generate warning information based on the target detection image, and then send the warning information to the target terminal, so as to ensure the safety of the user corresponding to the target terminal. It should be noted that when performing calculations such as target detection or identity detection, the present application can also send the image to be detected to the server to instruct the server to complete these detection operations, so as to reduce the calculation of the smart door lock and speed up the target detection rate.

[0136] In addition, if the similarity does not reach the corresponding similarity threshold, the warning information may not be generated, that is, the warning information will not be sent to the target terminal. At this time, the smart door lock can obtain the next image to be detected, and determine again whether the similarity of the next image to be detected reaches the similarity threshold, that is, the position feature of the next image to be detected is used as the target feature, and the similarity between the target position feature and at least one position reference feature is obtained.

[0137] A target detection method proposed in the present application is applied to a smart door lock, which can improve the accuracy of target detection to a great extent by matching the target position feature with at least one position reference feature. Specifically, the present application can obtain the image to be detected collected by the smart door lock, perform target detection on the target object in the image to be detected to obtain the target position feature corresponding to the target object, and then obtain the similarity between the target position feature and at least one position reference feature, wherein the position reference feature can be determined based on the position of the target reference object in multiple sample images, and if the similarity reaches the corresponding similarity threshold, an early warning message is generated according to the image to be detected and sent to the target terminal. The present application can improve the accuracy of target detection by matching the target position feature with at least one position reference feature using a smart door lock, thereby avoiding the image to be detected from being misidentified, that is, improving the user experience. In addition, the embodiment of the present application reduces the sending of unnecessary information by identifying the identity of the target object.

[0138] See also Fig.11 The present application embodiment proposes a target detection device 400, which is described by taking the device applied to a computer device (or electronic device) as an example. The computer device may be Figure 1 The electronic device in the apparatus may be a server, a gateway or a smart home device. The apparatus includes a first image acquisition module 410, a first similarity acquisition module 420 and a first result determination module 430.

[0139] The first image acquisition module 410 is used to acquire an image to be detected, perform target detection on a target object in the image to be detected, and obtain a target position feature corresponding to the target object.

[0140] The first similarity acquisition module 420 is used to acquire the similarity between the target position feature and at least one position reference feature, where the position reference feature is determined based on the position of the target reference object in multiple sample images.

[0141] Furthermore, the first similarity acquisition module 420 is further used to determine whether the similarity is greater than a similarity threshold; if so, it is determined that there is a target object in the image to be detected.

[0142] The first result determination module 430 is used to obtain a detection result based on the similarity and the corresponding similarity threshold.

[0143] Furthermore, the target detection device 400 further includes an information generation module and an information sending module. The information generation module is used to generate warning information according to the image to be detected if the detection result indicates that the target object exists in the image to be detected. The information sending module is used to send the warning information to the target terminal.

[0144] Furthermore, the target detection device 400 may also include: Fig.12 An image acquisition module 440 , a location extraction module 450 , and a clustering module 460 are shown.

[0145] The image acquisition module 440 is used to acquire a plurality of sample images, wherein the sample images include marked target reference objects.

[0146] The position extraction module 450 is used to extract sample target features corresponding to each target reference object from each sample image.

[0147] The clustering module 460 is used to perform clustering processing on each sample target feature and generate at least one position reference feature according to the central feature in each clustering result.

[0148] A target detection device proposed in the present application can improve the accuracy of target detection to a great extent by matching the target position feature with at least one position reference feature. Specifically, the present application can perform target detection on the target object in the acquired image to be detected to obtain the target position feature corresponding to the target object, and then obtain the similarity between the target position feature and at least one position reference feature, wherein the position reference feature can be determined based on the position of the target reference object in multiple sample images, and the detection result is obtained based on the similarity and the corresponding similarity threshold. The present application can improve the accuracy of target detection by matching the target position feature with at least one position reference feature, thereby avoiding the image to be detected from being misidentified, that is, improving the user experience.

[0149] See also Fig.13 An embodiment of the present application proposes a target detection device 500, which can be applied to a smart door lock. The device includes a second image acquisition module 520, a second similarity acquisition module 520 and a sending module 530.

[0150] The second image acquisition module 510 is used to acquire the image to be detected collected by the smart door lock, perform target detection on the target object in the image to be detected, and obtain the target position feature corresponding to the target object.

[0151] Furthermore, the target detection device 500 may also include a human body detection module and a start module. The human body detection module is used to detect whether there is a human body within a preset range of the smart door lock. The start module is used to start the image acquisition device if there is a human body, and the image acquisition device is used to acquire the image to be detected.

[0152] The second similarity acquisition module 520 is used to acquire the similarity between the target position feature and at least one position reference feature, where the position reference feature is determined based on the position of the target reference object in the multiple sample images.

[0153] The sending module 530 is used to generate warning information according to the image to be detected and send it to the target terminal if the similarity reaches the corresponding similarity threshold.

[0154] Furthermore, the sending module 530 is also used to obtain the identity information of the target object corresponding to the target position feature if the similarity reaches the corresponding similarity threshold; determine whether the target object belongs to a specified type based on the identity information of the target object, and objects of the specified type have safety hazards; if the target object belongs to the specified type, generate warning information based on the image to be detected and send it to the target terminal.

[0155] An embodiment of the present invention provides an electronic device, which includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement a target detection method provided in the above method embodiment.

[0156] In addition, the electronic device may be a smart door lock, and the smart door lock may be equipped with an image acquisition device, which may be used to acquire an image outside the door lock. In addition, the smart door lock may include a front silicone pad, a front bottom plate, a front limit cover, a bearing, a front middle frame, a reversing screw, a front cover assembly, a torsion spring, and a handle assembly.

[0157] The memory can be used to store software programs and modules. The processor executes various functional applications and target detection by running the software programs and modules stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, application programs required for functions, etc.; the data storage area can store data created according to the use of the device, etc. In addition, the memory can include a high-speed random access memory and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory can also include a memory controller to provide the processor with access to the memory.

[0158] Fig.14 is a hardware structure block diagram of an electronic device of a target detection method provided by an embodiment of the present invention. Fig.14As shown, the electronic device 1100 may have relatively large differences due to different configurations or performances, and may include one or more processors (Processing Units, CPU) 1110 (the processor 1110 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 1130 for storing data, and one or more storage media 1120 (such as one or more mass storage devices) for storing application programs 1123 or data 1122. Among them, the memory 1130 and the storage medium 1120 can be short-term storage or permanent storage. The program stored in the storage medium 1120 may include one or more modules, each of which may include a series of instruction operations on the server. Furthermore, the processor 1110 can be configured to communicate with the storage medium 1120 and execute a series of instruction operations in the storage medium 1120 on the electronic device 1100. The electronic device 1100 may also include one or more power supplies 1160, one or more wired or wireless network interfaces 1150, one or more input and output interfaces 1140, and / or one or more operating systems 1121, such as WindowSServerTM, MacOSXTM, UnixTM, LinuxTM, FreeBSDTM, etc.

[0159] The input / output interface 1140 may be used to receive or send data via a network. The specific example of the network may include a wireless network provided by a communication provider of the electronic device 1100. In one example, the input / output interface 1140 includes a network adapter (Network Interface Controller, NIC), which may be connected to other network devices via a base station so as to communicate with the Internet. In one example, the input / output interface 1140 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0160] It can be understood by those skilled in the art that Fig.14 The structure shown is only for illustration and does not limit the structure of the above electronic device. Fig.14 More or fewer components as shown, or with Fig.14 Different configurations are shown.

[0161] Fig.15 The hardware structure diagram of a mobile terminal for implementing various embodiments of the present invention is provided. The target terminal in the present invention is the mobile terminal.

[0162] The mobile terminal 1200 electronic device 1100 includes but is not limited to: a radio frequency unit 1210, a network module 1220, an audio output unit 1230, an input unit 1240, a sensor 1250, a display unit 1260, a user input unit 1270, an interface unit 1280, a memory 1290, a processor 1300, and a power supply 1310. Those skilled in the art can understand that the electronic device 1100 Fig.15 The mobile terminal structure shown in the figure does not constitute a limitation on the mobile terminal, and the mobile electronic device 1100 may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. In the embodiment of the electronic device 1100 of the present invention, the mobile terminal includes but is not limited to a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted mobile terminal, a wearable device of the electronic device 1100, and a pedometer, etc.

[0163] The processor 1300 is used to determine the target detection item; obtain the lock tongue status of the door lock; determine whether the target detection item and the lock tongue status are in a corresponding relationship; if not, determine that the door lock is faulty.

[0164] It should be understood that in the embodiment of the present invention, 1210 can be used for receiving and sending signals of the electronic device 1100 during the process of sending and receiving information or making calls. Specifically, after receiving the downlink data from the base station, it is sent to the processor 1300 for processing; in addition, the uplink data of the electronic device 1100 is sent to the base station. Generally, 1210 includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, etc. In addition, 1210 can also communicate with the network and other electronic devices 1100 through a wireless communication system.

[0165] The mobile terminal provides users with wireless broadband Internet access through the network module 1220, such as helping users to send and receive emails, browse web pages, and access streaming media.

[0166] The audio output unit 1230 can convert the audio data received by the mobile terminal 1210 or the network module 1220 or stored in the memory 1290 into an audio signal and output it as sound. Moreover, the audio output unit 1230 can also provide audio output related to a specific function performed by the mobile terminal 100 (for example, a call signal reception sound, a message reception sound, etc.). The audio output unit 1230 includes a speaker, a buzzer, a receiver, etc.

[0167] The input unit 1240 is used to receive audio or video signals. The input unit 1240 may include a graphics processor (Graphics Processing Unit, GPU) 1041 and a microphone 1242, and the graphics processor 1241 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The processed image frame can be displayed on the display unit 1260. The image frame processed by the graphics processor 1241 can be stored in the memory 1290 (or other storage medium) or sent via 1210 or the network module 1220. The microphone 1242 can receive sound and can process such sound into audio data. The processed audio data can be converted into a format output that can be sent to a mobile communication base station via 1210 in the case of a telephone call mode.

[0168] The mobile terminal 100 also includes at least one sensor 1250, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor includes an ambient light sensor and a proximity sensor, wherein the ambient light sensor can adjust the brightness of the display panel 1261 according to the brightness of the ambient light, and the proximity sensor can turn off the display panel 1261 and / or the backlight when the mobile terminal 100 is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in each direction (generally three axes), and can detect the magnitude and direction of gravity when stationary, which can be used to identify the posture of the mobile terminal (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; the sensor 1250 can also include a fingerprint sensor, a pressure sensor, an iris sensor, a molecular sensor, a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, etc., which will not be repeated here.

[0169] The display unit 1260 is used to display information input by the user or information provided to the user. The display unit 1260 may include a display panel 1261, which may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0170] The user input unit 1270 can be used to receive input digital or character information, and to generate key signal input related to the user settings and function control of the mobile terminal. Specifically, the user input unit 1270 includes a touch panel 1271 and other input devices 1272. The touch panel 1271, also known as a touch screen, can collect the user's touch operations on or near it (such as the user's operation on the touch panel 1271 or near the touch panel 1271 using any suitable object or accessory such as a finger or stylus). The touch panel 1271 may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into the coordinates of the contact point, and then sends it to the processor 1300, receives the command sent by the processor 1300 and executes it. In addition, the touch panel 1271 can be implemented in various types such as resistive, capacitive, infrared and surface acoustic wave. In addition to the touch panel 1271, the user input unit 1270 may also include other input devices 1272. Specifically, other input devices 1272 may include but are not limited to physical keyboards, function keys (such as volume control keys, switch keys, etc.), trackballs, mice, and joysticks, which are not described in detail here.

[0171] Furthermore, the touch panel 1271 may be overlaid on the display panel 1261. When the touch panel 1271 detects a touch operation on or near it, it transmits the information to the processor 1300 to determine the type of the touch event. Then, the processor 1300 provides a corresponding visual output on the display panel 1261 according to the type of the touch event. Fig.15 In the figure, the touch panel 1271 and the display panel 1261 are used as two independent components to implement the input and output functions of the mobile terminal. However, in some embodiments, the touch panel 1271 and the display panel 1261 can be integrated to implement the input and output functions of the mobile terminal, which is not limited here.

[0172] The interface unit 1280 is an interface for connecting an external device to the mobile terminal 100. For example, the external device may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device with an identification module, an audio input / output (I / O) port, a video I / O port, a headphone port, etc. The interface unit 1280 may be used to receive input (e.g., data information, power, etc.) from an external device and transmit the received input to one or more elements within the mobile terminal 100 or may be used to transmit data between the mobile terminal 100 and an external device.

[0173] The memory 1290 can be used to store software programs and various data. The memory 1290 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory 1290 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0174] The processor 1300 is the control center of the mobile terminal. It uses various interfaces and lines to connect various parts of the entire mobile terminal. It executes various functions of the mobile terminal and processes data by running or executing software programs and / or modules stored in the memory 1290, and calling data stored in the memory 1290, so as to monitor the mobile terminal as a whole. The processor 1300 may include one or more processing units; preferably, the processor 1300 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface and application programs, etc., and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 1300.

[0175] The mobile terminal 100 may also include a power supply 1310 (such as a battery) for supplying power to various components. Preferably, the power supply 1310 may be logically connected to the processor 1300 via a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system.

[0176] In addition, the mobile terminal 100 includes some functional modules not shown, which will not be described in detail here.

[0177] An embodiment of the present invention also provides a mobile terminal, including a processor 1300, a memory 1290, and a computer program stored in the memory 1290 and executable on the processor 1300. When the computer program is executed by the processor 1300, the various processes of the above-mentioned information method embodiment are implemented and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0178] The embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, each process of the above-mentioned information method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0179] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0180] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0181] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation modes, which are merely illustrative rather than restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims, all of which are within the protection of the present invention.

Claims

1. A target detection method, characterized in that: The method comprises: Acquire an image to be detected, perform target detection on a target object in the image to be detected, and obtain a target position feature corresponding to the target object, wherein the image to be detected is collected by a smart door lock or a smart robot; Acquire a similarity between the target position feature and at least one position reference feature, wherein the position reference feature is determined based on the position of the target reference object in a plurality of sample images; Based on the similarity and the corresponding similarity threshold, a detection result is obtained. If the detection result is that the similarity reaches the corresponding similarity threshold, a warning message is generated according to the image to be detected and sent to the target terminal.

2. The method according to claim 1, characterized in that: Before acquiring the image to be detected, the method further includes: Acquire the plurality of sample images, wherein the sample images include marked target reference objects; Extracting sample target features corresponding to each target reference object from each sample image; Clustering is performed on each of the sample target features, and the at least one position reference feature is generated according to a central feature in each clustering result.

3. The method according to claim 1, characterized in that The obtaining of a detection result based on the similarity and the corresponding similarity threshold comprises: Determining whether the similarity is greater than a similarity threshold; If it is greater than, it is determined that there is a target object in the image to be detected.

4. The method according to any one of claims 1 to 3, characterized in that: After obtaining the detection result based on the similarity and the corresponding similarity threshold, the method includes: If the detection result indicates that the target object exists in the image to be detected, generating warning information according to the image to be detected; The warning information is sent to the target terminal.

5. A target detection method, characterized in that: The method is applied to a smart door lock, and the method comprises: Acquire the image to be detected collected by the smart door lock, perform target detection on the target object in the image to be detected, and obtain the target position feature corresponding to the target object; Acquire a similarity between the target position feature and at least one position reference feature, wherein the position reference feature is determined based on the position of the target reference object in a plurality of sample images; If the similarity reaches the corresponding similarity threshold, a warning message is generated according to the image to be detected and sent to the target terminal.

6. The method according to claim 5, characterized in that The smart door lock includes an image acquisition device, and before acquiring the image to be detected acquired by the smart door lock, the method further includes: Detecting whether there is a human body within a preset range of the smart door lock; If it exists, the image acquisition device is started, and the image acquisition device is used to acquire the image to be detected.

7. The method according to claim 5, characterized in that If the similarity reaches a corresponding similarity threshold, generating warning information according to the image to be detected and sending it to the target terminal includes: If the similarity reaches the corresponding similarity threshold, then obtaining the identity information of the target object corresponding to the target location feature; Determine whether the target object belongs to a specified type according to the identity information of the target object, and whether the object of the specified type has a security risk; If the target object belongs to a specified type, a warning message is generated according to the image to be detected and sent to the target terminal.

8. A target detection device, characterized in that: The device comprises: A first image acquisition module is used to acquire an image to be detected, perform target detection on a target object in the image to be detected, and obtain a target position feature corresponding to the target object. The image to be detected is collected by a smart door lock or a smart robot; A first similarity acquisition module, configured to acquire a similarity between the target position feature and at least one position reference feature, wherein the position reference feature is determined based on the position of the target reference object in a plurality of sample images; The first result determination module is used to obtain a detection result based on the similarity and the corresponding similarity threshold. If the detection result is that the similarity reaches the corresponding similarity threshold, a warning message is generated according to the image to be detected and sent to the target terminal.

9. A target detection device, characterized in that: The device is applied to a smart door lock, and the device is applied to a smart door lock, and the device comprises: A second image acquisition module is used to acquire the image to be detected collected by the smart door lock, perform target detection on the target object in the image to be detected, and obtain a target position feature corresponding to the target object; A second similarity acquisition module is used to acquire the similarity between the target position feature and at least one position reference feature, where the position reference feature is determined based on the position of the target reference object in the plurality of sample images; The sending module is used to generate warning information according to the image to be detected and send it to the target terminal if the similarity reaches the corresponding similarity threshold.

10. An electronic device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the target detection method according to any one of claims 1 to 7.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the target detection method according to any one of claims 1 to 7 are implemented.

Images