Monitoring method, monitoring device and monitoring camera

Abstract

The present application relates to the field of security technology, and provides a monitoring method, a monitoring device, and a monitoring camera. The monitoring method includes obtaining an image to be detected of a target scene; identifying the image to be detected to obtain a first target object and a second target object, wherein the second target object is an object supported on the first target object; determining a dangerous area based on the second target object when the vertical projection of the second target object is within the first target object and the second target object is adjacent to any side of the first target object; and sending a first prompt message to a target terminal to prompt that the target scene has the dangerous area. The monitoring method of the present application can automatically and accurately identify safety hazards, and send the first prompt message corresponding to the dangerous area to the target terminal, thereby ensuring the accuracy of the monitoring results, and does not require manual monitoring by the user, thereby improving the user experience.

Description

Technical Field

[0001] The present application relates to the field of security technology, and in particular to a monitoring method, a monitoring device and a monitoring camera. Background Art

[0002] With the development of technology, more and more surveillance cameras are being used in the field of security monitoring. Users can view surveillance images or videos obtained by surveillance cameras through terminal devices to identify dangerous sources in the monitored scene. However, manually checking surveillance images and videos will take too much time and energy, and will inevitably lead to misreading and missing, resulting in poor monitoring results. Summary of the invention

[0003] The present application aims to solve at least one of the technical problems existing in the prior art. To this end, the present application proposes a monitoring method that can improve monitoring efficiency without manual monitoring, thereby reducing user burden and improving user experience.

[0004] The application also proposes a monitoring device.

[0005] The application also proposes a surveillance camera.

[0006] The application also provides an electronic device.

[0007] The present application also proposes a non-transitory computer-readable storage medium.

[0008] The present application also proposes a computer program product.

[0009] The monitoring method according to the first embodiment of the present application includes:

[0010] Acquire the image to be detected of the target scene;

[0011] Recognize the image to be detected to obtain a first target object and a second target object, wherein the second target object is an object supported on the first target object;

[0012] In a case where a vertical projection of the second target object is located within the first target object and the second target object is adjacent to any side of the first target object, determining a dangerous area based on the second target object;

[0013] A first prompt message is sent to the target terminal to prompt that the dangerous area exists in the target scene.

[0014] According to the monitoring method of the embodiment of the present application, the dangerous area is determined by identifying the first target object and the second target object in the detected image, which can automatically and accurately identify safety hazards and send the first prompt information corresponding to the dangerous area to the target terminal, thereby ensuring the accuracy of the monitoring results and eliminating the need for manual monitoring by users, thereby improving the user experience.

[0015] According to one embodiment of the present application, any side of the second target object adjacent to the first target object is determined by the following steps:

[0016] Determine the outline of the first target object as a first target area, determine a second target area corresponding to the second target object in the image to be detected, and the second target area is larger than the outline of the second target object;

[0017] In the case that at least a portion of the second target area exceeds the first target area, it is determined that the second target object is adjacent to any side of the first target object.

[0018] By identifying the outline of the first target object and determining the second target area corresponding to the second target object, the positional relationship between the first target object and the second target object can be accurately obtained, thereby improving the accuracy of the monitoring result.

[0019] According to an embodiment of the present application, determining a second target area corresponding to the second target object in the image to be detected includes:

[0020] Based on the second target object, determining a target point in the image to be detected, wherein the target point is located on a central axis of the second target object;

[0021] A circular area with the target point as the center and the target length as the radius is determined as the second target area, and the ratio of the farthest distance from the target point to the edge of the second target object to the target length is less than 1 and greater than the first target value.

[0022] By demarcating a circular area according to the target point on the central axis of the second target object to obtain the second target area, it can be ensured that a certain area around the second target object can be in a state of being monitored, thereby ensuring the comprehensiveness of the monitoring area.

[0023] According to one embodiment of the present application, any side of the second target object adjacent to the first target object is determined by the following steps:

[0024] determining the shortest distance from any side of the second target object to any side of the first target object;

[0025] When the shortest distance is less than a second target value, it is determined that the second target object is adjacent to any side of the first target object.

[0026] According to an embodiment of the present application, identifying the image to be detected to obtain the first target object and the second target object includes:

[0027] Inputting the image to be detected into a recognition model;

[0028] Acquire the first target object and the second target object output by the recognition model;

[0029] The recognition model is obtained by training with sample images in the target scene as samples and with annotation information of the sample first target object and the sample second target object in the sample images in the target scene as sample labels.

[0030] According to an embodiment of the present application, after determining the dangerous area based on the second target object, the method further includes:

[0031] In the case where a third target object is obtained by identifying the image to be detected, determining target position information of the third target object in the image to be detected;

[0032] Determining a distance between the third target object and the dangerous area based on the target position information and the position information of the dangerous area;

[0033] When the distance is less than the third target value, second prompt information is sent to the target terminal.

[0034] By monitoring the third target object, the second prompt information can be sent to the user's target terminal after the third target object enters the dangerous area, which can facilitate the user to instantly obtain the alarm information so that the user can take timely response measures.

[0035] A monitoring device according to an embodiment of the second aspect of the present application includes:

[0036] An acquisition module is used to acquire the image to be detected of the target scene;

[0037] an identification module, used to identify the image to be detected to obtain a first target object and a second target object, wherein the second target object is an object supported on the first target object;

[0038] A first processing module, configured to determine a dangerous area based on the second target object when a vertical projection of the second target object is located within the first target object and the second target object is adjacent to any side of the first target object;

[0039] The first sending module is used to send first prompt information to the target terminal to prompt that the dangerous area exists in the target scene.

[0040] According to the monitoring device of the embodiment of the present application, the dangerous area is determined by identifying the first target object and the second target object in the detected image, and the safety hazards can be automatically identified accurately and efficiently, and the first prompt information corresponding to the dangerous area is sent to the target terminal, thereby ensuring the accuracy of the monitoring results and eliminating the need for manual monitoring by users, thereby improving the user experience.

[0041] The surveillance camera according to the third aspect of the present application includes any one of the above-mentioned surveillance devices.

[0042] According to an electronic device of an embodiment of the fourth aspect of the present application, the electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the monitoring method described above is implemented.

[0043] According to the non-transitory computer-readable storage medium of the fifth aspect embodiment of the present application, a computer program is stored thereon, and when the computer program is executed by a processor, it implements any of the monitoring methods described above.

[0044] According to the computer program product of the sixth aspect of the present application, the computer program includes a computer program, and when the computer program is executed by a processor, it implements any of the monitoring methods described above.

[0045] The above one or more technical solutions in the embodiments of the present application have at least one of the following technical effects:

[0046] By identifying the first target object and the second target object in the image to be detected to determine the dangerous area, the safety hazard can be automatically and accurately identified efficiently, and the first prompt information corresponding to the dangerous area is sent to the target terminal, thereby ensuring the accuracy of the monitoring results and eliminating the need for manual monitoring by the user, thereby improving the user experience;

[0047] Furthermore, by identifying the outline of the first target object and determining the second target area corresponding to the second target object, the positional relationship between the first target object and the second target object can be accurately obtained, thereby improving the accuracy of the monitoring result;

[0048] Furthermore, by demarcating a circular area according to the target point on the central axis of the second target object to obtain the second target area, it can be ensured that a certain area around the second target object can be in a focused monitoring state, thereby ensuring the comprehensiveness of the monitoring area;

[0049] Furthermore, by monitoring the third target object, a second prompt message can be sent to the user's target terminal after the third target object enters the danger zone, so that the user can instantly obtain the alarm information and take timely response measures.

[0050] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0052] Figure 1 It is a flowchart of the monitoring method provided in the embodiment of the present application;

[0053] Figure 2 is a schematic diagram of a home scene image to be detected in the monitoring method provided in an embodiment of the present application;

[0054] Figure 3 is a schematic diagram of the structure of a monitoring device provided in an embodiment of the present application;

[0055] Figure 4 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0056] The following is a further detailed description of the implementation of the present application in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present application but cannot be used to limit the scope of the present application.

[0057] In the description of the embodiments of the present application, it should be noted that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, which are only for the convenience of describing the embodiments of the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the embodiments of the present application. In addition, the terms "first", "second", and "third" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance.

[0058] In the description of the embodiments of the present application, it should be noted that, unless otherwise clearly specified and limited, the terms "connected" and "connection" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms in the embodiments of the present application can be understood according to specific circumstances.

[0059] In the embodiments of the present application, unless otherwise clearly specified and limited, a first feature being "above" or "below" a second feature may mean that the first and second features are in direct contact, or the first and second features are in indirect contact through an intermediate medium. Moreover, a first feature being "above", "above" or "above" a second feature may mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is higher in level than the second feature. A first feature being "below", "below" or "below" a second feature may mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature is lower in level than the second feature.

[0060] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the embodiments of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0061] Combine the following Figure 1-Figure 4 The monitoring method, monitoring device and monitoring camera of the embodiments of the present application are described.

[0062] Reference Figure 1 The monitoring method of the embodiment of the present application includes step 110, step 120, step 130 and step 140.

[0063] The execution subject of the monitoring method in the embodiment of the present application may be a controller or a server. The monitoring method in the embodiment of the present application is described below based on the execution subject being the controller.

[0064] Step 110: Acquire an image to be detected of the target scene.

[0065] It should be noted that the image to be detected is a monitoring image under a target scene, and the target scene is a scene that the user needs to monitor.

[0066] The target scene may be located indoors, outdoors, or a combination of indoors and outdoors. There is no restriction on the location of the target scene.

[0067] There are multiple objects that can be monitored in the target scene. The monitored objects can be fixed facilities in the target scene, movable objects in the target scene, and of course, animals, humans or other intelligent entities that can move autonomously.

[0068] When monitoring a target scene, the target scene can be monitored through a surveillance camera or other equipment with photo taking and video recording functions.

[0069] Step 120: Identify the image to be detected to obtain a first target object and a second target object.

[0070] In the process of monitoring the target scene, the first target object and the second target object in the target scene are identified by performing image recognition on the image to be detected in the target scene.

[0071] The images to be detected can be collected at a certain frequency in order to identify each object in the target scene.

[0072] It should be noted that the second target object is an object supported and placed on the first target object, that is, the second target object is placed on the first target object, and the first target object supports the second target object.

[0073] For example, in a home scene, when the first target object is a table, the second target object may be a kettle, a teacup, or a computer placed on the table, etc. When the first target object is a kitchen stove, the second target object may be a dinner plate, a bowl, or a knife placed on the stove, etc.

[0074] For another example, in a warehouse scenario, when the first target object is a shelf, the second target object may be various goods placed on the shelf.

[0075] Of course, in the above scenarios and other scenarios, the first target object and the second target object can be determined according to monitoring needs, and there is no limitation on the first target object and the second target object.

[0076] It is understandable that there is a certain positional relationship between different objects. The first target object and the second target object satisfy the positional relationship that the second target object is placed on the first target object, and the first target object satisfies the condition of supporting the second target object.

[0077] When the positional relationship between the first target object and the second target object is in a safe state, the positional relationship between the first target object and the second target object is relatively stable and not easy to change, and the support of the first target object to the second target object is in a relatively stable state, and it is not easy to cause safety hazards or danger sources in the target scene.

[0078] For example, in a home scene, the monitored objects may include people at home, a table, a kettle placed on the table, and other objects on the edge of the table, etc. In this case, the first target object may be the table, and the second target object may be the kettle placed on the table.

[0079] The kettle filled with hot water is placed on the table away from the edge. The kettle is in a safe position. The kettle will not fall off the table during the user's normal drinking process to constitute a source of danger and cause damage to people and objects at the edge of the table.

[0080] When the positional relationship between the first target object and the second target object is in an unsafe state, the positional relationship between the first target object and the second target object is relatively unstable and easy to change, and the state of support provided by the first target object to the second target object is easy to change, which may easily cause safety hazards or danger sources in the target scene.

[0081] For example, in a home scenario, the second target object, a kettle filled with hot water, is placed on the edge of the first target object's table. The positional relationship between the kettle and the table is not in a safe state. If the kettle is touched, it is easy to fall off the table and cause damage to objects such as infants and young children near the edge of the table.

[0082] The monitoring method of the embodiment of the present application can be based on monitoring the position status between different objects in the target scene. When the first target object and the second target object are determined, the position status of the first target object and the second target object is monitored in detail to identify the danger source or safety hazard.

[0083] Step 130: When a vertical projection of the second target object is located within the first target object and the second target object is adjacent to any side of the first target object, determine a dangerous area based on the second target object.

[0084] It can be understood that the second target object is placed on the first target object, and the vertical projection of the second target object may be located within the first target object, and the vertical projection of the second target object may also be partially located within the first target object.

[0085] When the vertical projection of the second target object is located within the first target object, the positional relationship between the first target object and the second target object is extremely unstable and prone to change. The state of the first target object supporting the second target object is very easy to change, which is very likely to cause safety hazards or danger sources in the target scene.

[0086] For example, in a home scene, the first target object is a table, and the second target object is a laptop placed on the table. When the laptop is placed on the edge of the table and a part of the laptop exceeds the edge of the table, the laptop is easy to fall off the table and become a danger source.

[0087] In this case, the laptop is a very obvious source of danger, and the user can clearly perceive the danger and take certain protective measures.

[0088] Therefore, the monitoring method of the embodiment of the present application can perform monitoring and identification when the danger source cannot be obviously detected.

[0089] In this embodiment, a vertical projection of the second target object is located within the first target object, and the second target object is adjacent to any side of the first target object.

[0090] When determining that the second target object is adjacent to any side of the first target object, the determination may be performed in the following two ways.

[0091] 1. Delineate a second target area based on the second target object for confirmation.

[0092] In this embodiment, the outline of the first target object is determined as the first target area, and the second target area corresponding to the second target object in the image to be detected is determined.

[0093] It can be understood that, in the process of identifying the image to be detected, the contours of the first target object and the second target object can be identified, and the area where the contour of the first target object is located is the first target area.

[0094] When the first target object supports the second target object, the contour area of ​​the second target object does not exceed the first target area. When the position of the second target object changes, a certain range outside the contour area of ​​the second target object can be affected by the second target object.

[0095] Therefore, the second target area determined according to the second target object is larger than the outline of the second target object.

[0096] According to the monitoring method provided in the embodiment of the present application, by identifying the outline of the first target object and determining the second target area corresponding to the second target object, the positional relationship between the first target object and the second target object can be accurately obtained, thereby improving the accuracy of the monitoring results.

[0097] In some embodiments, the method for determining the second target area may include: determining a target point in the image to be detected based on the second target object.

[0098] It is understandable that the second target object is prone to generate a danger source when the state at the current position changes. In order to make the second target area corresponding to the second target object more comprehensively cover the area where the second target object may cause safety hazards, a certain area within the surrounding range of the second target object can be used as the second target area.

[0099] When determining the second target area, in order to adapt to different second target objects in different scenes, a target point may be selected on the second target object, and the target point may be located on the central axis of the second target object.

[0100] In this embodiment, after the target point is determined, a circular area with the target point as the center and the target length as the radius is determined as the second target area.

[0101] It is understandable that the second target area is larger than the area of ​​the vertical projection of the second target object on the first target object. In the case where the target length is larger than the farthest distance from the target point to the side of the second target object, the second area can cover the projection area of ​​the second target object. That is, the ratio of the farthest distance from the target point to the side of the second target object to the target length is less than 1.

[0102] In order to minimize the area of ​​the safe area included in the second target area, the ratio of the maximum distance from the target point to the edge of the second target object to the target length is greater than the first target value. The first target value can be determined according to the actual application scenario.

[0103] In some embodiments, the confirmation can be performed based on the sizes of the first target object and the second target object. For example, when the size of the second target object in the vertical direction is larger, the first target value can be determined as a smaller value such as 0.2.

[0104] In other embodiments, the second target object is confirmed based on the type of the danger source corresponding to the second target object. For example, when the second target object is a container containing a liquid that is easily spilled, the first target value can be determined as a smaller value such as 0.1.

[0105] Of course, the first target value may also be confirmed in other ways, which are not limited here.

[0106] When the second target area is determined, the second target area is compared with the first target area.

[0107] In the case where at least a portion of the second target area exceeds the first target area, it is determined that the second target object is adjacent to any side of the first target object.

[0108] In this case, the second target area corresponding to the second target object partially exceeds the first target area, and the second target object is adjacent to any side of the first target object. The influence range of the danger source that may be generated by the second target object will exceed the first target area, posing a greater safety hazard.

[0109] like Figure 2 As shown, in a home scene, the first target object may be a table, the first target area may be the area where the table top is located, and the second target object may be a kettle or a teacup placed on the table.

[0110] The second target area B corresponding to the kettle exceeds the first target area A where the desktop is located, that is, part of the second target area exceeds the first target area. In this case, it can be determined that the kettle is close to the edge of the desktop.

[0111] In this case, the kettle is likely to fall off the table and pose a safety hazard, and the second target area corresponding to the kettle is the dangerous area.

[0112] The second target area C corresponding to the teacup does not exceed the first target area A where the table is located, so the second target area C cannot be used as a dangerous area.

[0113] It is understandable that the second target object in the danger zone is prone to changes in position and state. When the second target object changes position or state, the second target object is likely to become a source of danger and a safety hazard is likely to occur in the second target zone. Therefore, the second target zone is determined as a danger zone.

[0114] According to the monitoring method provided in the embodiment of the present application, by demarcating a circular area according to the target point on the central axis of the second target object to obtain the second target area, it can be ensured that a certain area around the second target object can be in a state of key monitoring, thereby ensuring the comprehensiveness of the monitoring area.

[0115] 2. Confirm by the distance from the edge of the first target object.

[0116] In this embodiment, the positional relationship between the second target object and the first target object is directly confirmed by the shortest distance between the second target object and the edge of the first target object.

[0117] When the shortest distance from any side of the second target object to any side of the first target object is determined, the shortest distance is compared with the second target value.

[0118] When the shortest distance is less than the second target value, it is determined that the second target object is adjacent to any side of the first target object.

[0119] It is understandable that the second target value can be confirmed according to different first target objects and second target objects in actual application scenarios, and the confirmation method of the second target value is not limited here.

[0120] According to a monitoring method provided in an embodiment of the present application, the positional relationship between the second target object and the first target object is determined by confirming the shortest distance between the second target object and the edge of the first target object. This determination method is convenient and simple and can effectively improve the speed of identification and judgment during monitoring.

[0121] Step 140: Send a first prompt message to the target terminal to prompt that a dangerous area exists in the target scene.

[0122] It is understandable that the first prompt information may include the type of dangerous area and the location of the dangerous area. Of course, the information corresponding to the dangerous area may also include potential risks in the dangerous area. The content of the information corresponding to the dangerous area is not limited here.

[0123] The type and potential risk of the dangerous area can be determined based on the first target object and the second target object. Different first target objects and second target objects can correspond to dangerous areas of different types and potential risks.

[0124] For example, when the second target object is a cup or a kettle, the danger zone may be a danger zone where there is a risk of scalding. When the first target object is a table, the danger zone may be a danger zone where falling objects may injure people. Of course, the types and potential risks of the danger zone may also be other types and contents, which are not limited here.

[0125] In some embodiments, the type and potential risk of the dangerous area may also be determined according to different types of target scenes.

[0126] For example, in a home scenario, dangerous areas that pose safety risks to infants and young children can be identified. In a warehouse scenario, dangerous areas that may damage the goods themselves can be identified. Of course, the types and potential risks of dangerous areas can also be other types and contents, which are not limited here.

[0127] The location of the danger zone can be determined by marking it in the image under the target scene for easy viewing.

[0128] After determining the first prompt information, the first prompt information can be sent to the target terminal in real time. The first prompt information can be sent in the form of a picture or a combination of a picture and text. Of course, in some other embodiments, the sending form can also be a video, and the sending form is not limited here.

[0129] The target terminal is used to facilitate users to obtain the first prompt information corresponding to the dangerous area. The target terminal can be a mobile phone, a computer, a tablet, or other terminal devices. There is no restriction on the type of the target terminal here.

[0130] The user can obtain the first prompt information of the dangerous area in real time through the target terminal, so as to make corresponding prevention and response measures for the dangerous area in the first time.

[0131] In this embodiment, by identifying the first target object and the second target object in the image to be detected, when it is determined that the second target object is adjacent to any side of the first target object, the dangerous area where safety hazards exist is automatically determined based on the positional relationship between the first target area and the second target area, and then the first prompt information corresponding to the dangerous area is sent to the target terminal. The user can obtain the information corresponding to the dangerous area without self-monitoring, which can facilitate the user to timely obtain the potential risks in the monitored target scene.

[0132] According to the monitoring method provided in the embodiment of the present application, the dangerous area is determined by identifying the first target object and the second target object in the detected image, so that safety hazards can be automatically identified accurately and efficiently, and the first prompt information corresponding to the dangerous area is sent to the target terminal, thereby ensuring the accuracy of the monitoring results and eliminating the need for manual monitoring by users, thereby improving the user experience.

[0133] In some embodiments, step 120 recognizes the image to be detected to obtain the first target object and the second target object, including inputting the image to be detected into a recognition model.

[0134] It is understandable that during the monitoring process, the image to be detected can be input into the recognition model at a certain frequency.

[0135] The recognition model may include an instance segmentation neural network model, such as FCN, R-CNN, or YOLO series neural network models. There is no restriction on the specific type of neural network model used in the recognition model.

[0136] The recognition model can recognize the first target object and the second target object in the image to be detected, and divide the first target area and the second target area according to the first target object and the second target object, and determine the dangerous area.

[0137] The recognition model is obtained by training with sample images in the target scene as samples and with annotation information of the sample first target object and the sample second target object in the sample images in the target scene as sample labels.

[0138] In other words, the training samples of the recognition model are sample images in the target scene, and the training labels are the annotation information of the sample first target object and the sample second target object. The annotation information of the sample first target object and the sample second target object can be manually annotated in advance, or obtained by expanding the capacity of part of the sample set. Through the deep learning method, the accuracy and recognition type of recognition can be increased.

[0139] For example, in a home scene, first mark common home appliances in the home scene as the first target object of the sample, such as tables, chairs and other home appliances. Then mark the dangerous items that may occur in the scene, that is, mark the second target object of the sample, such as kettles, tea cups, etc.

[0140] It is understandable that, in an actual monitoring process, the recognition model can directly output a recognition result, namely, the first target object and the second target object, and segment the contours of the first target object and the second target object.

[0141] After acquiring the first target object and the second target object output by the recognition model, the recognition model may be further processed, such as determining the first target area and the second target area and determining the dangerous area.

[0142] According to the monitoring method provided in the embodiment of the present application, by inputting the image to be detected into the recognition model, the first target object and the second target object can be quickly and accurately identified, and then the first target area, the second target area and the dangerous area can be conveniently determined, that is, precise monitoring is conveniently achieved, and the accuracy and efficiency of monitoring are improved.

[0143] In some scenarios, when people with poor awareness of prevention enter dangerous areas, they are vulnerable to harm.

[0144] For example, in a home scene, the first target object is a table, and the second target object is a kettle placed on the table. When the kettle is close to the edge of the table, the kettle is likely to fall off the table and cause harm to people near the table.

[0145] For some people with poor awareness of prevention, such as infants and young children, it is necessary to monitor the position of the infants and young children for safety precautions.

[0146] Therefore, after step 130, the monitoring method of the embodiment of the present application further includes: in the case where the third target object is obtained by identifying the image to be detected, determining the position information of the third target object in the image to be detected.

[0147] The third target object is a living object, and the third target object may be a person or animal with poor awareness of prevention or difficulty in perceiving potential safety hazards, etc. For example, the third target object may be an infant, a pet, etc.

[0148] In some embodiments, the third target object can be detected by facial liveness detection or infrared liveness detection. Of course, in other embodiments, the preset third target object appearing in the monitoring range can also be identified by image recognition. The detection method of the third target object is not limited here.

[0149] When it is determined that the third target object appears within the monitoring range, the position information of the third target object in the image to be detected may be determined.

[0150] In this embodiment, the third target object can be detected by image recognition. During the training process of the recognition model of the third target object, the sample image includes an image containing a preset third target object, wherein the sample label of the recognition model during the training process includes the marking information of the sample third target object.

[0151] In the process of identifying the image to be detected, when the third target object is identified in the image to be detected, target position information of the third target object in the image to be detected is determined.

[0152] After determining the target position information of the third target object in the image to be detected, the distance between the third target object and the dangerous area is determined based on the target position information and the position information of the dangerous area.

[0153] It can be understood that, when the distance is less than the third target value, it can be determined that the third target object has entered the danger zone and there is a risk of the third target object being harmed.

[0154] It can be understood that the third target value can be determined according to different third target objects, target scenes and dangerous areas.

[0155] For example, in a home scene, the danger zone is the danger zone corresponding to the kettle on the table. When the third target object is a pet dog with a fast moving speed, the third target value can be set to a larger value, such as 1 meter.

[0156] In this case, the second prompt information can be sent to the target terminal according to the distance between the third target object and the dangerous area. The second prompt information can include the first prompt information corresponding to the dangerous area and the risk prompt information for the third target object, and the second prompt information is used to prompt the user that the third target object is in the dangerous area.

[0157] For example, in a home scene, the first target object is a table, the second target object is a kettle placed on the table, and the third target object may be an infant or a pet.

[0158] When the kettle is close to the edge of the table, it is easy to fall off the table and cause harm to infants or pets near the table.

[0159] When it is monitored that the distance between the infant and the dangerous area is less than the third target value, a second prompt message can be sent to the user's terminal. The second prompt message may include the distance between the infant and the dangerous area, as well as potential risk information of the infant being hit by a kettle or scalded by hot water.

[0160] The second prompt information may be sent in the form of a picture or a combination of a picture and text. Of course, in some other embodiments, the second prompt information may be sent in the form of a video. The sending form is not limited here.

[0161] According to the monitoring method provided in the embodiment of the present application, by monitoring the third target object, a second prompt message can be sent to the user's target terminal after the third target object enters the danger zone, which can facilitate the user to instantly obtain the alarm information so that the user can take timely response measures.

[0162] The monitoring device provided in the embodiment of the present application is described below. The monitoring device described below and the monitoring method described above can be referenced to each other.

[0163] Reference Figure 3 The monitoring device provided in the embodiment of the present application includes an acquisition module 310, an identification module 320, a first processing module 330 and a first sending module 340.

[0164] The acquisition module 310 is used to acquire the image to be detected of the target scene;

[0165] The recognition module 320 is used to recognize the image to be detected, and obtain a first target object and a second target object, where the second target object is an object supported on the first target object;

[0166] The first processing module 330 is used to determine the danger zone based on the second target object when the vertical projection of the second target object is located within the first target object and the second target object is adjacent to any side of the first target object;

[0167] The first sending module 340 is used to send first prompt information to the target terminal to prompt that there is a dangerous area in the target scene.

[0168] According to the monitoring device provided in the embodiment of the present application, the dangerous area is determined by identifying the first target object and the second target object in the detected image, and the safety hazards can be automatically identified accurately and efficiently, and the first prompt information corresponding to the dangerous area is sent to the target terminal, thereby ensuring the accuracy of the monitoring results and eliminating the need for manual monitoring by users, thereby improving the user experience.

[0169] In some embodiments, the first processing module 330 is also used to determine the outline of the first target object as the first target area, determine the second target area corresponding to the second target object in the image to be detected, and the second target area is larger than the outline of the second target object; when at least part of the second target area exceeds the first target area, determine that the second target object is adjacent to any side of the first target object.

[0170] According to the monitoring device provided in the embodiment of the present application, by identifying the outline of the first target object and determining the second target area corresponding to the second target object, the positional relationship between the first target object and the second target object can be accurately obtained, thereby improving the accuracy of the monitoring results.

[0171] In some embodiments, the first processing module 330 is also used to determine a target point in the image to be detected based on the second target object, where the target point is located on the central axis of the second target object; a circular area determined with the target point as the center and the target length as the radius is determined as the second target area, and the ratio of the farthest distance from the target point to the edge of the second target object to the target length is less than 1 and greater than the first target value.

[0172] According to the monitoring device provided in the embodiment of the present application, by demarcating a circular area according to the target point on the central axis of the second target object to obtain the second target area, it can be ensured that a certain area around the second target object can be in a state of key monitoring, thereby ensuring the comprehensiveness of the monitoring area.

[0173] In some embodiments, the first processing module 330 is further used to determine the shortest distance from any edge of the second target object to any edge of the first target object; when the shortest distance is less than the second target value, determine that the second target object is adjacent to any edge of the first target object.

[0174] According to the monitoring device provided in the embodiment of the present application, the positional relationship between the second target object and the first target object is determined by confirming the shortest distance between the second target object and the edge of the first target object. This determination method is convenient and simple and can effectively improve the speed of identification and judgment during monitoring.

[0175] In some embodiments, the recognition module 320 is also used to input the image to be detected into a recognition model; obtain the first target object and the second target object output by the recognition model; wherein the recognition model is trained by taking the sample image under the target scene as the sample, and taking the annotation information of the sample first target object and the sample second target object in the sample image under the target scene as the sample label.

[0176] According to the monitoring device provided in the embodiment of the present application, the first target object and the second target object can be quickly and accurately identified by inputting the image to be detected into the recognition model, thereby conveniently determining the first target area, the second target area and the dangerous area, that is, facilitating precise monitoring and improving the accuracy and efficiency of monitoring.

[0177] In some embodiments, the monitoring device further includes a second processing module, a third processing module and a second sending module.

[0178] The second processing module is used to determine the target position information of the third target object in the image to be detected when the third target object is obtained by identifying the image to be detected;

[0179] The third processing module is used to determine the distance between the third target object and the dangerous area based on the target position information and the position information of the dangerous area;

[0180] The second sending module is used to send second prompt information to the target terminal when the distance is less than the third target value.

[0181] According to the monitoring device provided in the embodiment of the present application, by monitoring the third target object, a second prompt message can be sent to the user's target terminal after the third target object enters the danger zone, which can facilitate the user to instantly obtain the alarm information so that the user can take timely response measures.

[0182] An embodiment of the present application also provides a monitoring camera, which may include the above-mentioned monitoring device.

[0183] It is understandable that the surveillance camera may also include a controller that executes the above-mentioned surveillance method. In this case, by identifying the first target object and the second target object in the image to be detected to determine the dangerous area, the safety hazard can be accurately and efficiently automatically identified, and the first prompt information of the dangerous area can be sent to the target terminal, thereby ensuring the accuracy of the monitoring results and eliminating the need for manual monitoring by the user, thereby improving the user experience.

[0184] Figure 4 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 4As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430 and a communication bus 440, wherein the processor 410, the communication interface 420 and the memory 430 communicate with each other through the communication bus 440. The processor 410 may call the logic instructions in the memory 430 to execute the monitoring method, which includes: acquiring an image to be detected of a target scene; identifying the image to be detected to obtain a first target object and a second target object, wherein the second target object is an object supported on the first target object; determining a dangerous area based on the second target object when the vertical projection of the second target object is located within the first target object and the second target object is adjacent to any side of the first target object; and sending a first prompt message to the target terminal to prompt that a dangerous area exists in the target scene.

[0185] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art, and the computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk.

[0186] Furthermore, the present application also provides a computer program product, which includes a computer program, and the computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the monitoring method provided by the above-mentioned method embodiments, and the method includes: acquiring an image to be detected of a target scene; identifying the image to be detected to obtain a first target object and a second target object, the second target object being an object supported on the first target object; determining a dangerous area based on the second target object when a vertical projection of the second target object is located within the first target object and the second target object is adjacent to any side of the first target object; and sending a first prompt information to the target terminal to prompt that a dangerous area exists in the target scene.

[0187] On the other hand, an embodiment of the present application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the monitoring method provided by the above-mentioned embodiments, the method comprising: acquiring an image to be detected of a target scene; identifying the image to be detected to obtain a first target object and a second target object, the second target object being an object supported on the first target object; determining a dangerous area based on the second target object when a vertical projection of the second target object is located within the first target object and the second target object is adjacent to any side of the first target object; and sending a first prompt information to the target terminal to prompt the existence of a dangerous area in the target scene.

[0188] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0189] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0190] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

[0191] The above implementation modes are only used to illustrate the present application, but not to limit the present application. Although the present application is described in detail with reference to the embodiments, a person skilled in the art should understand that various combinations, modifications or equivalent substitutions of the technical solutions of the present application do not depart from the spirit and scope of the technical solutions of the present application, and should be included in the scope of the claims of the present application.

Claims

1. A monitoring method, characterized in that: include: Acquire the image to be detected of the target scene; Recognize the image to be detected to obtain a first target object and a second target object, wherein the second target object is an object supported on the first target object; In a case where a vertical projection of the second target object is located within the first target object and the second target object is adjacent to any side of the first target object, determining a dangerous area based on the second target object; Sending a first prompt message to the target terminal to prompt that the dangerous area exists in the target scene; Any side of the second target object adjacent to the first target object is determined by the following steps: Determine the outline of the first target object as a first target area, determine a second target area corresponding to the second target object in the image to be detected, and the second target area is larger than the outline of the second target object; In a case where at least a portion of the second target area exceeds the first target area, determining that the second target object is adjacent to any side of the first target object; Alternatively, the second target object is adjacent to any side of the first target object by the following steps: determining the shortest distance from any side of the second target object to any side of the first target object; When the shortest distance is less than a second target value, it is determined that the second target object is adjacent to any side of the first target object.

2. The monitoring method according to claim 1, characterized in that: The determining a second target area corresponding to the second target object in the image to be detected includes: Based on the second target object, determining a target point in the image to be detected, wherein the target point is located on a central axis of the second target object; A circular area with the target point as the center and the target length as the radius is determined as the second target area, and the ratio of the farthest distance from the target point to the edge of the second target object to the target length is less than 1 and greater than the first target value.

3. The monitoring method according to claim 1 or 2, characterized in that: Identifying the image to be detected to obtain a first target object and a second target object includes: Inputting the image to be detected into a recognition model; Acquire the first target object and the second target object output by the recognition model; The recognition model is obtained by training with a sample image as a sample and with the annotation information of a sample first target object and a sample second target object in the sample image as a sample label.

4. The monitoring method according to claim 1 or 2, characterized in that: After determining the dangerous area based on the second target object, the method further includes: In the case where a third target object is obtained by identifying the image to be detected, determining target position information of the third target object in the image to be detected; Determining a distance between the third target object and the dangerous area based on the target position information and the position information of the dangerous area; When the distance is less than the third target value, second prompt information is sent to the target terminal.

5. A monitoring device, characterized in that: include; An acquisition module is used to acquire the image to be detected of the target scene; an identification module, used to identify the image to be detected to obtain a first target object and a second target object, wherein the second target object is an object supported on the first target object; A first processing module, configured to determine a dangerous area based on the second target object when a vertical projection of the second target object is located within the first target object and the second target object is adjacent to any side of the first target object; A first sending module, configured to send first prompt information to a target terminal to prompt that the target scene has the dangerous area; The first processing module is further used to determine the outline of the first target object as the first target area, determine the second target area corresponding to the second target object in the image to be detected, and the second target area is larger than the outline of the second target object; when at least part of the second target area exceeds the first target area, determine that the second target object is adjacent to any side of the first target object; Alternatively, the first processing module is further used to determine the shortest distance from any side of the second target object to any side of the first target object; when the shortest distance is less than the second target value, determine that the second target object is adjacent to any side of the first target object.

6. A surveillance camera, characterized in that: Comprising the monitoring device as claimed in claim 5.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the monitoring method according to any one of claims 1 to 4 is implemented.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the monitoring method according to any one of claims 1 to 4 is implemented.

9. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the monitoring method according to any one of claims 1 to 4 is implemented.

Images