Safety monitoring method, monitoring system and intelligent household electrical appliance

Abstract

The invention provides a safety monitoring method, a monitoring system and an intelligent household electrical appliance, and the method comprises the steps: obtaining the scene information of a target scene, the scene information comprising one or more of image information, sound information and environment parameter information; according to image information and / or sound information in the scene information, whether a specific object moves in the target scene or not is judged, and in response to the specific object moving in the target scene, a special monitoring mode is entered; judging whether a security risk occurs in the target scene according to the scene information; and in response to the safety risk, sending a risk warning signal. Based on the technical scheme of the invention, whether a specific object is moving and whether a security risk exists can be judged according to various scene information in the home scene, and when the security risk exists, the response is performed at the first time and the warning signal is sent, so that the home security is ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of home safety monitoring, and in particular to a safety monitoring method, a monitoring system and a smart home appliance. Background Art

[0002] At present, people are paying more and more attention to the safety of their homes and the monitoring of safety. For the monitoring of home safety, usually some cameras are set up for monitoring. However, the cameras can only collect images in real time, which can be used as a basis for judging whether there are safety problems, but they cannot actually judge whether there are safety problems. Therefore, once a safety problem occurs, the current monitoring methods cannot respond immediately, but can only play the role of recording the problem.

[0003] Take the kitchen area, where safety issues are most common, as an example. Since the kitchen is a place where open flames can be generated, once human negligence occurs, it often becomes the source of residential fires. For high-rise residential buildings, it is difficult for people to escape in the event of a fire, so fire safety monitoring in the kitchen is very important. The fire alarms currently installed in homes can only respond to fires of a certain scale, and cannot monitor and respond to the initial stages of a fire. In addition, if some people who live alone or are alone at home suddenly fall down while at home, even if there are cameras for monitoring, it is not easy to be discovered in the first place, resulting in a lack of timely assistance.

[0004] Based on the above, current monitoring methods cannot respond to safety issues in home environments in the first time, so it is necessary to combine the increasingly developed big data, Internet of Things and artificial intelligence technologies to solve this problem. Summary of the invention

[0005] In order to solve the problem that existing home safety monitoring means cannot respond to safety risks in the first time, the present invention proposes a safety monitoring method, a monitoring system and a smart home appliance.

[0006] In a first aspect, the present invention provides a safety monitoring method, which comprises the following steps:

[0007] Acquire scene information of a target scene, wherein the scene information includes one or more of image information, sound information, and environmental parameter information;

[0008] Determine whether a specific object is active in the target scene according to the image information and / or sound information in the scene information, and enter a special monitoring mode in response to the specific object being active in the target scene;

[0009] Determining whether a security risk occurs in the target scene according to the scene information;

[0010] In response to the security risk that occurs, a risk warning signal is sent.

[0011] In one embodiment, the environmental parameter information includes one or more of temperature information, smoke concentration information, and dust concentration information.

[0012] In one embodiment, judging whether a specific object is active in the target scene according to the image information and / or sound information in the scene information includes the following sub-steps:

[0013] Determining, based on the image information and / or sound information in the scene information, whether there is an active object in the target scene that matches the specific object in category;

[0014] In response to the presence of an active object in the target scene that matches the specific object in category, comparing the image features and / or sound features of the active object with the pre-stored image features and / or sound features of the specific object;

[0015] According to the comparison result, it is determined whether a specific object is active in the target scene.

[0016] In one embodiment, in response to the specific object being active in the target scene, entering a special monitoring mode includes the following sub-steps:

[0017] Determining, based on the image information, a distance between the specific object and a risk device in the target scene;

[0018] In response to the specific object entering the warning range corresponding to the risk device, determining the risk level according to the distance value between the specific object and the risk device, wherein the smaller the distance value, the higher the risk level;

[0019] According to the risk level, the operating state of the risk equipment is controlled and a risk warning signal of a corresponding level is issued.

[0020] In one embodiment, according to the risk level, the operating state of the risk equipment is controlled and a risk warning signal of the corresponding level is issued, including the following sub-steps:

[0021] In response to the risk level being a primary risk, maintaining the operation of the risk device, sending a voice warning message to the specific object, and sending a risk warning message to an associated object associated with the specific object;

[0022] In response to the risk level being a medium risk, shutting down the risk device, sending a voice warning message to the specific object, and sending a risk warning message to an associated object associated with the specific object;

[0023] In response to the risk level being a high risk, the risk device is turned off, a voice warning message is sent to the specific object, a risk warning message is sent to an associated object associated with the specific object, and a medical assistance message is sent to a predetermined medical institution.

[0024] In one embodiment, in response to the specific object being active in the target scene, entering a special monitoring mode includes the following sub-steps:

[0025] In response to the specific object being a person, determining current positions of a plurality of predetermined feature points of a human skeleton according to the image information;

[0026] Determining whether there is a risk of a person falling according to the current positions of the plurality of predetermined feature points;

[0027] In response to a risk situation where a person falls, a risk warning signal is sent.

[0028] In one embodiment, judging whether there is a risk of a person falling according to the current positions of the plurality of predetermined feature points comprises the following sub-steps:

[0029] According to the fact that the head feature points, the neck feature points and the hip feature points are in the same plane, and the trend characteristics of the plane match the trend characteristics of the ground of the target plane, it is determined that there is a risk of a person falling, and the person is in a lying position;

[0030] According to the fact that the hip feature point and the foot feature point are in the same plane, the trend characteristics of the plane match the trend characteristics of the ground of the target scene, and the connecting straight line connecting the hip feature point to the head feature point and the neck feature point respectively, the angle between the connecting straight line and the plane is not greater than 90°, it is judged that there is a risk of a person falling, and the person is in a sitting position.

[0031] In one embodiment, in response to the specific object being active in the target scene, entering a special monitoring mode includes the following sub-steps:

[0032] Acquire a sound signal of the specific object in the sound information;

[0033] Analyze the sound signal to determine whether the specific object is making a distress call or making a distress call with a decibel or frequency higher than a preset value;

[0034] In response to the specific object emitting a distress voice or a distress sound, a risk warning signal is sent.

[0035] In one embodiment, judging whether a security risk occurs in the target scene according to the scene information includes the following sub-steps:

[0036] Determine, based on the image information in the scene information, whether there is a fire-using device in operation in the target scene and the flame range of the fire-using device;

[0037] In response to the flame range of the fire-using equipment exceeding the spatial range encompassed by the fire-using equipment, it is determined that a safety risk occurs in the target scene.

[0038] In one embodiment, judging whether a security risk occurs in the target scene according to the scene information includes the following sub-steps:

[0039] Determining whether there is any fire-using equipment in operation in the target scene according to the image information in the scene information;

[0040] Determine whether there is dust approaching the fire-using equipment in operation based on the image information in the scene information or further combined with the dust concentration information in the environmental parameter information;

[0041] In response to dust approaching the fire-using equipment in operation, it is determined that a safety risk has occurred in the target scene.

[0042] In a second aspect, the present invention provides a monitoring system, characterized in that it includes:

[0043] An information acquisition device, which is used to acquire scene information of a target scene;

[0044] an object detection device, which is used to determine whether a specific object is active in the target scene based on the image information and / or sound information in the scene information, and enter a special monitoring mode in response to the specific object being active in the target scene;

[0045] a risk determination device, which is used to determine whether a safety risk occurs in the target scene according to the scene information; and

[0046] The risk warning device is used to send a risk warning signal in response to a safety risk that occurs.

[0047] In a third aspect, the present invention proposes a smart home appliance, which includes the above-mentioned monitoring system and thus has all the technical effects thereof.

[0048] The above technical features can be combined in various suitable ways or replaced by equivalent technical features as long as the purpose of the present invention can be achieved.

[0049] The security monitoring method, monitoring system and smart home appliance provided by the present invention have at least the following beneficial effects compared with the prior art:

[0050] A security monitoring method, monitoring system and smart home appliance of the present invention can determine whether there are specific objects active and whether there are security risks through various scene information in the home scene. When there is a security risk, it can respond and send a warning signal in the first time to ensure the safety of the home. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The present invention will be described in more detail below based on embodiments and with reference to the accompanying drawings, wherein:

[0052] Figure 1 The main flow chart of the safety monitoring method of the present invention is shown;

[0053] Figure 2 A flowchart of the safety monitoring method in Example 1 of the present invention is shown;

[0054] Figure 3 The human body standing profile diagram corresponding to the embodiment 2 of the present invention is shown;

[0055] Figure 4 The human skeleton feature point map corresponding to the embodiment 2 of the present invention is shown;

[0056] Figure 5 The human skeleton feature point map in a sitting position corresponding to the embodiment 2 of the present invention is shown;

[0057] Figure 6 The characteristic point map of the human skeleton in a lying position corresponding to the embodiment 2 of the present invention is shown;

[0058] Figure 7 shows a hardware module block diagram of a monitoring system in Embodiment 4 of the present invention;

[0059] Figure 8 A schematic diagram showing one implementation mode when the smart home appliance in Example 5 of the present invention is a refrigerator;

[0060] Fig. 9 A schematic diagram showing another implementation method when the smart home appliance in Example 5 of the present invention is a refrigerator.

[0061] In the drawings, like reference numerals are used for like parts. The drawings are not necessarily to scale.

[0062] Reference numerals:

[0063] 1-head feature points, 2-neck feature points, 3-hip feature points, 4-foot feature points, 5-camera, 6-microphone, 7-speaker, 8-sensor. DETAILED DESCRIPTION

[0064] The present invention will be further described below in conjunction with the accompanying drawings.

[0065] Example 1

[0066] An embodiment of the present invention provides a safety monitoring method, which includes the following steps:

[0067] Step S100: Acquire scene information of a target scene, wherein the scene information includes one or more of image information, sound information, and environmental parameter information, and the environmental parameter information includes one or more of temperature information, smoke concentration information, and dust concentration information.

[0068] Specifically, depending on the target scene, the scene information may include image information obtained through a camera, sound information obtained through a microphone, and may also include environmental parameter information obtained through different sensors, such as temperature, smoke concentration, dust concentration, etc.

[0069] Step S200: judging whether a specific object is active in the target scene according to the image information and / or sound information in the scene information, and entering a special monitoring mode in response to the specific object being active in the target scene.

[0070] Specifically, in this embodiment, it can be determined based on the scene information whether there are corresponding specific objects in the target scene. The specific objects in this embodiment can be people with high security risks divided according to family members, such as the elderly, children or the disabled. Special monitoring mode needs to be adopted for such people.

[0071] Step S200 further includes the following sub-steps:

[0072] Step S210: judging whether there is an active object matching the specific object in category in the target scene according to the image information and / or sound information in the scene information.

[0073] Step S220: In response to the presence of an active object in the target scene that matches the specific object in category, the image features and / or sound features of the active object are compared with the pre-stored image features and / or sound features of the specific object.

[0074] Step S230: According to the comparison result, determine whether there is a specific object active in the target scene.

[0075] Specifically, in this embodiment, the judgment of whether there is a specific object in the target scene is mainly divided into two parts. First, it is judged whether there is an active object that matches the specific object in category. Taking the specific object as a group of people with high security risks as an example, firstly, it is judged whether there are active people in the target scene based on image information or sound information. This step is to exclude interference such as pets or some intelligent robots that can move. Then, by comparing the image or sound of the active object in the target scene with the pre-stored image or sound of the specific object, it is judged whether the active object is the corresponding specific object.

[0076] Step S300: Determine whether a security risk occurs in the target scene based on the scene information.

[0077] Step S310: Determine the distance between the specific object and the risk device in the target scene according to the image information.

[0078] Step S320: In response to the specific object entering the warning range corresponding to the risk device, the risk level is determined according to the distance value between the specific object and the risk device, wherein the smaller the distance value, the higher the risk level.

[0079] Specifically, this embodiment is mainly aimed at children in the high-risk group or other people with imperfect safety awareness, and their activities near some areas with safety risks need to be monitored. Taking the target scene as a kitchen, the area of ​​safety risks in the kitchen includes the area near the stove, which is the corresponding risk equipment, and it is necessary to monitor whether the specific object is close to the stove. Therefore, the position of the specific object is determined based on the image information to determine whether it enters the warning range with safety risks around the risk equipment. Once the specific object enters the warning range, it indicates that there is a safety risk; and the closer the specific object is to the risk equipment, the higher the level of safety risk.

[0080] Step S400: In response to the security risk that occurs, a risk warning signal is sent.

[0081] Step S410: According to the risk level, the operating state of the risk equipment is controlled, and a risk warning signal of the corresponding level is issued.

[0082] Step S411: In response to the risk level being primary risk, the risk device is kept running, a voice warning message is sent to a specific object, and a risk warning message is sent to an associated object associated with the specific object.

[0083] Step S412: In response to the risk level being a medium risk, shutting down the risk device, sending a voice warning message to the specific object, and sending a risk warning message to the associated objects associated with the specific object.

[0084] Step S413: In response to the risk level being a high risk, the risk device is turned off, a voice warning message is sent to the specific object, a risk warning message is sent to the associated object associated with the specific object, and a medical assistance message is sent to a predetermined medical institution.

[0085] Specifically, referring to the accompanying drawings Figure 2 , take the case of children and other people with incomplete safety awareness approaching a burning stove in the kitchen as an example: when the distance between the child and the stove is 1 to 1.2 meters, a voice warning is played to the child through the speaker to remind the child to keep a safe distance; at the same time, a call is made to the child's guardian (i.e., the associated object) to remind the risk. When the distance between the child and the stove is within 1 meter (e.g., 0.5 to 1 meter), the stove is first turned off, and then a voice warning is played to the child through the speaker to remind the child to keep a safe distance; at the same time, a call is made to the child's guardian (i.e., the associated object) to remind the risk. When the distance between the child and the stove is close to 0 (i.e., direct contact), the stove is first turned off, and then combined with whether there is a child crying or screaming in the sound information, and whether the temperature of the child's body part closest to the stove detected by the infrared sensor is much higher than the normal temperature, it is determined whether it is a high-level risk; under high-level risk, a voice warning is played to the child through the speaker to remind the child to keep a safe distance, and the child is comforted verbally; in addition, a call is made to the child's guardian (i.e., the associated object) to remind the risk, and a medical assistance call is made to the medical institution.

[0086] Example 2

[0087] An embodiment of the present invention provides a safety monitoring method, which includes the following steps:

[0088] Step S100: Acquire scene information of a target scene, wherein the scene information includes one or more of image information, sound information, and environmental parameter information, and the environmental parameter information includes one or more of temperature information, smoke concentration information, and dust concentration information.

[0089] Specifically, depending on the target scene, the scene information may include image information obtained through a camera, sound information obtained through a microphone, and may also include environmental parameter information obtained through different sensors, such as temperature, smoke concentration, dust concentration, etc.

[0090] Step S200: judging whether a specific object is active in the target scene according to the image information and / or sound information in the scene information, and entering a special monitoring mode in response to the specific object being active in the target scene.

[0091] Specifically, in this embodiment, it can be determined based on the scene information whether there are corresponding specific objects in the target scene. The specific objects in this embodiment can be people with high security risks divided according to family members, such as the elderly, children or the disabled. Special monitoring mode needs to be adopted for such people.

[0092] Step S200 further includes the following sub-steps:

[0093] Step S210: judging whether there is an active object matching the specific object in category in the target scene according to the image information and / or sound information in the scene information.

[0094] Step S220: In response to the presence of an active object in the target scene that matches the specific object in category, the image features and / or sound features of the active object are compared with the pre-stored image features and / or sound features of the specific object.

[0095] Step S230: According to the comparison result, determine whether there is a specific object active in the target scene.

[0096] Specifically, in this embodiment, the judgment of whether there is a specific object in the target scene is mainly divided into two parts. First, it is judged whether there is an active object that matches the specific object in category. Taking the specific object as a group of people with high security risks as an example, firstly, it is judged whether there are active people in the target scene based on image information or sound information. This step is to exclude interference such as pets or some intelligent robots that can move. Then, by comparing the image or sound of the active object in the target scene with the pre-stored image or sound of the specific object, it is judged whether the active object is the corresponding specific object.

[0097] Step S300: Determine whether a security risk occurs in the target scene based on the scene information.

[0098] Step S310: In response to the specific object being a person, current positions of a plurality of predetermined feature points of the human skeleton are determined according to the image information.

[0099] Step S320: Determine whether there is a risk of a person falling according to the current positions of the plurality of predetermined feature points.

[0100] Specifically, this embodiment is mainly used to monitor the risk of falling of a specific object, and the specific object in this embodiment is a family member. Figure 3 and Figure 4 As shown, a plurality of feature points can be extracted from the human skeleton according to different parts of the trunk and joint points. This embodiment can use some of the feature points to determine the situation where a person falls.

[0101] Step S321: Based on the fact that the head feature point 1, the neck feature point 2 and the hip feature point 3 are in the same plane, and the trend characteristics of the plane match the trend characteristics of the ground of the target plane, it is determined that there is a risk of a person falling, and the person is in a lying position.

[0102] Step S322: Based on the fact that the hip feature point 3 and the foot feature point 4 are in the same plane, the trend characteristics of the plane match the trend characteristics of the ground of the target scene, and the connecting straight line connecting the hip feature point 3 to the head feature point 1 and the neck feature point 2 respectively, the angle between the connecting straight line and the plane is not greater than 90°, it is determined that there is a risk of a person falling, and the person is in a sitting position.

[0103] Specifically, as shown in the attached figure Figures 4 to 6 As shown, this embodiment extracts the head feature point 1, the neck feature point 2, the hip feature point 3 and the foot feature point 4 from the feature points of the human skeleton to determine the specific situation of the fall. Figure 6 When the head feature point 1, the neck feature point 2, and the hip feature point 3 are located on the same horizontal plane, and the trend of the plane is basically consistent with the ground, it can be determined that the human body is lying on the ground. Figure 5 When the hip feature point 3 and the foot feature point 4 are in the same plane, and the trend of the plane is basically consistent with the ground, and the angle between the connecting line connecting the hip feature point 3 to the head feature point 1 and the neck feature point 2 and the plane where the hip feature point 3 and the foot feature point 4 are located (which can be approximately regarded as the angle between the upper torso and the lower torso of the human body) is not greater than 90°, the human body is judged to be sitting on the ground, which is mainly used to eliminate misjudgment when a person is squatting.

[0104] Step S400: In response to the security risk that occurs, a risk warning signal is sent.

[0105] Step S410: In response to a risk of a person falling, a risk warning signal is sent. The risk warning signal here mainly includes calling other family members associated with the fallen person and sending a warning sound signal to attract the attention of the property or neighbors.

[0106] In addition, in this embodiment, step S300 and step S400 may also be implemented as follows:

[0107] Step S300: Determine whether a security risk occurs in the target scene based on the scene information.

[0108] Step S310: Acquire the sound signal of the specific object in the sound information.

[0109] Step S320: Analyze the sound signal to determine whether the specific object is making a call for help (here refers to language with clear meaning, such as "help" or "someone come") or whether it is making a call for help with a decibel or frequency higher than a preset value (here mainly refers to sounds without clear meaning, such as screams, cries; or sounds with ambiguous meanings, such as "ah" or "oops").

[0110] Step S300: In response to the security risk that occurs, a risk warning signal is sent.

[0111] Step S310: In response to a specific object sending out a voice or sound for help, sending a risk warning signal.

[0112] Specifically, when some short and repeated sound signals as listed in step S320 are monitored, it is determined that it is a cry for help that indicates a safety risk. Of course, the cry for help or the voice for help can also be preset with some specific sentences to trigger the alarm mechanism of the safety risk, such as "help", "alarm", "need help" or some specific sounds without text meaning.

[0113] In addition, when an emergency occurs, remote guidance can be introduced with one click. When remote guidance is activated, the remote guidance personnel can see the situation of the entire target scene through the camera, and can provide on-site personnel with operating instructions for equipment or devices, psychological comfort, emergency collaborative processing, and other operations.

[0114] Example 3

[0115] An embodiment of the present invention provides a safety monitoring method, which includes the following steps:

[0116] Step S100: Acquire scene information of a target scene, wherein the scene information includes one or more of image information, sound information, and environmental parameter information, and the environmental parameter information includes one or more of temperature information, smoke concentration information, and dust concentration information.

[0117] Specifically, depending on the target scene, the scene information may include image information obtained through a camera, sound information obtained through a microphone, and may also include environmental parameter information obtained through different sensors, such as temperature, smoke concentration, dust concentration, etc.

[0118] Step S200: judging whether a specific object is active in the target scene according to the image information and / or sound information in the scene information, and entering a special monitoring mode in response to the specific object being active in the target scene.

[0119] Step S300: Determine whether a security risk occurs in the target scene based on the scene information.

[0120] In this embodiment, fire monitoring is performed through the following steps S310 and S320, and dust explosion monitoring is performed through steps S330 to S350. Among them, the entirety of step S310 and step S320 and the entirety of step S330 to step S350 are two independent sections, and the step numbers are only for the convenience of description, and there is no order or progressive relationship.

[0121] Fire monitoring:

[0122] Step S310: Determine whether there is any fire-using equipment in operation and the flame range of the fire-using equipment in the target scene according to the image information in the scene information.

[0123] Step S320: In response to the flame range of the fire-using equipment exceeding the spatial range encompassed by the fire-using equipment, it is determined that a safety risk has occurred in the target scene.

[0124] Dust explosion monitoring:

[0125] Step S330: Determine whether there is any fire-using equipment in operation in the target scene based on the image information in the scene information.

[0126] Step S340: Determine whether there is dust approaching the fire-using equipment in operation based on the image information in the scene information or further combined with the dust concentration information in the environmental parameter information.

[0127] Step S350: In response to dust approaching the fire-using equipment in operation, it is determined that a safety risk has occurred in the target scene.

[0128] Step S400: In response to the security risk that occurs, a risk warning signal is sent.

[0129] It should be noted that the monitoring scheme in this embodiment is mainly used for general monitoring, which can be implemented when there are no specific objects in the target scene, that is, implemented outside the special monitoring mode; it can also be implemented when there are specific objects in the target scene, that is, implemented simultaneously in the special monitoring mode.

[0130] Example 4

[0131] An embodiment of the present invention provides a monitoring system, which includes:

[0132] An information acquisition device, which is used to acquire scene information of a target scene, wherein the scene information includes one or more of image information, sound information, and environmental parameter information;

[0133] An object detection device, which is used to determine whether a specific object is active in the target scene based on image information and / or sound information in the scene information, and enter a special monitoring mode in response to the specific object being active in the target scene;

[0134] A risk determination device, which is used to determine whether a safety risk occurs in a target scene according to the scene information; and

[0135] The risk warning device is used to send a risk warning signal in response to a safety risk that occurs.

[0136] Specifically, the monitoring system of this embodiment can implement the monitoring method of any of the above embodiments. Figure 7 As shown, the information acquisition device (camera, microphone, temperature sensor, smoke sensor) and risk warning device (speaker, wireless (3G / 4G / 5G) communication module) in the monitoring system are set on the corresponding main control board (the main control board is integrated with the main controller as a risk judgment device); the main control board is also preset with local large models of image recognition and voice recognition. The main control board is connected to the user's home wireless router through a network cable or wireless WiFi at the same time, and the wireless router is connected to the cloud server through a network cable. The AI ​​large model of image recognition and voice recognition can also run on the cloud server. The cloud server will return the recognition result to the main control board, and the main control board will perform corresponding actions according to the return result, such as warning broadcast, calling 119 for alarm, calling 120 for help, etc. IOT devices such as natural gas switches and sprinkler switches are connected to the cloud server through 3G / 4G / 5G, or connected to the cloud server after connecting to the home wireless router through WiFi. The cloud server will control the opening and closing of the IOT devices that have been connected to the network.

[0137] In addition, the judgment of scenes such as fire, smoke, falls, injuries (such as burns caused by contact with stoves, cuts caused by using knives, etc.) is judged by a graphic big data model, which can be a local graphic big model or a cloud AI model. The AI ​​model can be a CNN type or a Transformer architecture type. Accidents such as falls and injuries can also be judged by the voice picked up by the microphone to determine whether there is an emergency call from the user or a scream after being injured. The recognition of voice is judged by a local voice big model or a cloud AI voice big model. The AI ​​model can be an LLM type or a Transformer architecture type. In addition, fire monitoring or monitoring of other unexpected situations can also use temperature sensors to monitor the ambient temperature of the target scene. If the ambient temperature is detected to exceed the set threshold, the main controller is notified by interruption for processing. For smoke monitoring, when the smoke sensor is working, if the smoke concentration is detected to exceed the set threshold, the main controller is notified by interruption for processing.

[0138] The logical relationship between the relevant physical perception layer and the execution layer is shown in Table 1 below:

[0139]

[0140] Example 5

[0141] The embodiment of the present invention provides a smart home appliance, which includes the monitoring system of embodiment 4. Figure 8 and Fig. 9 As shown, the smart home appliance in this embodiment is a refrigerator, and the main control board in the monitoring system of Embodiment 4 and the main control board of the refrigerator, the camera 5, the microphone 6, the speaker 7, and the sensor 8 are correspondingly arranged on the top of the refrigerator. Figure 8 and Fig. 9 The main difference lies in whether the camera 5 is built-in or external. In addition, the camera 5, microphone 6, speaker 7, sensor 8 and other components can also be set in other positions of the refrigerator, such as the bottom, side, etc.

[0142] At the same time, according to needs, smart home appliances can also be dishwashers, range hoods, stoves in the kitchen, or televisions, washing machines, air conditioners and other appliances in the living room or other areas.

[0143] In the description of the present invention, it is necessary to understand that the terms "upper", "lower", "bottom", "top", "front", "back", "inside", "outside", "left", "right", etc. indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0144] Although the present invention is described herein with reference to specific embodiments, it should be understood that these embodiments are merely examples of the principles and applications of the present invention. It should therefore be understood that many modifications may be made to the exemplary embodiments and that other arrangements may be devised without departing from the spirit and scope of the present invention as defined by the appended claims. It should be understood that the various dependent claims and features described herein may be combined in a manner different from that described in the original claims. It should also be understood that features described in conjunction with individual embodiments may be used in other described embodiments.

Claims

1. A safety monitoring method, characterized in that: The following steps are involved: Acquire scene information of a target scene, wherein the scene information includes one or more of image information, sound information, and environmental parameter information; Determine whether a specific object is active in the target scene according to the image information and / or sound information in the scene information, and enter a special monitoring mode in response to the specific object being active in the target scene; Determining whether a security risk occurs in the target scene according to the scene information; In response to the security risk that occurs, a risk warning signal is sent.

2. The safety monitoring method according to claim 1, characterized in that: The environmental parameter information includes one or more of temperature information, smoke concentration information and dust concentration information.

3. The safety monitoring method according to claim 1, characterized in that: Judging whether a specific object is active in the target scene according to the image information and / or sound information in the scene information includes the following sub-steps: Determining, based on the image information and / or sound information in the scene information, whether there is an active object in the target scene that matches the specific object in category; In response to the presence of an active object in the target scene that matches the specific object in category, comparing the image features and / or sound features of the active object with the pre-stored image features and / or sound features of the specific object; According to the comparison result, it is determined whether a specific object is active in the target scene.

4. The safety monitoring method according to claim 1, characterized in that: In response to the specific object being active in the target scene, entering a special monitoring mode includes the following sub-steps: Determining, based on the image information, a distance between the specific object and a risk device in the target scene; In response to the specific object entering the warning range corresponding to the risk device, determining the risk level according to the distance value between the specific object and the risk device, wherein the smaller the distance value, the higher the risk level; According to the risk level, the operating state of the risk equipment is controlled and a risk warning signal of a corresponding level is issued.

5. The safety monitoring method according to claim 4, characterized in that: According to the risk level, the operating state of the risk equipment is controlled and a risk warning signal of a corresponding level is issued, including the following sub-steps: In response to the risk level being a primary risk, maintaining the operation of the risk device, sending a voice warning message to the specific object, and sending a risk warning message to an associated object associated with the specific object; In response to the risk level being a medium risk, shutting down the risk device, sending a voice warning message to the specific object, and sending a risk warning message to an associated object associated with the specific object; In response to the risk level being a high risk, the risk device is turned off, a voice warning message is sent to the specific object, a risk warning message is sent to an associated object associated with the specific object, and a medical assistance message is sent to a predetermined medical institution.

6. The safety monitoring method according to claim 1, characterized in that: In response to the specific object being active in the target scene, entering a special monitoring mode includes the following sub-steps: In response to the specific object being a person, determining current positions of a plurality of predetermined feature points of a human skeleton according to the image information; Determining whether there is a risk of a person falling according to the current positions of the plurality of predetermined feature points; In response to a risk situation where a person falls, a risk warning signal is sent.

7. The safety monitoring method according to claim 6, characterized in that: Judging whether there is a risk of a person falling according to the current positions of the plurality of predetermined feature points includes the following sub-steps: According to the fact that the head feature points, the neck feature points and the hip feature points are in the same plane, and the trend characteristics of the plane match the trend characteristics of the ground of the target plane, it is determined that there is a risk of a person falling, and the person is in a lying position; According to the fact that the hip feature point and the foot feature point are in the same plane, the trend characteristics of the plane match the trend characteristics of the ground of the target scene, and the connecting straight line connecting the hip feature point to the head feature point and the neck feature point respectively, the angle between the connecting straight line and the plane is not greater than 90°, it is judged that there is a risk of a person falling, and the person is in a sitting position.

8. The safety monitoring method according to claim 1, characterized in that: In response to the specific object being active in the target scene, entering a special monitoring mode includes the following sub-steps: Acquire a sound signal of the specific object in the sound information; Analyze the sound signal to determine whether the specific object is making a distress call or making a distress call with a decibel or frequency higher than a preset value; In response to the specific object emitting a distress voice or a distress sound, a risk warning signal is sent.

9. The safety monitoring method according to claim 1, characterized in that: Determining whether a security risk occurs in the target scene according to the scene information includes the following sub-steps: Determine, based on the image information in the scene information, whether there is a fire-using device in operation in the target scene and the flame range of the fire-using device; In response to the flame range of the fire-using equipment exceeding the spatial range encompassed by the fire-using equipment, it is determined that a safety risk occurs in the target scene.

10. The safety monitoring method according to claim 1 or 2, characterized in that: Determining whether a security risk occurs in the target scene according to the scene information includes the following sub-steps: Determining whether there is any fire-using equipment in operation in the target scene according to the image information in the scene information; Determine whether there is dust approaching the fire-using equipment in operation based on the image information in the scene information or further combined with the dust concentration information in the environmental parameter information; In response to dust approaching the fire-using equipment in operation, it is determined that a safety risk has occurred in the target scene.

11. A monitoring system, characterized in that: include: An information acquisition device, which is used to acquire scene information of a target scene; an object detection device, which is used to determine whether a specific object is active in the target scene based on the image information and / or sound information in the scene information, and enter a special monitoring mode in response to the specific object being active in the target scene; A risk determination device, which is used to determine whether a safety risk occurs in the target scene according to the scene information; as well as The risk warning device is used to send a risk warning signal in response to a safety risk that occurs.

12. A smart home appliance, characterized in that: Comprising a monitoring system as claimed in claim 11.

Images