Mouth and nose occlusion judging and reminding method and device, storage medium and electronic equipment

Abstract

The embodiment of the invention discloses a mouth and nose shielding judging and reminding method and device, a storage medium and electronic equipment, and relates to the technical field of computers. The method comprises the following steps: acquiring facial information of a target user, and extracting nose features and mouth features from the facial information; according to the nose feature and the mouth feature, judging whether the mouth and the nose of the target user are blocked or not; when it is determined that the mouth and nose of the target user are blocked, whether the duration of the mouth and nose blocking state of the target user reaches a preset duration or not is judged; and when it is determined that the duration of the mouth and nose shielding state of the target user reaches the preset duration, sending reminding information to target equipment. The reminding information is used for reminding a guardian to pay attention to the problem that the mouth and the nose of the person under guardianship are blocked, the blocked object is removed in time, and more serious safety problems of the person under guardianship are avoided.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, device, storage medium and electronic device for judging and reminding of mouth and nose occlusion. Background Art

[0002] Nowadays, with the development of smart homes, care for the elderly and children has become an important part of smart homes. Since the physical functions of young children have not yet matured, when they rest in their beds, they may be blocked by their mouths and noses due to some behaviors, which may lead to the risk of suffocation. The current mainstream method to solve this problem is to use cameras to monitor children's beds, add visual deep learning related algorithms, perform target detection and face detection plus classification network solutions, and perform an occlusion classification judgment.

[0003] However, the solution of directly performing target detection and face detection plus classification network requires a high generalization ability of the model and needs to collect as many pictures as possible in different environments and with different degrees of face occlusion. It requires a huge amount of data and is relatively cumbersome to implement. Once the amount of data is insufficient, the accuracy of the prediction results will decrease. Summary of the invention

[0004] The embodiments of the present application provide a method, device, storage medium and electronic device for judging and reminding mouth and nose occlusion, which can solve the above problems. The technical solution is as follows:

[0005] In a first aspect, an embodiment of the present application provides a method for judging and reminding mouth and nose occlusion, the method comprising:

[0006] Acquire facial information of a target user, and extract nose features and mouth features from the facial information;

[0007] Determining whether the target user has mouth and nose occlusion according to the nose feature and the mouth feature;

[0008] When it is determined that the target user has an obstruction of the mouth and nose, determining whether the duration of the obstruction of the mouth and nose of the target user reaches a preset duration;

[0009] When it is determined that the duration of the target user being in the mouth and nose blocking state reaches the preset duration, a reminder message is sent to the target device.

[0010] In a second aspect, an embodiment of the present application provides a mouth and nose occlusion judgment and reminder device, the device comprising:

[0011] A feature extraction module, used to obtain facial information of a target user and extract nose features and mouth features from the facial information;

[0012] A first judgment module is used to judge whether the target user has mouth and nose occlusion according to the nose feature and the mouth feature;

[0013] The second judgment module is used to judge whether the duration of the target user being in the mouth and nose blocking state reaches a preset duration when it is determined that the target user is in the mouth and nose blocking state;

[0014] The occlusion reminder module is used to send a reminder message to the target device when it is determined that the duration of the target user's mouth and nose occlusion state reaches the preset duration.

[0015] In a third aspect, an embodiment of the present application provides a computer storage medium, wherein the computer storage medium stores a plurality of instructions, wherein the instructions are suitable for being loaded by a processor and executing the above-mentioned method steps.

[0016] In a fourth aspect, an embodiment of the present application provides an electronic device, which may include: a processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the above-mentioned method steps.

[0017] The beneficial effects brought about by the technical solutions provided by some embodiments of the present application include at least:

[0018] In the present application, the facial information of the target user who is the guardian is obtained, and the nose features and mouth features are extracted from the facial information. The nose features represent the feature data corresponding to the area where the nose of the target user is located, and the mouth features represent the feature data corresponding to the area where the mouth of the target user is located. Furthermore, it is determined based on the nose features and the features whether the target user has mouth and nose obstruction, and whether the duration of the target user's mouth and nose obstruction state reaches a preset duration. Only when it is determined that the duration of the target user's mouth and nose obstruction state reaches a preset duration, a reminder message is sent to the target device. The reminder message is used to remind the guardian to pay attention to the problem of mouth and nose obstruction of the guardian, and to remove the obstruction in time to avoid more serious safety problems for the guardian.

[0019] In this application, by setting a preset time to distinguish between accidental, short-term mouth and nose occlusion and continuous, risky mouth and nose occlusion, false alarms triggered by short-term occlusion can be avoided, unnecessary reminders and interference can be reduced, and user experience can be improved. In this application, the mouth and nose occlusion can be monitored to guide guardians to pay attention to the health and even safety issues of the guardian. For example, for infants and young children, mouth and nose occlusion detection can be used to guide infants and young children to develop correct sleeping habits to reduce the risk of mouth and nose being blocked. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0021] Figure 1 It is a schematic diagram of the architecture of a method for judging and reminding mouth and nose occlusion provided in an embodiment of the present application;

[0022] Figure 2 It is a flowchart of a method for judging and reminding mouth and nose occlusion provided in an embodiment of the present application;

[0023] Figure 3 This is a scene diagram of a method for judging and reminding mouth and nose occlusion provided in an embodiment of the present application;

[0024] Figure 4 This is a page schematic diagram of a mouth and nose cover reminder setting provided in an embodiment of the present application;

[0025] Figure 5 It is a flowchart of a method for judging and reminding mouth and nose occlusion provided in an embodiment of the present application;

[0026] Figure 6 It is a flowchart of a method for judging and reminding mouth and nose occlusion provided in an embodiment of the present application;

[0027] Figure 7 is a schematic diagram of an adjustment monitoring device provided in an embodiment of the present application;

[0028] Figure 8 It is a structural schematic diagram of a mouth and nose occlusion judgment and reminder device provided in an embodiment of the present application;

[0029] Fig. 9 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0031] In the description of the present application, it should be understood that the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In the description of the present application, it should be noted that, unless otherwise clearly specified and limited, "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood in specific circumstances. In addition, in the description of the present application, unless otherwise specified, "multiple" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B, which can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are an "or" relationship.

[0032] The present application is described in detail below with reference to specific embodiments.

[0033] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in the embodiments of this specification are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions. For example, the features, information and data involved in this specification are all obtained with full authorization.

[0034] Nowadays, with the development of smart homes, care for the elderly and children has become an important part of smart homes. Since the physical functions of young children have not yet matured, when they rest in their beds, they may be blocked by their mouths and noses due to some behaviors, which may lead to the risk of suffocation. The current mainstream method to solve this problem is to use cameras to monitor children's beds, add visual deep learning related algorithms, perform target detection and face detection plus classification network solutions, and perform an occlusion classification judgment.

[0035] like Figure 1 As shown, Figure 1 It is a schematic diagram of the architecture of a method for judging and reminding mouth and nose occlusion provided in an embodiment of this specification. Figure 1 The system includes multiple electronic devices, which at least include multiple monitoring devices and a server 101 for executing the method for determining and reminding the mouth and nose occlusion. The multiple monitoring devices at least include monitoring device 1021, monitoring device 1022, and monitoring device 1023. It is understandable that Figure 1 The number and type of the multiple monitoring devices and servers 101 shown in the figure are for illustration only, and this application does not impose any limitation on this.

[0036] The above-mentioned server 101 includes but is not limited to physical or virtual processors, mobile stations (MS), mobile terminals (Mobile Terminal), mobile phones (Mobile Telephone), handsets and portable equipment (portable equipment), Bluetooth headsets, smart watches and other types. The electronic equipment can communicate with one or more core networks via a radio access network (Radio Access Network, RAN).

[0037] For example, the server 101 is a plurality of physical servers, and the plurality of physical servers are independent in hardware. Alternatively, the server 101 is a plurality of virtual servers, and the plurality of virtual servers are deployed in the same hardware resource pool, and the deployment methods of the virtual servers include but are not limited to: VMware, Virtual Box, and Virtual PC.

[0038] It is understandable that the server 101 also has other service capabilities and functions to complete the tasks in the following embodiments. For example, the server 101 also provides portal services, resource management services, and CI / CD services.

[0039] In the present application, monitoring devices such as monitoring device 1021, monitoring device 1022 and monitoring device 1023 are equipped with camera devices, and the camera devices can be various devices that can realize image recording functions. For example, the camera device can be any type of camera device such as a fixed camera, a pan / tilt camera, an infrared camera, a PTZ (Pan / Tilt / Zoom) camera, a high-definition network camera, a miniature camera, a thermal imaging camera, a panoramic camera, etc. Among them, the fixed camera is installed in a fixed position, such as a wall, a ceiling or a pole, and can capture video images of a wide angle or a specific area. The pan / tilt camera device can be rotated horizontally and vertically by a pan / tilt (i.e., a mechanical device) to achieve large-scale monitoring. The user can adjust the direction and angle of the pan / tilt camera device through remote control to flexibly monitor scenes in multiple directions or areas. The panoramic camera device uses special lenses and image processing technology to capture and synthesize 360-degree panoramic images or videos, and can cover the entire monitoring area without rotation. In the present application, the user sets one or more camera devices as required to monitor the sleep state of the target user, and the present application does not impose any restrictions on this.

[0040] A display device may also be installed on a certain type of monitoring device or a device that establishes a communication link with the monitoring device. The display device may be any device that can realize a display function, for example, the display device may be a cathode ray tube display (CR), a light-emitting diode display (LED), an electronic ink screen, a liquid crystal display (LCD), a plasma display panel (PDP), etc. For example, a user may use the display device to view a sleep report obtained by the server 101 executing the method for determining and reminding the mouth and nose occlusion described in the present application.

[0041] Multiple monitoring devices and the server 101 that executes the mouth and nose occlusion judgment and reminder method can communicate through a communication link established by a communication protocol, such as the gRPC protocol. gRPC is a high-performance, general-purpose open source remote processor call (RPC) framework, which is mainly designed for mobile application development and based on the HTTP / 2 protocol standard, based on the protocol buffer (PB) serialization protocol development, and supports many development languages. In addition, the communication link can also be a wireless communication link or a wired communication link, for example: a wired communication link includes an optical fiber, a twisted pair or a coaxial cable, and a wireless communication link includes a Bluetooth communication link, a wireless fidelity (WIreless-FIdelity, Wi-Fi) communication link or a microwave communication link, etc.

[0042] However, the solution of directly performing target detection and face detection plus classification network requires a high generalization ability of the model and needs to collect as many pictures as possible in different environments and with different degrees of face occlusion. It requires a huge amount of data and is relatively cumbersome to implement. Once the amount of data is insufficient, the accuracy of the prediction results will decrease.

[0043] Therefore, in order to solve the above problems, the present application proposes a method for judging and reminding mouth and nose occlusion to solve the above problems. Figure 2 The figure is a schematic diagram of a method for judging and reminding mouth and nose occlusion provided by an embodiment of the present application. The method can be implemented by a computer program and can be run on a mouth and nose occlusion judgment and reminder device based on the von Neumann system. The computer program can be integrated into an application or run as an independent tool application.

[0044] Specifically, the mouth and nose occlusion judgment and reminder method includes:

[0045] S101. Obtain facial information of a target user, and extract nose features and mouth features from the facial information.

[0046] The target user can be understood as the monitored object. The method of determining a user as the monitored object can be through preset or specified by a user with specified authority, which is not limited in this application. Facial information can be understood as changes on a person's face that can be observed from the outside, and these changes can reflect a person's emotional state or physiological condition.

[0047] like Figure 3 As shown, Figure 3 1 is a scene diagram of a method for judging and reminding mouth and nose occlusion provided by an embodiment of the present application. In this embodiment, the monitoring device 1021 is set on the fence of the crib, and the monitoring range is the range of the entire crib. The monitoring device collects facial information of the baby 103 as the target user.

[0048] In the present application, nose features and mouth features are extracted from facial information by a key point algorithm. The key point algorithm accurately describes facial features by locating a series of key points on the face (such as the corners of the eyes, the corners of the mouth, the tip of the nose, etc.). Specifically, when extracting nose features, the tip of the nose is usually selected as the key feature point, and auxiliary points such as the points on both sides of the nose or the midpoint of the line connecting the centers of the nostrils can also be considered. According to the coordinates of key points such as the tip of the nose, nose features such as length, width, inclination angle, etc. can be calculated. When extracting mouth features, the corners of the mouth are usually selected as key feature points, and auxiliary points such as the points of the upper and lower contours of the lips or the center point of the lips can also be considered. According to the coordinates of key points such as the corners of the mouth, mouth features such as width, height, opening angle, etc. can be calculated.

[0049] In another embodiment, nose features and mouth features may be extracted from facial information based on a deep learning-based facial key point detection algorithm (such as MTCNN, Dlib, etc.).

[0050] In one embodiment, based on the facial information of the target user whose mouth and nose are not blocked, the preset general facial model and key point matching algorithm are adjusted to obtain the target facial model and target key point matching algorithm; based on the facial information of the target user, the nose features and mouth features are extracted from the facial information according to the target facial model and the target key point matching algorithm.

[0051] Specifically, firstly, a group of facial images or video frames of the target user whose mouth and nose are not blocked are collected, and the facial images are preprocessed, including face detection, alignment, normalization and other steps, to ensure that the facial images are in similar positions and scales. For example, the parents of the target user can collect facial information of the target user 103 whose mouth and nose are not blocked by shooting with a mobile phone or other methods, and send the facial information to the server 101.

[0052] Furthermore, the universal facial model is a predefined 3D or 2D model that includes a series of facial key points (such as eye corners, mouth corners, nose, etc.) and the connection relationship between these points. Furthermore, according to the facial features of the target user, the key points in the universal facial model are repositioned and adjusted to ensure that they can accurately match the actual facial features of the target user. For example, the universal facial model is deformed by machine learning or statistical methods (such as PCA) to obtain a target facial model. The target facial model can better adapt to the facial shape of the target user than the traditional facial model.

[0053] Furthermore, the initial key point matching algorithm may be based on template matching, regression model (such as convolutional neural network) or other methods. Use data enhancement technology to adjust certain parameters in the key point matching algorithm, and use the adjusted target facial model as part of the training data to retrain or fine-tune the key point matching algorithm, and adjust the parameters or network structure of the key point matching algorithm as needed through evaluation of indicators such as precision and recall rate to obtain the target key point matching algorithm.

[0054] Finally, the nose and mouth features are extracted from the target user's facial information based on the target facial model and the target key point matching algorithm. By adjusting the target user's specific facial features, the general facial model and key point matching algorithm can be made closer to the actual situation of a certain user, thereby more accurately locating the nose and mouth features, which helps to improve the accuracy of subsequent mouth and nose occlusion judgment.

[0055] S102: Determine whether the target user's nose and mouth are blocked based on the nose and mouth features.

[0056] If the target user has occlusion of the mouth and nose, it can be understood that there are occluders in the user's mouth and nose areas at the same time. In one embodiment, the occlusion detection algorithm can be used to determine whether the target user has occlusion of the mouth and nose. Specifically, based on the judgment of a single-label classifier, whether the mouth area and the nose area are occluded is used as a label respectively, and a classifier (such as SVM, random forest, neural network, etc.) is used to classify the mouth features and nose features respectively. Alternatively, through a multi-label classifier, multiple labels are set for different occluded parts, and a multi-label classification algorithm is used for detection.

[0057] In another embodiment, a deep learning algorithm can also be used for occlusion detection to determine whether the target user has occlusion of the mouth and nose. Specifically, a convolutional neural network (CNN) model is trained to directly identify the occluders corresponding to the nose features and mouth features, and a mask or boundary map of the occluded area is generated. The occluders corresponding to the mouth area and nose area are further located by using a target detection algorithm (such as Faster R-CNN, YOLO), and an image segmentation algorithm (such as Mask R-CNN) is used to accurately segment the occluded area, thereby determining whether the mouth and nose of the target user are simultaneously occluded by the occluder.

[0058] S103: When it is determined that the target user has mouth and nose blocked, determine whether the duration of the target user's mouth and nose blocked state reaches a preset duration.

[0059] The preset duration can be any duration, for example, the preset duration is 30 seconds or 1 minute. The preset duration can be set by the relevant user as needed, or it can be preset, and this application does not impose any restrictions on this. When it is detected that the target user is in a mouth and nose occlusion state, the time point when the occlusion starts is recorded, and whether the state persists is continuously monitored. By continuously comparing the current time with the start time of the occlusion, the duration of the target user's mouth and nose occlusion state can be calculated, and the duration can be compared to see whether it reaches the preset duration.

[0060] S104: When it is determined that the duration of the target user's mouth and nose covering state reaches a preset duration, a reminder message is sent to the target device.

[0061] When the server detects that the target user is in a state of mouth and nose blocking for a preset duration, for example, when the target user's infant is detected to be in a state of mouth and nose blocking for 30 seconds, the server can send a reminder message to the target terminal (such as a smart phone, tablet computer or smart watch) that has established a communication link with the server. The content of the reminder message can provide sufficient information that the target user is in a state of mouth and nose blocking so that the user as a guardian can take corresponding actions. For example, a text message reminder message with the content "Baby covers his head, please adjust the baby's posture" can be sent to the user.

[0062] In one embodiment, Figure 4 As shown, Figure 4 201. The electronic device provides a setting page 201 for the mouth and nose cover reminder to the user as the guardian. The user confirms the control on the setting page 201, and sends a reminder message to the target device when it is determined that the target user as the guardian is in the mouth and nose cover state for a preset time. The specific content of the reminder message can also be obtained through Figure 4The page for setting the mouth and nose cover reminder is shown.

[0063] In the present application, the facial information of the target user who is the guardian is obtained, and the nose features and mouth features are extracted from the facial information. The nose features represent the feature data corresponding to the area where the nose of the target user is located, and the mouth features represent the feature data corresponding to the area where the mouth of the target user is located. Furthermore, it is determined based on the nose features and the features whether the target user has mouth and nose obstruction, and whether the duration of the target user's mouth and nose obstruction state reaches a preset duration. Only when it is determined that the duration of the target user's mouth and nose obstruction state reaches a preset duration, a reminder message is sent to the target device. The reminder message is used to remind the guardian to pay attention to the problem of mouth and nose obstruction of the guardian, and to remove the obstruction in time to avoid more serious safety problems for the guardian.

[0064] In this application, by setting a preset time to distinguish between accidental, short-term mouth and nose occlusion and continuous, risky mouth and nose occlusion, false alarms triggered by short-term occlusion can be avoided, unnecessary reminders and interference can be reduced, and user experience can be improved. In this application, the mouth and nose occlusion can be monitored to guide guardians to pay attention to the health and even safety issues of the guardian. For example, for infants and young children, mouth and nose occlusion detection can be used to guide infants and young children to develop correct sleeping habits to reduce the risk of mouth and nose being blocked.

[0065] In one embodiment, Figure 5 The figure is a schematic diagram of a method for judging and reminding mouth and nose occlusion provided by an embodiment of the present application. The method can be implemented by a computer program and can be run on a mouth and nose occlusion judgment and reminder device based on the von Neumann system. The computer program can be integrated into an application or run as an independent tool application.

[0066] Specifically, the mouth and nose occlusion judgment and reminder method includes:

[0067] S201. Obtain facial information of a target user, and extract nose features and mouth features from the facial information.

[0068] Refer to the above S101, which will not be repeated here.

[0069] S202: Determine whether the degree of occlusion of the target user's nose meets a first preset condition based on the nose feature, and determine whether the degree of occlusion of the target user's mouth meets a second preset condition based on the mouth feature, so as to determine whether the target user has occlusion of the mouth and nose.

[0070] Determining whether the target user has mouth and nose obstruction includes determining whether the degree of obstruction of the target user's nose meets a first preset condition, and determining whether the degree of obstruction of the target user's mouth meets a second preset condition. Only when both the first preset condition and the second preset condition are met, it is determined that the target user has mouth and nose obstruction.

[0071] In other words, the degree of occlusion of the user's nose and the degree of occlusion of the user's mouth are detected respectively. When information representing that the degree of occlusion of the user's nose and the degree of occlusion of the user's mouth are both severely occluded is detected, it is determined that the target user has occlusion of the mouth and nose.

[0072] The first preset condition for determining the degree of obstruction of the target user's nose and the second preset condition for determining the degree of obstruction of the target user's mouth are not completely the same. For example, when the degree of obstruction of the nose is 50% and the degree of obstruction of the mouth is 90%, it is determined that the target user has obstruction of the nose and mouth.

[0073] In this embodiment, by separately setting a first preset condition for determining the degree of occlusion of the target user's nose and a second preset condition for determining the degree of occlusion of the target user's mouth, the accuracy of the judgment can be improved. This is because the nose and mouth have significant differences in facial structure and function, so the impact on the user and the degree of attention required when they are occluded may also be different. By setting different preset conditions for them respectively, the actual occlusion of each can be more accurately reflected, thereby improving the accuracy of the judgment.

[0074] S203: When it is determined that the first preset condition and the second preset condition are met, it is determined that the target user has mouth and nose obstruction, and whether the duration of the target user's mouth and nose obstruction reaches a preset duration.

[0075] When it is determined that the first preset condition is not met or the second preset condition is not met, it is determined that the target user is not in a state of mouth and nose occlusion, and the target user continues to be monitored.

[0076] In one embodiment, the head information of the target user is obtained, and the state of the target user is determined to be in the head lying state or the head lying state according to the head information; and the first preset condition and the second preset condition matching the state are determined according to the state of the target user.

[0077] Specifically, a monitoring device is used to obtain the target user's head information, which may include the position, angle, inclination, etc. of the head. By accurately calculating the head posture and comparing it with a preset posture model, it is determined whether the target user is currently in a side-lying state or a flat-lying state.

[0078] For different head states, different preset conditions are set to determine the mouth and nose occlusion situation. The preset conditions include a first preset condition and a second preset condition.

[0079] When the target user is in a state of lying on the side, the natural position of the mouth and nose will be biased to one side due to gravity, making it easier for the target user to have difficulty breathing. Even a relatively mild degree of mouth and nose occlusion will cause the target user to have safety problems due to difficulty breathing, so the first preset condition and the second preset condition are more relaxed. For example, when the target user is in a state of lying on the side, when it is detected that the occlusion degree of the nose is 30% and the occlusion degree of the mouth is 40%, it is determined that the target user has mouth and nose occlusion.

[0080] When the target user is in a state of lying flat on his head, the mouth and nose are located in the center of the face, and the target user is less likely to have difficulty breathing. Therefore, the first preset condition and the second preset condition are more relaxed than the first preset condition and the second preset condition corresponding to the state of the target user lying on the side of the head. For example, when the target user is in a state of lying on the side of the head, when it is detected that the occlusion degree of the nose is 80% and the occlusion degree of the mouth is 90%, it is determined that the target user has occlusion of the mouth and nose.

[0081] In this embodiment, by distinguishing the head state and setting the corresponding first preset condition and second preset condition, the mouth and nose occlusion situation can be judged more accurately, so that the mouth and nose occlusion judgment and reminder method provided in this application can adapt to more usage scenarios and user habits, reducing the possibility of misjudgment and missed judgment.

[0082] In one embodiment, the age stage of the target user is determined to be an infant or an elderly stage; and according to the age stage of the user, a first preset condition, a second preset condition, and a preset duration matching the age stage are acquired.

[0083] Age stages include infant and elderly stages. In this embodiment, the age stage of the target user is determined, and matching preset conditions and preset durations are set according to different age stages to determine whether the target user has mouth and nose occlusion. This is because people of different ages have different physiological characteristics and environmental needs, so customized monitoring standards and methods are required. Therefore, the preset conditions and preset durations required for customized sleep monitoring can significantly improve the accuracy and practicality of mouth and nose occlusion judgments and reminders.

[0084] It is understandable that the infant stage generally refers to children aged 0 to 3, and the elderly stage generally refers to adults over 65. According to the age stage, the preset conditions matching it are obtained.

[0085] In this embodiment, the age stage of the target user can be analyzed by combining the facial features collected by the monitoring device with a deep learning model. Specifically, a deep learning model (such as a convolutional neural network (CNN)) is used to detect key points of the face, such as eyes, nose, mouth, etc., and further extract features from these key points, such as the distance between the eyes, the length of the bridge of the nose, the facial contour, etc., and an age estimation model is trained to obtain the age stage of the target user based on the above features. The model can predict the user's age based on the extracted facial features. Commonly used models include regression-based models (such as support vector regression SVR) and classification-based models (such as multi-layer perceptron MLP), and this embodiment does not impose any restrictions on this.

[0086] According to the age stage of the target user, the first preset condition, the second preset condition and the preset duration that match the age stage are obtained. Target users in the infant stage are not only more likely to have their mouth and nose blocked, but also unable to solve the problem of mouth and nose blocked by themselves. These characteristics should be considered to set a looser first preset condition, a second preset condition, and a shorter preset duration, so that the system can also send a reminder message to the user as a guardian when it is determined that the infant is in a lower degree of mouth and nose blocked.

[0087] For target elderly users, due to long-term user habits (for example, they like to cover their mouths with quilts when sleeping) and the ability to solve the problem of mouth and nose obstruction on their own, you can consider setting stricter first preset conditions, second preset conditions, and longer preset times.

[0088] In this embodiment, matching preset conditions and preset durations are set according to the characteristics of different age groups, which can not only improve the accuracy and reliability of the mouth and nose occlusion judgment and reminder method, but also provide users with more personalized and safe services.

[0089] S204: When it is determined that the duration of the target user's mouth and nose blocking state reaches a preset duration, a reminder message is sent to the target device.

[0090] See the above S104, which will not be described again here.

[0091] In the present application, the facial information of the target user who is the guardian is obtained, and the nose features and mouth features are extracted from the facial information. The nose features represent the feature data corresponding to the area where the nose of the target user is located, and the mouth features represent the feature data corresponding to the area where the mouth of the target user is located. Furthermore, it is determined based on the nose features and features whether the target user has mouth and nose obstruction, and whether the duration of the target user's mouth and nose obstruction state reaches a preset duration. Only when it is determined that the duration of the target user's mouth and nose obstruction state reaches a preset duration, a reminder message is sent to the target device. The reminder message is used to remind the guardian to pay attention to the problem of mouth and nose obstruction of the ward, and to remove the obstruction in time to avoid more serious safety problems for the ward.

[0092] In this application, by setting a preset time to distinguish between accidental, short-term mouth and nose occlusion and continuous, risky mouth and nose occlusion, false alarms triggered by short-term occlusion can be avoided, unnecessary reminders and interference can be reduced, and user experience can be improved. In this application, the mouth and nose occlusion can be monitored to guide guardians to pay attention to the health and even safety issues of the guardian. For example, for infants and young children, mouth and nose occlusion detection can be used to guide infants and young children to develop correct sleeping habits to reduce the risk of mouth and nose being blocked.

[0093] In one embodiment, Figure 6 The figure is a schematic diagram of a method for judging and reminding mouth and nose occlusion provided by an embodiment of the present application. The method can be implemented by a computer program and can be run on a mouth and nose occlusion judgment and reminder device based on the von Neumann system. The computer program can be integrated into an application or run as an independent tool application.

[0094] Specifically, the mouth and nose occlusion judgment and reminder method includes:

[0095] S301: Detect whether a target user in a sleeping area enters a sleeping state.

[0096] The sleeping area can be understood as a space or location where the target user sleeps, such as a bed, sofa, or other suitable place for sleeping. In this application, it refers to a specific range monitored by a monitoring device, within which the monitoring device collects information such as facial expressions and body movements of the target user through a photographic device. The sleeping area can be specified by default or by a user with specified permissions, and this is not limited in this application.

[0097] When awake, users usually adjust their posture and surroundings to avoid having their mouth and nose blocked. When asleep, the body is in a relaxed state and users may not be able to make adjustments in time, so the risk of mouth and nose occlusion is higher. Second, performing mouth and nose occlusion detection while awake may cause false alarms due to the user's normal behavior (such as wearing a mask, covering the mouth and nose with hands, etc.).

[0098] Therefore, in this embodiment, mouth and nose occlusion detection is performed on the target user only after the target user enters a sleeping state, which can reduce computing resources and energy consumption, and reduce the disturbance to the user caused by false alarms.

[0099] S302: When it is determined that the target user in the sleeping area enters a sleeping state, facial information of the target user is obtained, and nose features and mouth features are extracted from the facial information.

[0100] See the above S101, which will not be repeated here.

[0101] S103: Determine whether the target user's nose and mouth are blocked based on the nose and mouth features.

[0102] See the above S102, which will not be described again here.

[0103] In step S304, when it is determined that the target user has mouth and nose obstructions, the monitoring direction of the monitoring device is adjusted according to the completeness of the facial features in the facial information to obtain the adjusted facial information of the target user.

[0104] The degree of completeness of the facial features in the facial information is adjusted to be greater than the degree of completeness of the facial features in the facial information. In this embodiment, the best detection direction can be calculated based on the analysis result of the occlusion degree, or the best monitoring direction can be determined in the process of adjusting the monitoring direction.

[0105] Specifically, by adjusting the angle and focal length of the camera or using a multi-camera system, the occlusion is minimized while maximizing the integrity of the target user's facial features. For example, facial tracking technology is used to continuously track the target user's face and dynamically adjust the monitoring direction.

[0106] like Figure 7 As shown, Figure 7 301 is a schematic diagram of an adjustment monitoring device provided in an embodiment of the present application. The monitoring device 301 includes a camera and a support rod, and the monitoring direction of the camera is adjusted by rotating the support rod to obtain the adjusted facial information of the target user.

[0107] For example, since the target user adjusts from a head lying flat state to a head lying sideways state, the monitoring device that was once located directly above the target user's face detects that the facial features in the collected facial information are incomplete, and the facial features corresponding to the target user's lower half face cannot be collected. Therefore, the monitoring direction of the monitoring device is adjusted so that the adjusted monitoring device is still facing the target user's face, thereby obtaining the target user's adjusted facial information. At this time, the completeness of the facial features in the adjusted facial information is greater than the completeness of the facial features in the facial information.

[0108] In this embodiment, the monitoring direction of the monitoring device is adjusted to obtain adjusted facial information of the target user, thereby improving the completeness of the facial features included in the facial information. The completeness of the extraction of nose features and mouth features can be increased, thereby improving the accuracy of judging whether the target user has mouth and nose occlusion.

[0109] S305: Based on the nose features and mouth features in the adjusted facial information, determine again whether the target user has mouth and nose obstruction.

[0110] See the above S102, which will not be described again here.

[0111] S306: When it is determined that the target user has mouth and nose blocked, determine whether the duration of the target user's mouth and nose blocked state reaches a preset duration.

[0112] See the above S103, which will not be described again here.

[0113] S307: When it is determined that the duration of the target user's mouth and nose covering state reaches a preset duration, a reminder message is sent to the target device.

[0114] See the above S104, which will not be repeated here.

[0115] In the present application, the facial information of the target user who is the guardian is obtained, and the nose features and mouth features are extracted from the facial information. The nose features represent the feature data corresponding to the area where the nose of the target user is located, and the mouth features represent the feature data corresponding to the area where the mouth of the target user is located. Furthermore, it is determined based on the nose features and the features whether the target user has mouth and nose obstruction, and whether the duration of the target user's mouth and nose obstruction state reaches a preset duration. Only when it is determined that the duration of the target user's mouth and nose obstruction state reaches a preset duration, a reminder message is sent to the target device. The reminder message is used to remind the guardian to pay attention to the problem of mouth and nose obstruction of the guardian, and to remove the obstruction in time to avoid more serious safety problems for the guardian.

[0116] In this application, by setting a preset time to distinguish between accidental, short-term mouth and nose occlusion and continuous, risky mouth and nose occlusion, false alarms triggered by short-term occlusion can be avoided, unnecessary reminders and interference can be reduced, and user experience can be improved. In this application, the mouth and nose occlusion can be monitored to guide guardians to pay attention to the health and even safety issues of the guardian. For example, for infants and young children, mouth and nose occlusion detection can be used to guide infants and young children to develop correct sleeping habits to reduce the risk of mouth and nose being blocked.

[0117] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.

[0118] See also Figure 8 , which shows a schematic diagram of the structure of a mouth and nose occlusion judgment and reminder device provided by an exemplary embodiment of the present application. The mouth and nose occlusion judgment and reminder device can be implemented as all or part of the device through software, hardware or a combination of both. The mouth and nose occlusion judgment and reminder device includes a feature extraction module 401, a first judgment module 402, a second judgment module 403 and an occlusion reminder module 404.

[0119] The feature extraction module 401 is used to obtain facial information of the target user and extract nose features and mouth features from the facial information;

[0120] A first judgment module 402 is used to judge whether the target user has mouth and nose occlusion according to the nose feature and the mouth feature;

[0121] The second judgment module 403 is used to judge whether the duration of the target user being in the mouth and nose blocking state reaches a preset duration when it is determined that the target user is in the mouth and nose blocking state;

[0122] The occlusion reminder module 404 is used to send a reminder message to the target device when it is determined that the duration of the target user being in the mouth and nose occlusion state reaches the preset duration.

[0123] In one embodiment, the first determination module 402 includes:

[0124] a condition judgment unit, configured to judge whether the degree of obstruction of the nose of the target user satisfies a first preset condition according to the nose feature, and to judge whether the degree of obstruction of the mouth of the target user satisfies a second preset condition according to the mouth feature, so as to judge whether the mouth and nose of the target user are obstructed;

[0125] The second judgment module 403 includes:

[0126] The duration judgment unit is used to determine whether the mouth and nose of the target user are blocked when it is determined that the first preset condition is met and the second preset condition is met, and whether the duration of the mouth and nose blocking state of the target user reaches a preset duration.

[0127] In one embodiment, the first determining module 402 further includes:

[0128] A state acquisition unit, used for acquiring head information of a target user, and determining, according to the head information of the target user, that the target user is in a state of lying on the side of the head or lying flat on the head;

[0129] The state matching unit is used to determine a first preset condition and a second preset condition that match the state according to the state of the target user.

[0130] In one embodiment, the first determining module 402 further includes:

[0131] An age acquisition unit, used to determine whether the age stage of the target user is an infant or an elderly person;

[0132] The age matching unit is used to obtain, according to the age stage to which the user belongs, a first preset condition, a second preset condition and a preset duration that match the age stage.

[0133] In one embodiment, the mouth and nose occlusion judgment and reminder device further includes:

[0134] An algorithm adjustment module, used to adjust a preset universal facial model and a key point matching algorithm according to facial information of a target user who is not in a mouth and nose occlusion state, to obtain a target facial model and a target key point matching algorithm;

[0135] The feature extraction module 401 includes:

[0136] The first extraction unit is used to obtain facial information of a target user and extract nose features and mouth features from the facial information according to the target facial model and the target key point matching algorithm.

[0137] In one embodiment, the mouth and nose occlusion judgment and reminder device further includes:

[0138] A third judgment module is used to adjust the monitoring direction of the monitoring device to obtain the adjusted facial information of the target user according to the completeness of the facial features in the facial information when it is determined that the mouth and nose of the target user are blocked; wherein the completeness of the facial features in the adjusted facial information is greater than the completeness of the facial features in the facial information;

[0139] The fourth judgment module is used to judge again whether the target user has mouth and nose obstruction according to the nose features and mouth features in the adjusted facial information.

[0140] In one embodiment, the mouth and nose occlusion judgment and reminder device further includes:

[0141] A sleep detection module, used to detect whether the target user in the sleep area enters a sleep state;

[0142] The feature extraction module 401 includes:

[0143] The second extraction unit is used to obtain facial information of the target user when it is determined that the target user in the sleeping area enters a sleeping state, and extract nose features and mouth features from the facial information.

[0144] In the present application, the facial information of the target user who is the guardian is obtained, and the nose features and mouth features are extracted from the facial information. The nose features represent the feature data corresponding to the area where the nose of the target user is located, and the mouth features represent the feature data corresponding to the area where the mouth of the target user is located. Furthermore, it is determined based on the nose features and features whether the target user has mouth and nose obstruction, and whether the duration of the target user's mouth and nose obstruction state reaches a preset duration. Only when it is determined that the duration of the target user's mouth and nose obstruction state reaches a preset duration, a reminder message is sent to the target device. The reminder message is used to remind the guardian to pay attention to the problem of mouth and nose obstruction of the ward, and to remove the obstruction in time to avoid more serious safety problems for the ward.

[0145] In this application, by setting a preset time to distinguish between accidental, short-term mouth and nose occlusion and continuous, risky mouth and nose occlusion, false alarms triggered by short-term occlusion can be avoided, unnecessary reminders and interference can be reduced, and user experience can be improved. In this application, the mouth and nose occlusion can be monitored to guide guardians to pay attention to the health and even safety issues of the guardian. For example, for infants and young children, mouth and nose occlusion detection can be used to guide infants and young children to develop correct sleeping habits to reduce the risk of mouth and nose being blocked.

[0146] It should be noted that the mouth and nose occlusion judgment and reminder device provided in the above embodiment only uses the division of the above functional modules as an example when executing the mouth and nose occlusion judgment and reminder method. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the mouth and nose occlusion judgment and reminder device provided in the above embodiment and the mouth and nose occlusion judgment and reminder method embodiment belong to the same concept, and the implementation process thereof is detailed in the method embodiment, which will not be repeated here.

[0147] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0148] The present application also provides a computer storage medium, which can store multiple instructions, and the instructions are suitable for being loaded and executed by a processor as described above. Figure 1 - Figure 7 The method for judging and reminding mouth and nose occlusion in the embodiment shown in the figure can be found in the specific implementation process. Figure 1 - Figure 7 The specific description of the illustrated embodiment will not be repeated here.

[0149] The present application also provides a computer program product, which stores at least one instruction, and the at least one instruction is loaded and executed by the processor as described above. Figure 1 - Figure 7 The method for judging and reminding mouth and nose occlusion in the embodiment shown in the figure can be found in the specific implementation process. Figure 1 - Figure 7 The specific description of the illustrated embodiment will not be repeated here.

[0150] See also Fig. 9 , is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Fig. 9 As shown, the electronic device 500 may include: at least one processor 501 , at least one network interface 504 , a user interface 503 , a memory 505 , and at least one communication bus 502 .

[0151] The communication bus 502 is used to realize the connection and communication between these components.

[0152] The user interface 503 may include a display screen (Display) and a camera (Camera), and the optional user interface 503 may also include a standard wired interface and a wireless interface.

[0153] The network interface 504 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0154] Among them, the processor 501 may include one or more processing cores. The processor 501 uses various interfaces and lines to connect various parts within the entire server 500, and executes various functions and processes data of the server 500 by running or executing instructions, programs, code sets or instruction sets stored in the memory 505, and calling data stored in the memory 505. Optionally, the processor 501 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 501 can integrate one or more combinations of a processor (Central Processing Unit, CPU), an image processor (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 501, and it can be implemented separately through a chip.

[0155] Among them, the memory 505 may include a random access memory (Random Access Memory, RAM) and may also include a read-only memory (Read-Only Memory). Optionally, the memory 505 includes a non-transitory computer-readable storage medium. The memory 505 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 505 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 505 may also be optionally at least one storage device located away from the aforementioned processor 501. As Fig. 9 As shown, the memory 505 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a mouth and nose occlusion judgment and reminder application.

[0156] exist Fig. 9In the electronic device 500 shown, the user interface 503 is mainly used to provide an input interface for the user and obtain the data input by the user; and the processor 501 can be used to call the mouth and nose occlusion judgment and reminder application stored in the memory 505, and specifically perform the following operations:

[0157] Acquire facial information of a target user, and extract nose features and mouth features from the facial information;

[0158] Determining whether the target user has mouth and nose occlusion according to the nose feature and the mouth feature;

[0159] When it is determined that the target user has an obstruction of the mouth and nose, determining whether the duration of the obstruction of the mouth and nose of the target user reaches a preset duration;

[0160] When it is determined that the duration of the target user being in the mouth and nose blocking state reaches the preset duration, a reminder message is sent to the target device.

[0161] In one embodiment, the processor 501 performs the step of determining, based on the nose feature and the mouth feature, whether the target user has mouth and nose occlusion, by specifically performing:

[0162] Determining whether the degree of obstruction of the target user's nose meets a first preset condition according to the nose feature, and determining whether the degree of obstruction of the target user's mouth meets a second preset condition according to the mouth feature, so as to determine whether the target user has mouth and nose obstruction;

[0163] When it is determined that the target user has a mouth and nose blocked state, it is determined whether the duration of the mouth and nose blocked state of the target user reaches a preset duration, specifically performing:

[0164] When it is determined that the first preset condition is met and the second preset condition is met, it is determined that the target user has mouth and nose obstruction, and it is determined whether the duration of the target user's mouth and nose obstruction reaches a preset duration.

[0165] In one embodiment, before the processor 501 performs the step of determining whether the target user has nose and mouth occlusion according to the nose feature and the mouth feature, it further performs:

[0166] Acquire the head information of the target user, and determine the state of the target user as being in a head-lying state or a head-lying state according to the head information of the target user;

[0167] A first preset condition and a second preset condition matching the state are determined according to the state of the target user.

[0168] In one embodiment, before the processor 501 performs the step of determining whether the target user has nose and mouth occlusion according to the nose feature and the mouth feature, it further performs:

[0169] Determining that the age stage of the target user is an infant stage or an elderly stage;

[0170] According to the age stage of the user, a first preset condition, a second preset condition and a preset duration matching the age stage are acquired.

[0171] In one embodiment, before the processor 501 executes the step of acquiring the facial information of the target user and extracting the nose feature and the mouth feature from the facial information, it further executes:

[0172] According to the facial information of the target user whose mouth and nose are not blocked, the preset general facial model and key point matching algorithm are adjusted to obtain the target facial model and the target key point matching algorithm;

[0173] The processor 501 executes the acquisition of the target user's facial information and extracts the nose feature and the mouth feature from the facial information, specifically performing:

[0174] According to the facial information of the target user, nose features and mouth features are extracted from the facial information according to the target facial model and the target key point matching algorithm.

[0175] In one embodiment, after the processor 501 executes the step of determining whether the target user has mouth and nose occlusion according to the nose feature and the mouth feature, before determining whether the duration of the target user being in the mouth and nose occlusion state reaches a preset duration when determining that the target user has mouth and nose occlusion, further executes:

[0176] When it is determined that the target user has an obstruction of the mouth and nose, adjusting the monitoring direction of the monitoring device according to the completeness of the facial features in the facial information to obtain the adjusted facial information of the target user; wherein the completeness of the facial features in the adjusted facial information is greater than the completeness of the facial features in the facial information;

[0177] According to the nose features and mouth features in the adjusted facial information, it is determined again whether the target user has mouth and nose occlusion.

[0178] In one embodiment, before the processor 501 executes the step of acquiring the facial information of the target user and extracting the nose feature and the mouth feature from the facial information, it further executes:

[0179] Detect whether the target user in the sleeping area has entered a sleeping state;

[0180] The processor 501 executes the acquisition of the target user's facial information and extracts the nose feature and the mouth feature from the facial information, specifically performing:

[0181] When it is determined that the target user in the sleeping area enters a sleeping state, facial information of the target user is acquired, and nose features and mouth features are extracted from the facial information.

[0182] In the present application, the facial information of the target user who is the guardian is obtained, and the nose features and mouth features are extracted from the facial information. The nose features represent the feature data corresponding to the area where the nose of the target user is located, and the mouth features represent the feature data corresponding to the area where the mouth of the target user is located. Furthermore, it is determined based on the nose features and features whether the target user has mouth and nose obstruction, and whether the duration of the target user's mouth and nose obstruction state reaches a preset duration. Only when it is determined that the duration of the target user's mouth and nose obstruction state reaches a preset duration, a reminder message is sent to the target device. The reminder message is used to remind the guardian to pay attention to the problem of mouth and nose obstruction of the ward, and to remove the obstruction in time to avoid more serious safety problems for the ward.

[0183] In this application, by setting a preset time to distinguish between accidental, short-term mouth and nose occlusion and continuous, risky mouth and nose occlusion, false alarms triggered by short-term occlusion can be avoided, unnecessary reminders and interference can be reduced, and user experience can be improved. In this application, the mouth and nose occlusion can be monitored to guide guardians to pay attention to the health and even safety issues of the guardian. For example, for infants and young children, mouth and nose occlusion detection can be used to guide infants and young children to develop correct sleeping habits to reduce the risk of mouth and nose being blocked.

[0184] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only storage memory, or a random access memory, etc.

[0185] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0186] The above disclosure is only the preferred embodiment of the present application, which certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A method for judging and reminding mouth and nose occlusion, characterized in that: The method comprises: Acquire facial information of a target user, and extract nose features and mouth features from the facial information; Determining whether the target user has mouth and nose occlusion according to the nose feature and the mouth feature; When it is determined that the target user has an obstruction of the mouth and nose, determining whether the duration of the obstruction of the mouth and nose of the target user reaches a preset duration; When it is determined that the duration of the target user being in the mouth and nose blocking state reaches the preset duration, a reminder message is sent to the target device.

2. The method for judging and reminding mouth and nose occlusion according to claim 1, characterized in that: The determining, based on the nose feature and the mouth feature, whether the target user has mouth and nose occlusion includes: Determining whether the degree of obstruction of the target user's nose meets a first preset condition according to the nose feature, and determining whether the degree of obstruction of the target user's mouth meets a second preset condition according to the mouth feature, so as to determine whether the target user has mouth and nose obstruction; When it is determined that the target user has a mouth and nose blocked state, determining whether the duration of the target user being in the mouth and nose blocked state reaches a preset duration includes: When it is determined that the first preset condition is met and the second preset condition is met, it is determined that the target user has mouth and nose obstruction, and it is determined whether the duration of the target user's mouth and nose obstruction reaches a preset duration.

3. The method for judging and reminding mouth and nose occlusion according to claim 2, characterized in that: Before determining whether the target user has nose and mouth obstruction according to the nose feature and the mouth feature, the method further includes: Acquire the head information of the target user, and determine the state of the target user as being in a head-lying state or a head-lying state according to the head information of the target user; A first preset condition and a second preset condition matching the state are determined according to the state of the target user.

4. The method for judging and reminding mouth and nose occlusion according to claim 2, characterized in that: Before determining whether the target user has nose and mouth obstruction according to the nose feature and the mouth feature, the method further includes: Determining that the age stage of the target user is an infant stage or an elderly stage; According to the age stage of the user, a first preset condition, a second preset condition and a preset duration matching the age stage are acquired.

5. The method for judging and reminding mouth and nose occlusion according to claim 1, characterized in that: Before acquiring the facial information of the target user and extracting the nose feature and the mouth feature from the facial information, the method further includes: According to the facial information of the target user whose mouth and nose are not blocked, the preset general facial model and key point matching algorithm are adjusted to obtain the target facial model and the target key point matching algorithm; The step of acquiring facial information of the target user and extracting nose features and mouth features from the facial information includes: According to the facial information of the target user, nose features and mouth features are extracted from the facial information according to the target facial model and the target key point matching algorithm.

6. The method for judging and reminding mouth and nose occlusion according to claim 1, characterized in that: After determining whether the target user has mouth and nose obstruction according to the nose feature and the mouth feature, before determining whether the duration of the target user being in the mouth and nose obstruction state reaches a preset duration when determining that the target user has mouth and nose obstruction, the method further includes: When it is determined that the target user has an obstruction of the mouth and nose, adjusting the monitoring direction of the monitoring device according to the completeness of the facial features in the facial information to obtain the adjusted facial information of the target user; wherein the completeness of the facial features in the adjusted facial information is greater than the completeness of the facial features in the facial information; According to the nose features and mouth features in the adjusted facial information, it is determined again whether the target user has mouth and nose occlusion.

7. The method for judging and reminding mouth and nose occlusion according to claim 1, characterized in that: Before acquiring the facial information of the target user and extracting the nose feature and the mouth feature from the facial information, the method further includes: Detect whether the target user in the sleeping area has entered a sleeping state; The step of acquiring facial information of the target user and extracting nose features and mouth features from the facial information includes: When it is determined that the target user in the sleeping area enters a sleeping state, facial information of the target user is acquired, and nose features and mouth features are extracted from the facial information.

8. A mouth and nose occlusion judgment and reminder device, characterized in that: The device comprises: A feature extraction module, used to obtain facial information of a target user and extract nose features and mouth features from the facial information; A first judgment module is used to judge whether the target user has mouth and nose occlusion according to the nose feature and the mouth feature; The second judgment module is used to judge whether the duration of the target user being in the mouth and nose blocking state reaches a preset duration when it is determined that the target user is in the mouth and nose blocking state; The occlusion reminder module is used to send a reminder message to the target device when it is determined that the duration of the target user's mouth and nose occlusion state reaches the preset duration.

9. A computer storage medium, characterized in that The computer storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the method steps according to any one of claims 1 to 7.

10. An electronic device, characterized in that: include: A processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the method steps as claimed in any one of claims 1 to 7.

Images