A method, device, terminal and storage medium for detecting user leaving bed

Abstract

The present invention provides a method, device, terminal and storage medium for detecting a user leaving bed, which belongs to the field of video surveillance technology. The method includes: upon receiving a user leaving bed detection instruction, obtaining a real-time monitoring video stream of a target area, wherein the target area is the area where the user's bed is located; obtaining an initial background template based on the first frame of the real-time monitoring video stream, and updating the background template frame by frame for each subsequent frame and calculating a motion significance index; when the motion significance index is greater than a first preset threshold, performing user status detection based on the real-time monitoring video stream, and determining whether the user has left the bed based on the result of the user status detection. The present invention can effectively improve the accuracy of detecting when a user has left bed through a dual detection mechanism of motion significance index judgment and user status detection.

Description

Technical Field

[0001] The present invention relates to the field of video surveillance technology, and in particular to a method, device, terminal and storage medium for detecting when a user leaves bed. Background Art

[0002] With the increasing demand for intelligent monitoring in the healthcare industry, bed exit detection technology plays a vital role in the health management of the elderly, infants, and patients with limited mobility, providing significant support for individual safety. Current mainstream solutions mostly use sensor-based bed exit detection methods, which detect the user's in-bed / out-of-bed status by installing a sensor array in the mattress or bed. This detection method relies on a set threshold. When a person's weight or center of gravity changes, the fixed threshold may not accurately distinguish the actual situation, resulting in low bed exit detection accuracy. Furthermore, when the user's body position changes significantly, such as lying on their side, curling up, or being covered by bedding, the sensor may not be able to effectively sense the presence of the human body, resulting in a monitoring blind spot, further affecting the accuracy of bed exit detection.

[0003] Therefore, the existing technology has defects and needs to be improved and developed. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a method, device, terminal and storage medium for detecting user leaving bed in response to the above-mentioned defects of the prior art, aiming to solve the problem of low accuracy of user leaving bed detection in the prior art.

[0005] The technical solutions adopted by the present invention to solve the technical problems are as follows:

[0006] In a first aspect, an embodiment of the present invention provides a method for detecting when a user leaves bed, the method comprising:

[0007] When receiving a user leaving bed detection instruction, obtaining a real-time monitoring video stream of a target area, where the target area is the area where the user's bed is located;

[0008] Obtaining an initial background template based on the first frame of the real-time monitoring video stream, and updating the background template frame by frame for subsequent frames and calculating a motion saliency index;

[0009] When the motion significance index is greater than a first preset threshold, user status detection is performed based on the real-time monitoring video stream, and whether the user leaves the bed is determined according to the result of the user status detection.

[0010] In one embodiment, an initial background template is obtained based on the first frame of the real-time monitoring video stream, and the background template is updated frame by frame for each subsequent frame and a motion significance index is calculated, including:

[0011] Converting each frame of the real-time monitoring video stream into a grayscale image in real time;

[0012] The grayscale image of the first frame of the real-time monitoring video stream is used as the initial background template, and the following steps are performed frame by frame starting from the second frame:

[0013] Dynamically update the background template to obtain the background template corresponding to the grayscale image of the current frame;

[0014] A motion saliency index of the current frame is calculated based on the grayscale image of the current frame and a corresponding background template.

[0015] In one embodiment, dynamically updating the background template to obtain the background template corresponding to the grayscale image of the current frame includes:

[0016] Based on the grayscale image of the current frame, the background template corresponding to the grayscale image of the previous frame and a preset update coefficient, the background template corresponding to the grayscale image of the current frame is obtained.

[0017] In one embodiment, calculating the motion saliency index of the current frame based on the grayscale image of the current frame and the corresponding background template includes:

[0018] Calculating a difference map between the grayscale image of the current frame and the corresponding background template;

[0019] generating a Hanning window matrix having the same size as the difference map as an edge weight matrix, and obtaining a center weight matrix based on the edge weight matrix;

[0020] A motion saliency index of the current frame is obtained by performing calculation based on the difference map, the edge weight matrix, the center weight matrix, and a preset motion difference threshold.

[0021] In one embodiment, the motion significance index of the current frame is obtained by performing calculation based on the difference map, the edge weight matrix, the center weight matrix, and a preset motion difference threshold, including:

[0022] Substituting the difference map, the edge weight matrix, the center weight matrix, and a preset motion difference threshold into a motion quality calculation formula to obtain a motion significance index of the current frame;

[0023] The motion quality calculation formula is:

[0024] ;

[0025] Where qual is the motion saliency index of the current frame, d is the difference map, weight_boarder is the edge weight matrix, weight_center is the center weight matrix, and motion_thr is the preset motion difference threshold.

[0026] In one embodiment, after obtaining an initial background template based on the first frame of the real-time monitoring video stream, and updating the background template frame by frame for subsequent frames and calculating the motion significance index, the method further includes:

[0027] When the motion significance index is less than or equal to the first preset threshold, the background template is continuously updated frame by frame and the motion significance index is calculated, and the relationship between the motion significance index and the first preset threshold is determined until an end detection instruction is received.

[0028] In one embodiment, performing user status detection based on the real-time monitoring video stream and determining whether the user has left the bed according to the result of the user status detection includes:

[0029] Perform any one of face detection, human shape detection or motion detection based on the real-time monitoring video stream;

[0030] If face detection is performed based on the real-time monitoring video stream, it is determined that the user has left the bed when the number of times that the face is not detected within the first preset number of frames exceeds a second preset threshold;

[0031] If human shape detection is performed based on the real-time monitoring video stream, when the result of human shape detection is

[0032] When the number of times that the human body outline is not detected within the first preset number of frames exceeds a second preset threshold, it is determined that the user has left the bed;

[0033] If motion detection is performed based on the real-time monitoring video stream, it is determined that the user has left the bed when the result of the motion detection is that the proportion of motion vectors in the bed edge area that meet the trajectory continuity condition detected within a second preset number of frames exceeds a preset proportion;

[0034] The trajectory continuity condition is that the angle between the moving direction and the normal line of the bed edge is less than a preset angle threshold, and the moving speed is greater than a preset speed threshold.

[0035] In a second aspect, an embodiment of the present invention further provides a device for detecting when a user leaves bed, the device comprising:

[0036] A video acquisition module is used to acquire a real-time monitoring video stream of a target area after receiving a user leaving bed detection instruction, where the target area is the area where the user's bed is located;

[0037] A motion detection module, configured to obtain an initial background template based on the first frame of the real-time monitoring video stream, and to update the background template frame by frame for each subsequent frame and calculate a motion saliency index;

[0038] The bed leaving determination module is used to perform user status detection based on the real-time monitoring video stream when the motion significance index is greater than a first preset threshold, and determine whether the user has left the bed according to the result of the user status detection.

[0039] In a third aspect, an embodiment of the present invention further provides a terminal, comprising: a memory, a processor, and a user leaving bed detection program stored in the memory and executable on the processor, wherein the user leaving bed detection program, when executed by the processor, implements the steps of the user leaving bed detection method described above.

[0040] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a user leaving bed detection program, and the user leaving bed detection program can be executed to implement the steps of the user leaving bed detection method as described above.

[0041] Beneficial effects of the present invention: Upon receiving a user leaving bed detection instruction, the present invention obtains a real-time monitoring video stream of a target area, where the target area is the area where the user's bed is located; obtains an initial background template based on the first frame of the real-time monitoring video stream, and updates the background template frame by frame for each subsequent frame and calculates a motion significance index; when the motion significance index is greater than a first preset threshold, performs user status detection based on the real-time monitoring video stream, and determines whether the user has left the bed based on the result of the user status detection. The present invention can effectively improve the accuracy of user leaving bed detection through a dual detection mechanism of motion significance index judgment and user status detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 4 is a flow chart of a preferred embodiment of the method for detecting a user leaving bed in the present invention.

[0043] Figure 2 It is a schematic diagram of the process flow of detecting and processing the user leaving the bed in the present invention.

[0044] Figure 3 It is a structural diagram of a preferred embodiment of the device for detecting when a user leaves bed in the present invention.

[0045] Figure 4 It is a schematic diagram of the terminal structure of the present invention. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solutions and advantages of the present invention more clear and distinct, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0047] With the increasing demand for intelligent monitoring in the healthcare industry, bed exit detection technology plays a vital role in the health management of the elderly, infants, and patients with limited mobility, providing significant support for individual safety. Current mainstream solutions mostly use sensor-based bed exit detection methods, which detect the user's in-bed / out-of-bed status by installing a sensor array in the mattress or bed. This detection method relies on a set threshold. When a person's weight or center of gravity changes, the fixed threshold may not accurately distinguish the actual situation, resulting in low bed exit detection accuracy. Furthermore, when the user's body position changes significantly, such as lying on their side, curling up, or being covered by bedding, the sensor may not be able to effectively sense the presence of the human body, resulting in a monitoring blind spot, further affecting the accuracy of bed exit detection.

[0048] In response to the above-mentioned deficiencies in the prior art, the present invention provides a method, device, terminal, and storage medium for detecting when a user has left bed. The method comprises: obtaining a real-time monitoring video stream of a target area, where the target area is the area where the user's bed is located; obtaining an initial background template based on the first frame of the real-time monitoring video stream, and updating the background template frame by frame for each subsequent frame and calculating a motion significance index; when the motion significance index is greater than a first preset threshold, performing user status detection based on the real-time monitoring video stream, and determining whether the user has left bed based on the result of the user status detection. The present invention can effectively improve the accuracy of detecting when a user has left bed through a dual detection mechanism of motion significance index judgment and user status detection.

[0049] It should be noted that the real-time monitoring video stream acquisition and processing, face detection processing, human shape detection processing and motion detection processing involved in this application have all been authorized by the user or his legal guardian, and strictly comply with relevant laws, regulations and standards.

[0050] See Figure 1 The method for detecting when a user leaves bed according to an embodiment of the present invention comprises the following steps:

[0051] Step S100: After receiving a user leaving bed detection instruction, a real-time monitoring video stream of a target area is obtained, where the target area is the area where the user's bed is located.

[0052] Specifically, in order to meet the safety monitoring needs of users such as the elderly, infants and patients with limited mobility, the present invention deploys cameras in the user's living space and conducts real-time monitoring of the area where the user's bed is located. After receiving the user leaving bed detection instruction, the real-time monitoring video stream of the target area is obtained, and the real-time monitoring video stream is continuously processed to determine whether the user has left the bed. The initiation time of the user leaving bed detection instruction and the initiation time of the end detection instruction can be customized according to the user's work and rest time. For example, when the user is an elderly person, the user leaving bed detection instruction can be set to be initiated at 21:00 at night and the end detection instruction can be initiated at 8:00 in the morning; when the user is an infant, the user leaving bed detection instruction can be set to be initiated at 20:00 at night and the end detection instruction can be initiated at 10:00 in the morning. The system will stop detection during the non-detection time period, effectively saving system resources.

[0053] See Figure 1 The method for detecting when a user leaves the bed according to an embodiment of the present invention further includes the following steps:

[0054] Step S200: obtaining an initial background template based on the first frame of the real-time monitoring video stream, and updating the background template frame by frame for subsequent frames and calculating a motion significance index.

[0055] Specifically, by updating the background template frame by frame and calculating the motion significance index, changes in each frame can be accurately determined. The motion significance index quantifies the significance of abnormal motion in a video frame. Subsequent comparison of the motion significance index with a first preset threshold can determine whether the user is at risk of leaving the bed.

[0056] In one implementation, an initial background template is obtained based on the first frame of the real-time monitoring video stream, and the background template is updated frame by frame for each subsequent frame and a motion significance index is calculated, including:

[0057] Converting each frame of the real-time monitoring video stream into a grayscale image in real time;

[0058] The grayscale image of the first frame of the real-time monitoring video stream is used as the initial background template, and the following steps are performed frame by frame starting from the second frame:

[0059] Dynamically update the background template to obtain the background template corresponding to the grayscale image of the current frame;

[0060] A motion saliency index of the current frame is calculated based on the grayscale image of the current frame and a corresponding background template.

[0061] Specifically, after acquiring the monitoring video stream, it is converted into a grayscale image frame by frame. Subsequent calculations are based on the grayscale image corresponding to each frame, which can reduce computational complexity and bandwidth usage. Starting from the second frame, the background template is updated frame by frame. Based on the grayscale image of the current frame, the background template corresponding to the grayscale image of the previous frame, and the preset update coefficient, the background template corresponding to the grayscale image of the current frame is obtained. Specifically, the grayscale image of the current frame, the background template corresponding to the grayscale image of the previous frame, and the preset update coefficient are substituted into the background template update formula:

[0062] ;

[0063] In the formula, template n The background template corresponding to the grayscale image of the nth frame, template n-1 is the background template corresponding to the grayscale image of the n-1th frame, α is the update coefficient, which is between 0 and 1. n is the grayscale image of the current frame, and n is the subscript representing the frame number. Through weighted averaging, the background template gradually incorporates pixel information from the current frame. This allows the background template to be updated using only single-frame information, eliminating the need to store multiple historical frames. This reduces storage overhead and improves data processing efficiency. By updating the background template for each frame, interference from lighting changes and the movement of fixed objects can be eliminated, making it more accurate to determine when the user has left the bed.

[0064] In one implementation, calculating the motion saliency index of the current frame based on the grayscale image of the current frame and the corresponding background template includes:

[0065] Calculating a difference map between the grayscale image of the current frame and the corresponding background template;

[0066] generating a Hanning window matrix having the same size as the difference map as an edge weight matrix, and obtaining a center weight matrix based on the edge weight matrix;

[0067] A motion saliency index of the current frame is obtained by performing calculation based on the difference map, the edge weight matrix, the center weight matrix, and a preset motion difference threshold.

[0068] Specifically, the absolute difference between the grayscale image of the current frame and its corresponding background template is taken pixel by pixel to generate a difference map d. The larger the pixel difference in the difference map, the more significant the motion at that location (e.g., a sudden change in pixel brightness caused by human movement). A Hanning window matrix of the same size as the difference map is generated as the edge weight matrix weight_boarder. Then, through element-by-element complementation, 1 is subtracted from each element in the edge weight matrix to obtain the center weight matrix weight_center. The difference map, edge weight matrix, center weight matrix, and a preset motion difference threshold are substituted into the motion quality calculation formula to obtain the motion saliency index for the current frame.

[0069] The motion quality calculation formula is:

[0070] ;

[0071] Where qual is the motion saliency index for the current frame, d is the difference map, weight_boarder is the edge weight matrix, weight_center is the center weight matrix, and motion_thr is the preset motion difference threshold. Weighted processing effectively emphasizes image edges and avoids processing smoother areas. Multiplying the difference map d pixel-by-pixel by the edge weight matrix weight_boarder yields a weighted edge difference map. Multiplying the difference map d pixel-by-pixel by the center weight matrix weight_center yields a weighted center difference map. A preset motion difference threshold, motion_thr, is used to determine whether the difference is sufficiently significant. By comparing the edge difference map with the motion difference threshold, a binary mask, mask_border, is generated for the edge region. Pixels marked as 1 in this mask indicate significant motion, while pixels marked as 0 indicate the absence of significant motion. By comparing the center difference map with the motion difference threshold, a binary mask, mask_center, is generated for the center region. Pixels marked as 1 in this mask indicate significant motion, while pixels marked as 0 indicate the absence of significant motion. It can be understood that the numerator in the quality calculation formula is the total number of pixels with significant movement in the edge area, which reflects the intensity of the edge change. The denominator is the total number of pixels with significant movement in the center area plus 1. Adding a constant 1 to the denominator can avoid the situation where the denominator is 0 due to the user's complete lack of movement in the center area, which facilitates subsequent bed-leaving judgment. The present invention uses the Hanning window to assign high weights to the edge areas of the image and low weights to the center area, thereby enhancing the sensitivity to peripheral movement. By calculating the motion significance index, dynamic changes can be keenly perceived, and whether bed-leaving behavior occurs can be accurately judged, reducing false alarms caused by interference such as local limb movement. In addition, this method can also reduce false alarms caused by sensor aging, contamination or occlusion.

[0072] See Figure 1The method for detecting when a user leaves the bed according to an embodiment of the present invention further includes the following steps:

[0073] Step S300: When the motion significance index is greater than a first preset threshold, user status detection is performed based on the real-time monitoring video stream, and whether the user has left the bed is determined according to the result of the user status detection.

[0074] Specifically, when the motion significance index exceeds a first preset threshold, the system determines that a marginal event has been triggered, indicating a risk of leaving the bed. At this point, face detection, human shape detection, or motion detection is performed based on the real-time monitoring video stream, and the corresponding detection results determine whether the user has left the bed.

[0075] If face detection is performed based on real-time monitoring of the video stream, the user is determined to have left the bed if the number of times the face is not detected within a first preset number of frames exceeds a second preset threshold. If the number of times the face is not detected within the first preset number of frames does not exceed the second preset threshold, the user is determined to have not left the bed. Face detection can be performed by calling a pre-trained face detection model. The face detection model can be either YOLO-Face or MTCNN (Multi-task Cascaded Convolutional Networks). The YOLO-Face model is a deep learning model specifically designed and trained using the concepts and foundations of the YOLO (You Only Look Once) object detection architecture to quickly and accurately locate all faces in an image or video. The MTCNN model is a multi-task cascaded convolutional neural network that can handle face detection.

[0076] If human figure detection is performed based on a real-time monitored video stream, the user is determined to have left the bed if the number of times a human figure is not detected within a first preset number of frames exceeds a second preset threshold. If the number of times a human figure is not detected within the first preset number of frames does not exceed the second preset threshold, the user is determined to have not left the bed. Human figure detection can be performed by calling a pre-trained object detection model, which can be a YOLOv8 model.

[0077] If motion detection is performed based on the real-time monitoring video stream, then when the result of the motion detection is that the proportion of motion vectors in the edge area of ​​the bed that meet the trajectory continuity condition detected within the second preset number of frames exceeds the preset proportion, it is determined that the user has left the bed; when the result of the motion detection is that the proportion of motion vectors in the edge area of ​​the bed that meet the trajectory continuity condition detected within the second preset number of frames does not exceed the preset proportion, it is determined that the user has not left the bed. The trajectory continuity condition is that the angle between the direction of motion and the normal to the bed edge is less than a preset angle threshold, and the motion speed is greater than a preset speed threshold. Motion detection can be implemented using a motion analysis algorithm based on an optical flow field, such as the Lucas-Kanade optical flow method or the Farneback dense optical flow method, to detect pixel-level motion vectors in the edge area of ​​the bed in the real-time monitoring video stream.

[0078] When it is determined that the user has left the bed, an alarm message is generated, and a notification is then sent to preset contacts such as medical staff, guardians or family members through multiple channels such as phone calls, text messages and APP push. The present invention first uses motion significance detection to quickly screen out the time when the user may leave the bed, eliminate some irrelevant interference, and then use any one of face detection, human shape detection, and motion detection to distinguish between real leaving the bed and object interference. Through this dual detection mechanism, the accuracy of user leaving the bed can be effectively improved. The range of the first preset frame number is 60-100 frames, the range of the second preset threshold is 30-40 times, and the range of the second preset frame number is 50-70 frames.

[0079] In one implementation, Figure 2 As shown in the figure, after the video frames are obtained, the following processing is performed frame by frame: each frame is processed into a grayscale image; the background template is updated, and the boundary weight matrix and the center weight matrix are generated; the boundary difference map is obtained based on the boundary weight matrix and the difference map, and the center difference map is obtained based on the center weight matrix and the difference map; the boundary difference map and the center difference map are combined to determine whether an edge event is triggered; when an edge event is triggered (i.e., there is a risk of leaving the bed), the user status is further detected, and it is determined whether the user has left the bed based on the result of the user status detection.

[0080] In one implementation, after obtaining an initial background template based on the first frame of the real-time monitoring video stream, and updating the background template frame by frame for subsequent frames and calculating the motion significance index, the method further includes:

[0081] When the motion significance index is less than or equal to the first preset threshold, the background template is continuously updated frame by frame and the motion significance index is calculated, and the relationship between the motion significance index and the first preset threshold is determined until an end detection instruction is received.

[0082] Specifically, if the current frame's motion significance index is less than or equal to a first preset threshold, indicating that the current frame's motion significance is not significant, the system continuously determines, frame by frame, whether the motion significance index exceeds the first preset threshold. By continuously updating the background template and calculating the motion significance index frame by frame to determine whether the user has left the bed, the system can accurately and quickly respond to bed exit events, effectively ensuring user safety.

[0083] In addition to sensor-based bed exit detection methods, image-based signal analysis methods and neural network techniques have also been applied to bed exit detection. Among these, the amplitude and energy threshold method based on ballistocardiogram (BCG) signals determines whether the user has left the bed by setting amplitude and energy thresholds and monitoring whether the signal exceeds the thresholds. However, this method relies on signal amplitude and energy variations, poorly adapting to noise and interference in complex environments, and may be susceptible to external interference during the monitoring process, affecting the accuracy of bed exit detection. Furthermore, some methods using neural network technology for bed exit detection segment resting signals, body movement signals, discharge signals, and stationary signals, and input these as template signal groups into a neural network for training. Such models can automatically classify and recognize real-time online signals and output bed exit or bed presence results based on time series analysis. This method eliminates the need for manual threshold setting and offers high accuracy when processing complex signals. However, this method is prone to misjudgment in scenarios where image recognition technology cannot accurately capture complete monitoring information (e.g., when bedding is covered).

[0084] The solution of the present invention, combined with multi-dimensional image processing technology, can effectively circumvent the limitations of traditional sensors, such as susceptibility to aging and the need for physical contact. While reducing false alarm and missed alarm rates, it also overcomes signal noise interference (such as light changes and temporary occlusions) and image interference (such as missing monitoring information due to bedding obstructions) in complex environments. Furthermore, the present invention eliminates the need for deploying a large number of sensors, relying solely on visual algorithms to achieve high-precision monitoring, avoiding the cost of sensor maintenance and replacement, effectively reducing monitoring costs. Furthermore, the lack of sensor contact avoids user discomfort. This invention can provide users with safer, more intelligent, and more convenient health monitoring solutions in the fields of smart healthcare, elderly care, and infant health management.

[0085] In one implementation, after determining that the user has left the bed, the method further includes:

[0086] Generate a log to record the time when the user leaves the bed.

[0087] Specifically, each time a user leaves the bed, a log is generated to record the time the user leaves the bed. The log is stored in a structured format in the local database or cloud database. When a bed-leaving time check instruction is received, the user's bed-leaving time within the preset time range can be viewed. In this way, user behavior can be better understood, potential problems and risks can be discovered in a timely manner, and bed-leaving time can be converted into quantifiable, traceable, and analyzable behavioral indicators, providing strong data support for scenarios such as home health monitoring and institutional elderly care management.

[0088] In summary, the present invention obtains a real-time monitoring video stream of a target area upon receiving a user leaving bed detection instruction, where the target area is the area where the user's bed is located; obtains an initial background template based on the first frame of the real-time monitoring video stream, and updates the background template frame by frame for each subsequent frame and calculates a motion significance index; when the motion significance index is greater than a first preset threshold, performs user status detection based on the real-time monitoring video stream, and determines whether the user has left the bed based on the result of the user status detection. The present invention can effectively improve the accuracy of user leaving bed detection through a dual detection mechanism of motion significance index judgment and user status detection.

[0089] In one embodiment, if Figure 3 As shown, based on the above-mentioned method for detecting when a user leaves the bed, the present invention also provides a device for detecting when a user leaves the bed, the device comprising:

[0090] The video acquisition module 100 is used to acquire a real-time monitoring video stream of a target area after receiving a user leaving bed detection instruction, where the target area is the area where the user's bed is located;

[0091] A motion detection module 200 is configured to obtain an initial background template based on the first frame of the real-time monitoring video stream, and to update the background template frame by frame for each subsequent frame and calculate a motion significance index;

[0092] The bed leaving determination module 300 is configured to perform user status detection based on the real-time monitoring video stream when the motion significance index is greater than a first preset threshold, and determine whether the user has left the bed according to the result of the user status detection.

[0093] In one embodiment, the motion detection module includes:

[0094] An image preprocessing unit, configured to convert each frame of the real-time monitoring video stream into a grayscale image in real time;

[0095] The frame-by-frame processing unit is configured to use the grayscale image of the first frame of the real-time monitoring video stream as an initial background template and, starting from the second frame, perform the following steps frame by frame: dynamically update the background template to obtain a background template corresponding to the grayscale image of the current frame; and calculate a motion saliency index of the current frame based on the grayscale image of the current frame and the corresponding background template.

[0096] In one embodiment, the apparatus further comprises:

[0097] The background template updating unit is used to obtain the background template corresponding to the grayscale image of the current frame based on the grayscale image of the current frame, the background template corresponding to the grayscale image of the previous frame and a preset updating coefficient.

[0098] In one embodiment, the apparatus further comprises:

[0099] a difference map generating unit, configured to calculate a difference map between the grayscale image of the current frame and the corresponding background template;

[0100] a weight matrix generating unit, configured to generate a Hanning window matrix having the same size as the difference map as an edge weight matrix, and obtain a center weight matrix based on the edge weight matrix;

[0101] The motion significance index calculation unit is used to calculate based on the difference map, the edge weight matrix, the center weight matrix and a preset motion difference threshold to obtain the motion significance index of the current frame.

[0102] In one embodiment, the apparatus further comprises:

[0103] a formula substitution unit, configured to substitute the difference map, the edge weight matrix, the center weight matrix, and a preset motion difference threshold into a motion quality calculation formula to obtain a motion significance index;

[0104] The motion quality calculation formula is:

[0105] ;

[0106] Where qual is the motion saliency index, d is the difference map, weight_boarder is the edge weight matrix, weight_center is the center weight matrix, and motion_thr is the preset motion difference threshold.

[0107] In one embodiment, the apparatus further comprises:

[0108] The continuous detection unit is configured to continuously update the background template frame by frame and calculate the motion significance index when the motion significance index is less than or equal to the first preset threshold, and to determine the relationship between the motion significance index and the first preset threshold until an end detection instruction is received.

[0109] In one embodiment, the apparatus further comprises:

[0110] A user status detection unit, configured to perform any one of face detection, human shape detection or motion detection based on the real-time monitoring video stream;

[0111] a first result judgment unit, configured to, if face detection is performed based on the real-time monitoring video stream, determine that the user has left the bed when the number of times that no face is detected within a first preset number of frames exceeds a second preset threshold;

[0112] a second result judgment unit, configured to, if human shape detection is performed based on the real-time monitoring video stream, determine that the user has left the bed when the number of times that the human shape detection result shows that no human outline is detected within a first preset number of frames exceeds a second preset threshold;

[0113] A third result judgment unit is configured to, if motion detection is performed based on the real-time monitoring video stream, determine that the user has left the bed when a result of the motion detection is that a proportion of motion vectors detected in the bed edge area satisfying a trajectory continuity condition within a second preset number of frames exceeds a preset proportion; wherein the trajectory continuity condition is that the angle between the motion direction and the normal to the bed edge is less than a preset angle threshold and the motion speed is greater than a preset speed threshold.

[0114] Based on the above embodiment, the present invention further provides a terminal, whose structural diagram can be as follows: Figure 4 As shown. The terminal includes a processor, a memory, a network interface and a display screen connected via a device bus. The processor of the terminal is used to provide computing and control capabilities. The memory of the terminal includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating device and a user bed leaving detection program. The internal memory provides an environment for the operation of the operating device and the user bed leaving detection program in the non-volatile storage medium. The network interface of the terminal is used to communicate with an external terminal via a network connection. When the user bed leaving detection program is executed by the processor, the steps of any one of the above-mentioned user bed leaving detection methods are implemented. The display screen of the terminal can be a liquid crystal display or an electronic ink display.

[0115] Those skilled in the art will understand that Figure 4The structural schematic diagram shown in the figure is only a schematic diagram of a partial structure related to the solution of the present invention, and does not constitute a limitation on the terminal to which the solution of the present invention is applied. The specific terminal may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0116] In one embodiment, a terminal is provided, comprising a memory, a processor, and a user leaving bed detection program stored in the memory and executable on the processor. When the user leaving bed detection program is executed by the processor, the steps of any one of the user leaving bed detection methods provided in the embodiments of the present invention are implemented.

[0117] An embodiment of the present invention further provides a computer-readable storage medium, on which a user leaving bed detection program is stored. When the user leaving bed detection program is executed by a processor, the steps of any one of the user leaving bed detection methods provided in the embodiments of the present invention are implemented.

[0118] It should be understood that the sequence numbers of the steps in the above embodiments do not imply a specific order of execution; the order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0119] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the above-mentioned device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention. The specific working process of the units and modules in the above-mentioned device can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0120] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0121] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0122] In the embodiments provided by the present invention, it should be understood that the disclosed apparatus / terminal device and method can be implemented in other ways. For example, the apparatus / terminal device embodiments described above are merely illustrative. For example, the division of the modules or units described above is merely a logical functional division. In actual implementation, other division methods may be used. For example, multiple units or components may be combined or integrated into another device, or some features may be omitted or not implemented.

[0123] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, it should be understood by those skilled in the art that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the scope of protection of the present invention.

Claims

1. A method for detecting when a user leaves bed, characterized in that: The method comprises: When receiving a user leaving bed detection instruction, obtaining a real-time monitoring video stream of a target area, where the target area is the area where the user's bed is located; Converting each frame of the real-time monitoring video stream into a grayscale image in real time; The grayscale image of the first frame of the real-time monitoring video stream is used as the initial background template, and the following steps are performed frame by frame starting from the second frame: Dynamically update the background template to obtain the background template corresponding to the grayscale image of the current frame; Calculating a difference map between the grayscale image of the current frame and the corresponding background template; generating a Hanning window matrix having the same size as the difference map as an edge weight matrix, and obtaining a center weight matrix based on the edge weight matrix; Substituting the difference map, the edge weight matrix, the center weight matrix, and a preset motion difference threshold into a motion quality calculation formula to obtain a motion significance index of the current frame; The motion quality calculation formula is: ； Where qual is the motion saliency index of the current frame, d is the difference map, weight_boarder is the edge weight matrix, weight_center is the center weight matrix, and motion_thr is the preset motion difference threshold; When the motion significance index is greater than a first preset threshold, user status detection is performed based on the real-time monitoring video stream, and whether the user leaves the bed is determined according to the result of the user status detection.

2. The method for detecting when a user leaves bed according to claim 1, wherein: Dynamically update the background template to obtain the background template corresponding to the grayscale image of the current frame, including: Based on the grayscale image of the current frame, the background template corresponding to the grayscale image of the previous frame and a preset update coefficient, the background template corresponding to the grayscale image of the current frame is obtained.

3. The method for detecting when a user leaves bed according to claim 1, wherein: After obtaining an initial background template based on the first frame of the real-time monitoring video stream, and updating the background template frame by frame for subsequent frames and calculating the motion significance index, the method further includes: When the motion significance index is less than or equal to the first preset threshold, the background template is continuously updated frame by frame and the motion significance index is calculated, and the relationship between the motion significance index and the first preset threshold is determined until an end detection instruction is received.

4. The method for detecting when a user leaves bed according to claim 1, wherein: Performing user status detection based on the real-time monitoring video stream and determining whether the user has left the bed according to the result of the user status detection includes: Perform any one of face detection, human shape detection or motion detection based on the real-time monitoring video stream; If face detection is performed based on the real-time monitoring video stream, it is determined that the user has left the bed when the number of times that the face is not detected within the first preset number of frames exceeds a second preset threshold; If human figure detection is performed based on the real-time monitoring video stream, it is determined that the user has left the bed when the result of the human figure detection is that the number of times that the human body outline is not detected within the first preset number of frames exceeds a second preset threshold; If motion detection is performed based on the real-time monitoring video stream, it is determined that the user has left the bed when the result of the motion detection is that the proportion of motion vectors in the bed edge area that meet the trajectory continuity condition detected within a second preset number of frames exceeds a preset proportion; The trajectory continuity condition is that the angle between the moving direction and the normal line of the bed edge is less than a preset angle threshold, and the moving speed is greater than a preset speed threshold.

5. A device for detecting when a user leaves bed, characterized in that: include: A video acquisition module is used to acquire a real-time monitoring video stream of a target area after receiving a user leaving bed detection instruction, where the target area is the area where the user's bed is located; A motion detection module, configured to convert each frame of the real-time monitoring video stream into a grayscale image in real time; The grayscale image of the first frame of the real-time monitoring video stream is used as the initial background template, and the following steps are performed frame by frame starting from the second frame: Dynamically update the background template to obtain the background template corresponding to the grayscale image of the current frame; Calculating a difference map between the grayscale image of the current frame and the corresponding background template; generating a Hanning window matrix having the same size as the difference map as an edge weight matrix, and obtaining a center weight matrix based on the edge weight matrix; Substituting the difference map, the edge weight matrix, the center weight matrix, and a preset motion difference threshold into a motion quality calculation formula to obtain a motion significance index of the current frame; The motion quality calculation formula is: ； Where qual is the motion saliency index of the current frame, d is the difference map, weight_boarder is the edge weight matrix, weight_center is the center weight matrix, and motion_thr is the preset motion difference threshold; The bed leaving determination module is used to perform user status detection based on the real-time monitoring video stream when the motion significance index is greater than a first preset threshold, and determine whether the user has left the bed according to the result of the user status detection.

6. A terminal, characterized in that: The terminal includes: a memory, a processor, and a user leaving bed detection program stored in the memory and executable on the processor. When the user leaving bed detection program is executed by the processor, the steps of the user leaving bed detection method according to any one of claims 1 to 4 are implemented.

7. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a user leaving bed detection program. When the user leaving bed detection program is executed by the processor, the steps of the user leaving bed detection method according to any one of claims 1 to 4 are implemented.

Images