User off-bed detection method and device, terminal and storage medium

Abstract

The invention provides a user off-bed detection method and device, a terminal and a storage medium, and belongs to the technical field of video surveillance, and the method comprises the steps: obtaining a real-time monitoring video stream of a target region after receiving a user off-bed detection instruction, the target region being a region where a user bed is located; obtaining an initial background template based on the first frame of the real-time monitoring video stream, updating the background template for each subsequent frame frame by frame, and calculating a motion saliency index; and when the motion significance index is greater than a first preset threshold value, performing user state detection based on the real-time monitoring video stream, and determining whether the user leaves the bed or not according to a user state detection result. According to the invention, through a dual detection mechanism of motion saliency index judgment and user state detection, the accuracy of user off-bed detection can be effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of video surveillance, and particularly to a method, device, terminal and storage medium for detecting a user getting out of bed. Background Art

[0002] With the increasing demand for intelligent monitoring in the medical care industry, the technology of detecting getting out of bed plays an important role in the health management of the elderly, infants and patients with inconvenient mobility, providing an important boost to ensuring individual safety. The current mainstream solutions mostly adopt the method of detecting getting out of bed based on sensors, which detect the in-bed / out-of-bed state of the user by installing a sensor array in the mattress or the bed body. The detection of this method depends on the set threshold. When the body weight or the center of gravity of the human body changes, the fixed threshold may not be able to accurately distinguish the actual situation, resulting in a low accuracy rate of detecting getting out of bed. In addition, when the user's body position changes greatly, such as lying on the side, curling up or being covered by the quilt, etc., it may also cause the sensor to be unable to effectively sense the presence of the human body, thus forming a monitoring blind area and further affecting the accuracy of detecting getting out of bed.

[0003] Therefore, there are defects in the prior art and it needs to be improved and developed. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a method, device, terminal and storage medium for detecting a user getting out of bed, aiming at solving the problem of low accuracy rate of detecting a user getting out of bed in the prior art.

[0005] The technical solution adopted by the present invention to solve the technical problem is as follows: In the first aspect, an embodiment of the present invention provides a method for detecting a user getting out of bed, the method including: After receiving a user getting out of bed detection instruction, obtaining a real-time monitoring video stream of a target area, where the target area is the area where the user's bed is located; Obtaining an initial background template based on the first frame of the real-time monitoring video stream, and sequentially performing updating the background template and calculating a motion saliency index for each subsequent frame; When the motion saliency index is greater than a first preset threshold, performing user state detection based on the real-time monitoring video stream, and determining whether the user gets out of bed according to the result of the user state detection.

[0006] In an implementation manner, obtaining an initial background template based on the first frame of the real-time monitoring video stream, and sequentially performing updating the background template and calculating a motion saliency index for each subsequent frame includes: Sequentially converting each frame of the real-time monitoring video stream into a grayscale image in real time; Use the grayscale image of the first frame of the real-time monitoring video stream as the initial background template, and starting from the second frame, perform the following steps frame by frame: Dynamically update the background template to obtain the background template corresponding to the grayscale image of the current frame; Based on the grayscale image of the current frame and the corresponding background template, calculate the motion saliency index of the current frame.

[0007] In one embodiment, dynamically updating the background template to obtain the background template corresponding to the grayscale image of the current frame includes: Based on the grayscale image of the current frame, the background template corresponding to the grayscale image of the previous frame, and a preset update coefficient, obtain the background template corresponding to the grayscale image of the current frame.

[0008] In one embodiment, based on the grayscale image of the current frame and the corresponding background template, calculating the motion saliency index of the current frame includes: Calculate the difference map between the grayscale image of the current frame and the corresponding background template; Generate a Hanning window matrix with the same size as the difference map as the edge weight matrix, and obtain the center weight matrix based on the edge weight matrix; Based on the difference map, the edge weight matrix, the center weight matrix, and a preset motion difference threshold, perform calculations to obtain the motion saliency index of the current frame.

[0009] In one embodiment, based on the difference map, the edge weight matrix, the center weight matrix, and a preset motion difference threshold, perform calculations to obtain the motion saliency index of the current frame, including: Substitute the difference map, the edge weight matrix, the center weight matrix, and the preset motion difference threshold into the motion quality calculation formula to obtain the motion saliency index of the current frame; Wherein, the motion quality calculation formula is: ; In the formula, qual is the motion saliency index of the current frame, d is the difference map, weight_boarder is the edge weight matrix, weight_center is the center weight matrix, and motion_thr is the preset motion difference threshold.

[0010] In one embodiment, after obtaining the initial background template based on the first frame of the real-time monitoring video stream and performing background template update and motion saliency index calculation frame by frame for subsequent frames, it further includes: When the motion saliency index is less than or equal to the first preset threshold, continuously update the background template frame by frame, calculate the motion saliency index, and determine the relationship between the motion saliency index and the first preset threshold until an end detection instruction is received.

[0011] In one implementation, user status detection is performed based on the real-time monitoring video stream, and whether the user gets out of bed is determined according to the result of the user status detection, including: Performing any one of face detection, human shape detection, or motion detection based on the real-time monitoring video stream; If face detection is performed based on the real-time monitoring video stream, when the number of times of no face detected within the first preset number of frames exceeds the second preset threshold, it is determined that the user gets out of bed; If human shape detection is performed based on the real-time monitoring video stream, then when the result of the human shape detection is that The number of times of no human body contour detected within the first preset number of frames exceeds the second preset threshold, it is determined that the user gets out of bed; If motion detection is performed based on the real-time monitoring video stream, when the result of the motion detection is that the proportion of the motion vectors in the edge area of the bed detected within the second preset number of frames that satisfy the trajectory persistence condition exceeds the preset proportion, it is determined that the user gets out of bed; Wherein, the trajectory persistence condition is that the included angle between the motion direction and the normal line of the bed edge is less than the preset angle threshold, and the motion speed is greater than the preset speed threshold.

[0012] In a second aspect, an embodiment of the present invention further provides a user getting out of bed detection device, and the device includes: A video acquisition module, configured to acquire a real-time monitoring video stream of a target area when a user getting out of bed detection instruction is received, and the target area is the area where the user's bed is located; A motion detection module, configured to obtain an initial background template based on the first frame of the real-time monitoring video stream, and update the background template frame by frame for subsequent frames and calculate the motion saliency index; An out-of-bed determination module, configured to perform user status detection based on the real-time monitoring video stream when the motion saliency index is greater than the first preset threshold, and determine whether the user gets out of bed according to the result of the user status detection.

[0013] In a third aspect, an embodiment of the present invention further provides a terminal, and the terminal includes: a memory, a processor, and a user getting out of bed detection program stored on the memory and executable on the processor, and when the user getting out of bed detection program is executed by the processor, the steps of the user getting out of bed detection method as described above are implemented.

[0014] Fourthly, an embodiment of the present invention further provides a computer-readable storage medium storing a user getting out of bed detection program, which can be executed to implement the steps of the user getting out of bed detection method as described above.

[0015] Advantages of the present invention: After receiving a user getting out of bed detection instruction, the present invention obtains a real-time monitoring video stream of a target area, where the target area is the area where the user's bed is located; obtains an initial background template based on the first frame of the real-time monitoring video stream, and updates the background template and calculates a motion saliency index frame by frame for subsequent frames; when the motion saliency index is greater than a first preset threshold, performs user state detection based on the real-time monitoring video stream, and determines whether the user gets out of bed according to the result of the user state detection. The present invention can effectively improve the accuracy of user getting out of bed detection through a dual detection mechanism of motion saliency index judgment and user state detection. Description of the Drawings

[0016] Figure 1 is a flowchart of a preferred embodiment of the user getting out of bed detection method in the present invention.

[0017] Figure 2 is a schematic diagram of the user getting out of bed detection processing flow in the present invention.

[0018] Figure 3 is a schematic structural diagram of a preferred embodiment of the user getting out of bed detection device in the present invention.

[0019] Figure 4 is a schematic diagram of the terminal structure in the present invention. Detailed Embodiments

[0020] To make the objectives, technical solutions and advantages of the present invention clearer and more definite, the following further describes the present invention in detail with reference to the accompanying drawings and by way of examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0021] With the increasing demand for intelligent monitoring in the medical care industry, getting out of bed detection technology plays an important role in the health management of the elderly, infants and patients with limited mobility, providing an important boost to ensuring individual safety. The current mainstream solutions mostly adopt sensor-based getting out of bed detection methods, which detect the in-bed / out-of-bed state of the user by installing a sensor array in the mattress or the bed body. The detection of this method depends on the set threshold. When the body weight or the center of gravity of the human body changes, the fixed threshold may not be able to accurately distinguish the actual situation, resulting in a low accuracy of getting out of bed detection. In addition, when the user's body position changes greatly, such as lying on the side, curling up or being covered by the quilt, etc., the sensor may not be able to effectively sense the presence of the human body, thus forming a monitoring blind spot and further affecting the accuracy of getting out of bed detection.

[0022] In view of the above defects of the prior art, the present invention provides a method, device, terminal and storage medium for detecting a user getting out of bed. The method includes: obtaining a real-time monitoring video stream of a target area, where the target area is the area where the user's bed is located; obtaining an initial background template based on the first frame of the real-time monitoring video stream, and updating the background template and calculating a motion saliency index frame by frame for subsequent frames; when the motion saliency index is greater than a first preset threshold, performing user state detection based on the real-time monitoring video stream, and determining whether the user gets out of bed according to the result of the user state detection. The present invention can effectively improve the accuracy of user getting out of bed detection through a dual detection mechanism of motion saliency index judgment and user state detection.

[0023] It should be noted that the acquisition and processing of the real-time monitoring video stream, face detection processing, human form detection processing and movement detection processing involved in this application have all obtained the authorization of the user or their legal guardian, and strictly comply with relevant laws, regulations and standards.

[0024] Please refer to Figure 1 , the user getting out of bed detection method described in the embodiment of the present invention includes the following steps: Step S100, when receiving a user getting out of bed detection instruction, obtain a real-time monitoring video stream of a target area, where the target area is the area where the user's bed is located.

[0025] Specifically, to meet the safety monitoring needs of users such as the elderly, infants and patients with limited mobility, the present invention deploys a camera in the user's living space and monitors the area where the user's bed is located in real time. When receiving a user getting out of bed detection instruction, obtain a real-time monitoring video stream of the target area and continuously process the real-time monitoring video stream to determine whether the user gets out of bed. The start time of the user getting out of bed detection instruction and the start time of the end detection instruction can be customized according to the user's work and rest time. For example, when the user is an elderly person, the user getting out of bed detection instruction can be set to be initiated at 21:00 at night, and the end detection instruction can be initiated at 8:00 in the morning; when the user is an infant, the user getting out of bed detection instruction can be set to be initiated at 20:00 at night, and the end detection instruction can be initiated at 10:00 in the morning. The system will stop detection during non-detection time periods, effectively saving system resources.

[0026] Please refer to Figure 1 , the user getting out of bed detection method described in the embodiment of the present invention further includes the following steps: Step S200, obtain an initial background template based on the first frame of the real-time monitoring video stream, and update the background template and calculate a motion saliency index frame by frame for subsequent frames.

[0027] Specifically, by updating the background template frame by frame for each frame and calculating the motion saliency index, the change of each frame can be accurately judged. The motion saliency index is a value that quantifies the significance degree of abnormal motion regions in a video frame. Subsequently, by comparing the motion saliency index with a first preset threshold, it can be determined whether the user has a risk of getting out of bed.

[0028] In one implementation, an initial background template is obtained based on the first frame of the real-time monitoring video stream, and for each subsequent frame, updating the background template frame by frame and calculating the motion saliency index includes: Convert each frame of the real-time monitoring video stream into a grayscale image in real time; Use the grayscale image of the first frame of the real-time monitoring video stream as the initial background template, and starting from the second frame, perform the following steps frame by frame: Dynamically update the background template to obtain the background template corresponding to the grayscale image of the current frame; Based on the grayscale image of the current frame and the corresponding background template, calculate the motion saliency index of the current frame.

[0029] Specifically, after obtaining the monitoring video stream, convert it into a grayscale image frame by frame. Subsequent calculations are based on the grayscale image corresponding to each frame, which can reduce the computational complexity and bandwidth occupancy. Starting from the second frame, update the background template frame by frame. Based on the grayscale image of the current frame, the background template corresponding to the grayscale image of the previous frame, and a preset update coefficient, obtain the background template corresponding to the grayscale image of the current frame. Specifically, substitute the grayscale image of the current frame, the background template corresponding to the grayscale image of the previous frame, and the preset update coefficient into the background template update formula: ; In the formula, template n is the background template corresponding to the grayscale image of the nth frame, template n-1 is the background template corresponding to the grayscale image of the (n - 1)th frame, α is the update coefficient, which is between 0 and 1. im n is the grayscale image of the current frame, and n is the subscript representing the frame number. Through weighted averaging, the background template will gradually absorb the pixel information of the current frame. The background template can be updated only through single-frame information, without excessive storage of historical frames, reducing the storage overhead and improving the data processing efficiency. By updating the background template for each frame, interference information such as light changes and fixed object movements can be eliminated, making the judgment of the user getting out of bed more accurate.

[0030] In one implementation, based on the grayscale image of the current frame and the corresponding background template, calculating the motion saliency index of the current frame includes: Calculate the difference map between the grayscale image of the current frame and the corresponding background template; Generate a Hanning window matrix with the same size as the difference map as the edge weight matrix, and obtain the central weight matrix based on the edge weight matrix; Perform calculations based on the difference map, the edge weight matrix, the central weight matrix, and a preset motion difference threshold to obtain the motion saliency index of the current frame.

[0031] Specifically, take the absolute difference between the grayscale image of the current frame and its corresponding background template pixel by pixel to generate a difference map d. The larger the difference value of the pixels in the difference map, the more significant the motion at that position (such as the sudden change in pixel brightness caused by human movement). Generate a Hanning window matrix with the same size as the difference map as the edge weight matrix weight_boarder, and through element-wise complementary operation, subtract each element in the edge weight matrix from 1 to obtain the central weight matrix weight_center. Substitute the difference map, the edge weight matrix, the central weight matrix, and a preset motion difference threshold into the motion quality calculation formula to obtain the motion saliency index of the current frame.

[0032] Among them, the motion quality calculation formula is: ; Where qual is the motion saliency index of the current frame, d is the difference map, weight_boarder is the edge weight matrix, weight_center is the center weight matrix, and motion_thr is a preset motion difference threshold. The weighted processing can effectively emphasize the edge part of the image and avoid processing relatively smooth areas. Multiplying the difference map d by the edge weight matrix weight_boarder pixel by pixel can obtain the weighted edge difference map. Multiplying the difference map d by the center weight matrix weight_center pixel by pixel can obtain the weighted center difference map. A motion difference threshold motion_thr is preset to determine whether the difference value is significant enough. By comparing the edge difference map with the motion difference threshold, a binary mask mask_border for the edge region is generated. The pixels marked as 1 in this mask indicate significant motion, and the pixels marked as 0 indicate no significant motion. By comparing the center difference map with the motion difference threshold, a binary mask mask_center for the center region is generated. The pixels marked as 1 in this mask indicate significant motion, and the pixels marked as 0 indicate no significant motion. It can be understood that the numerator in the quality calculation formula is the total number of pixels with significant motion in the edge region, reflecting the intensity of edge changes. The denominator is the total number of pixels with significant motion in the center region plus 1. Adding the constant 1 to the denominator can avoid the situation where the denominator is 0 due to the user having no motion at all in the center region, facilitating subsequent determination of getting out of bed. The present invention uses a Hanning window to assign a high weight to the edge region of the image and a low weight to the center region, enhancing the sensitivity to peripheral motion. Through the calculation of the motion saliency index, dynamic changes can be keenly perceived, and it can accurately judge whether the getting out of bed behavior occurs, reducing false alarms caused by interference such as local movement of limbs. In addition, this method can also reduce false alarms caused by sensor aging, pollution or occlusion.

[0033] Please refer to Figure 1 , the method for detecting a user getting out of bed according to the embodiment of the present invention further includes the following steps: Step S300, when the motion saliency index is greater than a first preset threshold, perform user status detection based on the real-time monitoring video stream, and determine whether the user gets out of bed according to the result of the user status detection.

[0034] Specifically, when the motion saliency index exceeds the first preset threshold, the system determines that an edge event is triggered, that is, there is a risk of getting out of bed. At this time, perform any one of face detection, human shape detection or movement detection based on the real-time monitoring video stream, and determine whether the user gets out of bed according to the corresponding detection result.

[0035] If face detection is performed based on the real-time monitoring video stream, when the number of times no face is detected within the first preset number of frames in the face detection result exceeds the second preset threshold, it is determined that the user gets out of bed; when the number of times no face is detected within the first preset number of frames in the face detection result does not exceed the second preset threshold, it is determined that the user does not get out of bed. The face detection method can be to call a pre-trained face detection model for face detection. The face detection model can be any one of YOLO-Face and MTCNN (Multi-task Cascaded Convolutional Networks). The YOLO-Face model is specially designed and trained using the idea and basis of the YOLO (You Only Look Once) object detection architecture to quickly and accurately locate all faces in an image or video. The MTCNN model is a multi-task cascaded convolutional neural network that can handle face detection.

[0036] If human detection is performed based on the real-time monitoring video stream, when the number of times no human contour is detected within the first preset number of frames in the human detection result exceeds the second preset threshold, it is determined that the user gets out of bed; when the number of times no human contour is detected within the first preset number of frames in the human detection result does not exceed the second preset threshold, it is determined that the user does not get out of bed. The human detection method can be to call a pre-trained object detection model for human detection, and the object detection model can be the YOLOv8 model.

[0037] If motion detection is performed based on the real-time monitoring video stream, when the proportion of the motion vectors in the bed edge area that satisfy the trajectory persistence condition detected within the second preset number of frames in the motion detection result exceeds the preset proportion, it is determined that the user gets out of bed; when the proportion of the motion vectors in the bed edge area that satisfy the trajectory persistence condition detected within the second preset number of frames in the motion detection result does not exceed the preset proportion, it is determined that the user does not get out of bed. The trajectory persistence condition is that the angle between the motion direction and the normal of the bed edge is less than the preset angle threshold, and the motion speed is greater than the preset speed threshold. Motion detection can be implemented using a motion analysis algorithm based on the optical flow field. For example, the Lucas-Kanade optical flow method or the Farneback dense optical flow method can be used to detect the pixel-level motion vectors in the bed edge area of the real-time monitoring video stream.

[0038] After determining that the user has gotten out of bed, an alarm message is generated and then sent as a notification to preset contacts such as medical staff, guardians, or family members through multiple channels including phone calls, text messages, and APP push. First, through motion saliency detection, the present invention can quickly screen out the possible time of getting out of bed, excluding some irrelevant interferences, and then distinguish real getting out of bed from object interferences through any one of face detection, human shape detection, and movement detection. Through this dual-detection mechanism, the accuracy of user getting-out-of-bed detection can be effectively improved. The range of the first preset number of frames is 60 - 100 frames, the range of the second preset threshold is 30 - 40 times, and the range of the second preset number of frames is 50 - 70 frames.

[0039] In one implementation, as Figure 2 shown, after obtaining a video frame, the following processing is performed frame by frame: each frame is processed into a grayscale image; the background template is updated, and a boundary weight matrix and a center weight matrix are generated; a boundary difference map is obtained based on the boundary weight matrix and the difference map, and a center difference map is obtained based on the center weight matrix and the difference map; the boundary difference map and the center difference map are combined to determine whether an edge event is triggered; when an edge event is triggered (i.e., there is a risk of getting out of bed), the user status is further detected, and it is determined whether the user has gotten out of bed according to the result of the user status detection.

[0040] In one implementation, after obtaining an initial background template based on the first frame of the real-time monitoring video stream and performing frame-by-frame update of the background template and calculation of the motion saliency index for subsequent frames, it further includes: When the motion saliency index is less than or equal to the first preset threshold, continuously update the background template frame by frame, calculate the motion saliency index, and judge the relationship between the motion saliency index and the first preset threshold until an end detection instruction is received.

[0041] Specifically, when the motion saliency index of the current frame is less than or equal to the first preset threshold, it means that the motion saliency of the current frame is not obvious, so it is continuously judged frame by frame whether the motion saliency index exceeds the first preset threshold. By continuously updating the background template and calculating the motion saliency index frame by frame to judge whether the user has gotten out of bed, the getting-out-of-bed event can be accurately and quickly responded to, effectively ensuring the safety of the user.

[0042] In the prior art, in addition to the sensor-based out-of-bed detection method, the image-based signal analysis method and the neural network technology method are also applied in out-of-bed detection. Among them, the amplitude and energy threshold method based on the ballistocardiogram (BCG) signal monitors whether the signal exceeds the set threshold by setting the amplitude and energy thresholds to determine whether the user gets out of bed. However, this method depends on the amplitude and energy changes of the signal, and cannot well cope with the noise and interference of the signal in a complex environment, and may be affected by external interference during the monitoring process, affecting the accuracy of out-of-bed detection. In addition, some methods using neural network technology for out-of-bed detection segment static lying signals, body movement signals, discharge signals, and stable signals, and input them into the neural network as a template signal group for training. Such models can automatically classify and identify real-time collected online signals, and output the result of getting out of bed or being in bed according to time series analysis. This method no longer requires manual threshold setting and has high accuracy in processing complex signals. However, this method is prone to misjudgment in scenarios where the image recognition technology cannot accurately obtain complete monitoring information (such as being covered by quilts).

[0043] The solution of the present invention combines multi-dimensional image processing technology, which can effectively avoid the limitations of traditional sensors such as easy aging and physical contact, reduce the false alarm rate and missed alarm rate, and overcome the signal noise interference (such as light changes and temporary occlusion) and image interference (such as missing monitoring information caused by quilt occlusion) in a complex environment. In addition, the present invention does not require the deployment of a large number of sensors, and only relies on vision algorithms to achieve high-precision monitoring, avoiding the maintenance and replacement costs of sensors, and can effectively reduce the monitoring cost. At the same time, there is no need for sensor contact, avoiding user discomfort. The present invention can provide a safer, smarter and more convenient health monitoring solution for users in the fields of intelligent healthcare, elderly care and infant health management.

[0044] In one implementation, after determining that the user gets out of bed, it further includes: Generating a log to record the time when the user gets out of bed.

[0045] Specifically, each time it is determined that the user gets out of bed, a log is generated to record the time when the user gets out of bed. The log is stored in the local database or the cloud database in a structured format. When receiving an out-of-bed time viewing instruction, the out-of-bed time of the user within a preset time range can be viewed. In this way, the user's behavior can be better understood, potential problems and risks can be discovered in time, and the out-of-bed time can be converted into a quantifiable, traceable and analyzable behavior indicator, providing strong data support for scenarios such as home health monitoring and institutional elderly care management.

[0046] In summary, after receiving the user getting out of bed detection instruction, the present invention obtains the real-time monitoring video stream of the target area, where the target area is the area where the user's bed is located; obtains the initial background template based on the first frame of the real-time monitoring video stream, and updates the background template frame by frame for subsequent frames and calculates the motion saliency index; when the motion saliency index is greater than the first preset threshold, performs user status detection based on the real-time monitoring video stream, and determines whether the user gets out of bed according to the result of the user status detection. The present invention can effectively improve the accuracy of user getting out of bed detection through the dual detection mechanisms of motion saliency index judgment and user status detection.

[0047] In one embodiment, as Figure 3 shown, based on the above user getting out of bed detection method, the present invention also correspondingly provides a user getting out of bed detection device, and the device includes: A video acquisition module 100, configured to obtain the real-time monitoring video stream of the target area after receiving the user getting out of bed detection instruction, where the target area is the area where the user's bed is located; A motion detection module 200, configured to obtain the initial background template based on the first frame of the real-time monitoring video stream, and update the background template frame by frame for subsequent frames and calculate the motion saliency index; A getting out of bed determination module 300, configured to perform user status detection based on the real-time monitoring video stream when the motion saliency index is greater than the first preset threshold, and determine whether the user gets out of bed according to the result of the user status detection.

[0048] In one embodiment, the motion detection module includes: An image preprocessing unit, configured to convert each frame of the real-time monitoring video stream into a grayscale image in real time; A frame-by-frame processing unit, configured to use the grayscale image of the first frame of the real-time monitoring video stream as the initial background template, and perform the following steps frame by frame starting from the second frame: dynamically update the background template to obtain the background template corresponding to the grayscale image of the current frame; calculate the motion saliency index of the current frame based on the grayscale image of the current frame and the corresponding background template.

[0049] In one embodiment, the device further includes: A background template update unit, configured to obtain the background template corresponding to the grayscale image of the current frame based on the grayscale image of the current frame, the background template corresponding to the grayscale image of the previous frame, and a preset update coefficient.

[0050] In one embodiment, the device further includes: A difference map generation unit, configured to calculate the difference map between the grayscale image of the current frame and the corresponding background template; A weight matrix generation unit, configured to generate a Hanning window matrix having the same size as the difference map as an edge weight matrix, and obtain a center weight matrix based on the edge weight matrix; A motion saliency index calculation unit, configured to calculate based on the difference map, the edge weight matrix, the center weight matrix, and a preset motion difference threshold to obtain the motion saliency index of the current frame.

[0051] In one embodiment, the apparatus further includes: A formula substitution unit, configured to substitute the difference map, the edge weight matrix, the center weight matrix, and a preset motion difference threshold into a motion quality calculation formula to obtain a motion saliency index; Wherein, the motion quality calculation formula is: ; In the formula, qual is the motion saliency index, d is the difference map, weight_boarder is the edge weight matrix, weight_center is the center weight matrix, and motion_thr is the preset motion difference threshold.

[0052] In one embodiment, the apparatus further includes: A continuous detection unit, configured to continuously update the background template frame by frame and calculate the motion saliency index when the motion saliency index is less than or equal to a first preset threshold, and determine the relationship between the motion saliency index and the first preset threshold until an end detection instruction is received.

[0053] In one embodiment, the apparatus further includes: A user state detection unit, configured to perform any one of face detection, human shape detection, or movement detection based on the real-time monitoring video stream; A first result judgment unit, configured to, if face detection is performed based on the real-time monitoring video stream, determine that the user gets out of bed when the number of times of no face detected within a first preset number of frames exceeds a second preset threshold; A second result judgment unit, configured to, if human shape detection is performed based on the real-time monitoring video stream, determine that the user gets out of bed when the number of times of no human body contour detected within a first preset number of frames exceeds a second preset threshold; A third result judgment unit, configured to, if movement detection is performed based on the real-time monitoring video stream, determine that the user gets out of bed when the proportion of the motion vectors in the edge area of the bed detected within a second preset number of frames satisfying the trajectory persistence condition exceeds a preset proportion; wherein, the trajectory persistence condition is that the included angle between the motion direction and the normal line of the bed edge is less than a preset angle threshold, and the motion speed is greater than a preset speed threshold.

[0054] Based on the above embodiments, the present invention further provides a terminal, and its structural schematic diagram can be as Figure 4 shown. The above terminal includes a processor, a memory, a network interface, and a display screen connected through a device bus. Among them, the processor of the terminal is used to provide computing and control capabilities. The memory of the terminal includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating device and a user getting out of bed detection program. The internal memory provides an environment for the operation of the operating device and the user getting out of bed detection program in the non-volatile storage medium. The network interface of the terminal is used to communicate with an external terminal through a network connection. When the user getting out of bed detection program is executed by the processor, it implements the steps of any one of the above user getting out of bed detection methods. The display screen of the terminal can be a liquid crystal display screen or an electronic ink display screen.

[0055] Those skilled in the art can understand that Figure 4 the structural schematic diagram shown in

[0056] is only a schematic diagram of some structures related to the solution of the present invention, and does not constitute a limitation on the terminal to which the solution of the present invention is applied. The specific terminal may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0057] The embodiments of the present invention further provide a computer-readable storage medium, on which a user getting out of bed detection program is stored. When the user getting out of bed detection program is executed by a processor, it implements the steps of any one of the user getting out of bed detection methods provided by the embodiments of the present invention.

[0058] It should be understood that the sequence numbers of the above steps do not mean the order of execution is prior or posterior. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0059] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of each functional unit and module is used as an example. In actual applications, the above-mentioned functions can be allocated to different functional units and modules according to needs, that is, the internal structure of the above-mentioned device can be divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. In addition, the specific names of each functional unit and module are only for the convenience of mutual distinction and do not limit the protection scope of the present invention. The specific working process of the units and modules in the above-mentioned device can refer to the corresponding process in the foregoing method embodiment and will not be elaborated herein.

[0060] In the above embodiments, the descriptions of each embodiment have their own emphases. For the parts not detailed or recorded in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0061] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or by a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of the present invention.

[0062] In the embodiments provided by the present invention, it should be understood that the disclosed device / terminal device and method can be implemented in other ways. For example, the device / terminal device embodiments described above are only illustrative. For example, the above-mentioned division of modules or units is only a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed.

[0063] The above-mentioned embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features. And these modifications or replacements do not deviate from the spirit and scope of the technical solutions of the present invention in essence, and should all be included in the protection scope of the present invention.

Claims

1. A method for detecting a user getting out of bed, characterized in that The method includes: After receiving a user getting out of bed detection instruction, obtain a real-time monitoring video stream of a target area, where the target area is the area where the user's bed is located; Based on the first frame of the real-time monitoring video stream, obtain an initial background template, and update the background template frame by frame for subsequent frames and calculate the motion saliency index; When the motion saliency index is greater than a first preset threshold, perform user state detection based on the real-time monitoring video stream, and determine whether the user gets out of bed according to the result of the user state detection.

2. The user getting out of bed detection method according to claim 1, wherein, Based on the first frame of the real-time monitoring video stream, obtain an initial background template, and update the background template frame by frame for subsequent frames and calculate the motion saliency index, including: Convert each frame of the real-time monitoring video stream into a grayscale image in real time; Use the grayscale image of the first frame of the real-time monitoring video stream as the initial background template, and perform the following steps frame by frame starting from the second frame: Dynamically update the background template to obtain the background template corresponding to the grayscale image of the current frame; Based on the grayscale image of the current frame and the corresponding background template, calculate the motion saliency index of the current frame.

3. The user out-of-bed detection method according to claim 2, wherein, Dynamically update the background template to obtain the background template corresponding to the grayscale image of the current frame, including: Based on the grayscale image of the current frame, the background template corresponding to the grayscale image of the previous frame, and a preset update coefficient, obtain the background template corresponding to the grayscale image of the current frame.

4. The user out-of-bed detection method according to claim 2, wherein, Based on the grayscale image of the current frame and the corresponding background template, calculate the motion saliency index of the current frame, including: Calculate the difference map between the grayscale image of the current frame and the corresponding background template; Generate a Hanning window matrix with the same size as the difference map as the edge weight matrix, and obtain the center weight matrix based on the edge weight matrix; Perform calculations based on the difference map, the edge weight matrix, the center weight matrix, and a preset motion difference threshold to obtain the motion saliency index of the current frame.

5. The user out-of-bed detection method according to claim 4, wherein, Perform calculations based on the difference map, the edge weight matrix, the center weight matrix, and a preset motion difference threshold to obtain the motion saliency index of the current frame, including: Substitute the difference map, the edge weight matrix, the center weight matrix, and the preset motion difference threshold into the motion quality calculation formula to obtain the motion saliency index of the current frame; wherein, the motion quality calculation formula is: ； In the formula, qual is the motion saliency index of the current frame, d is the difference map, weight_boarder is the edge weight matrix, weight_center is the center weight matrix, and motion_thr is the preset motion difference threshold.

6. The user out-of-bed detection method according to claim 1, characterized in that After obtaining an initial background template based on the first frame of the real-time monitoring video stream, and updating the background template frame by frame for subsequent frames and calculating the motion saliency index, it further includes: When the motion saliency index is less than or equal to the first preset threshold, continuously update the background template frame by frame and calculate the motion saliency index, and judge the relationship between the motion saliency index and the first preset threshold until a detection end instruction is received.

7. The user out-of-bed detection method according to claim 1, characterized in that Performing user status detection based on the real-time monitoring video stream, and determining whether the user gets out of bed according to the result of the user status detection, including: Performing any one of face detection, human shape detection, or movement detection based on the real-time monitoring video stream; If face detection is performed based on the real-time monitoring video stream, when the number of times of no face detected within the first preset number of frames exceeds the second preset threshold, it is determined that the user gets out of bed; If human shape detection is performed based on the real-time monitoring video stream, when the number of times of no human contour detected within the first preset number of frames exceeds the second preset threshold, it is determined that the user gets out of bed; If movement detection is performed based on the real-time monitoring video stream, when the proportion of the motion vectors in the edge area of the bed detected within the second preset number of frames that meet the trajectory persistence condition exceeds the preset proportion, it is determined that the user gets out of bed; Wherein, the trajectory persistence condition is that the included angle between the motion direction and the normal line of the bed edge is less than the preset angle threshold, and the motion speed is greater than the preset speed threshold.

8. A user out-of-bed detection device, characterized in that, Including: A video acquisition module, configured to acquire the real-time monitoring video stream of the target area when receiving the user getting out of bed detection instruction, and the target area is the area where the user's bed is located; A motion detection module, configured to obtain an initial background template based on the first frame of the real-time monitoring video stream, and update the background template and calculate the motion saliency index frame by frame for subsequent frames; A getting out of bed determination module, configured to perform user status detection based on the real-time monitoring video stream when the motion saliency index is greater than the first preset threshold, and determine whether the user gets out of bed according to the result of the user status detection.

9. A terminal, characterized in that, The terminal includes: a memory, a processor, and a user getting out of bed detection program stored on the memory and executable on the processor. When the user getting out of bed detection program is executed by the processor, the steps of the user getting out of bed detection method according to any one of claims 1-7 are implemented.

10. A computer-readable storage medium, characterized in that, A user getting out of bed detection program is stored on the computer-readable storage medium. When the user getting out of bed detection program is executed by the processor, the steps of the user getting out of bed detection method according to any one of claims 1-7 are implemented.

Images