Abnormal behavior detection method and device based on infrared sensor

Abstract

The invention belongs to the field of anomaly detection, and particularly relates to an abnormal behavior detection method and device based on an infrared sensor, and the method comprises the steps: determining a safety region in a monitoring region, and a boundary with the risk of falling from a bed in the safety region; obtaining an infrared image of the monitoring area in real time, and extracting and screening a human body detection frame in the infrared image; key point detection is carried out on each human body detection frame to obtain human body skeleton key points, and the human body mass center, the angles between the left and right human body shanks and the vertical direction of the infrared image and the angle between the human body trunk and the vertical direction of the infrared image are calculated; and abnormal behavior detection is carried out in combination with the height-width ratio of each human body detection frame. The abnormal behavior detection coverage is complete, and the accuracy is high.

Description

Technical Field

[0001] This invention belongs to the field of anomaly detection, specifically relating to an abnormal behavior detection method and device based on an infrared sensor. Background Technology

[0002] Currently, fall and bed fall detection technology has become a key technology area for ensuring the safety of the elderly and vulnerable populations and improving health management. From the perspective of technological implementation, the current mainstream technologies can be divided into three main categories: wearable detection technology, non-wearable detection technology, and multimodal fusion detection technology. Each type of technology has its own characteristics in terms of principle, performance, and applicable scenarios, while also facing common and unique challenges such as accuracy, privacy protection, and environmental adaptability.

[0003] Existing technologies, such as the patent with publication number CN120220230A, disclose a multimodal fall detection method, device, and equipment. This method utilizes radar sensors to collect point cloud data of a target scene, combines it with image information collected by infrared sensors, generates fused three-dimensional coordinates, and generates target height change data. When the target height change meets preset fall conditions, the multimodal fused data is input into a deep learning behavior recognition module, which outputs a classification result of the fall behavior. This patent belongs to a multimodal fusion fall detection method, such as a radar + infrared multimodal fall detection algorithm. Although it offers advantages such as greater privacy protection and higher detection accuracy compared to visual solutions, it requires cross-modal alignment (e.g., feature-level fusion), resulting in high computational complexity and cost.

[0004] Existing technologies, such as the paper "Research and Application of Fall Detection Based on Pose Estimation," utilize pose estimation algorithms, such as yolov8n-pose, to obtain target detection boxes and their key points. This allows for the acquisition of multiple features, including the aspect ratio of the target human body rectangle, the velocity of the human centroid, and skeletal key points. A support vector machine is then used to classify and train the fall features, ultimately achieving fall detection. This paper represents a deep learning-based fall detection method. Because it does not integrate other auxiliary detection devices, such as radar or infrared sensors, it has lower costs. However, it suffers from several drawbacks: strong dependence on lighting conditions, making detection impossible in low-light / dark environments; reduced detection accuracy when images are overexposed in strong light / backlight, blurring the human body outline; and potential privacy violations. Summary of the Invention

[0005] The purpose of this invention is to provide an abnormal behavior detection method and device based on infrared sensors, which has comprehensive abnormal behavior detection coverage and high accuracy.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] Firstly, an abnormal behavior detection method based on an infrared sensor is provided, including:

[0008] Determine the safe zones within the monitored area, and the boundaries within those safe zones where there is a risk of falling from the bed;

[0009] Real-time acquisition of infrared images of the monitored area, extraction and filtering of human detection boxes in the infrared images;

[0010] For each human body detection frame, key point detection is performed to obtain human skeleton key points, and the human body centroid, the angles of the left and right lower legs of the human body with the infrared image and the angle of the human body torso with the infrared image are calculated.

[0011] Calculate the aspect ratio of each human body detection frame, and perform the first behavior screening based on the aspect ratio to determine whether the human behavior status is normal, no risk of falling from the bed, or risk of falling from the bed.

[0012] For each human behavior state without the risk of falling from the bed, a second behavior screening is performed based on the angle between the human's left and right lower legs and the infrared image and the coordinates of the left and right knees to determine whether the human behavior state is normal or abnormal.

[0013] For each human behavior state at risk of falling from the bed, a second behavior screening is performed based on the angle between the human torso and the infrared image perpendicular to determine whether the human behavior state is normal or lying down. A third behavior screening is then performed on the lying down human body based on the human center of mass and the boundary of the risk of falling from the bed, to determine whether the human behavior state is normal or a warning before falling from the bed.

[0014] If a human behavior state that provides a pre-fall warning exists, then continuous fall detection is triggered for the corresponding human until a fall occurs or there is no pre-fall warning human behavior state.

[0015] Preferably, determining the safe zone within the monitored area includes:

[0016] Obtain the outer border of the plane in the monitoring area used to indicate that the human target is lying down;

[0017] The center of each side of the outer border is used as the center of the corresponding boundary of the safe area, and the length of each side of the outer border is used as the length of the corresponding boundary of the safe area.

[0018] Preferably, the step of extracting and filtering human detection boxes in the infrared image includes:

[0019] Scale the infrared image to a fixed size;

[0020] Human target detection is performed on the scaled infrared image to obtain the human detection box;

[0021] Calculate the average pixel value within each human detection bounding box. If the average pixel value is greater than the average threshold, retain the human; otherwise, discard the human.

[0022] Preferably, the first behavior filtering based on aspect ratio includes:

[0023] If the aspect ratio of the human body detection frame is Within the specified range, the current human body's behavioral state is considered normal.

[0024] If the aspect ratio of the human body detection frame is Within the specified range, the current human body's behavioral state is considered to be free of the risk of falling out of bed;

[0025] If the aspect ratio of the human body detection frame is Within this range, the current human body's behavioral state is considered to be at risk of falling out of bed, and .

[0026] Preferably, the second behavior screening based on the angle between the left and right lower legs and the infrared image perpendicular to the ground, and the coordinates of the left and right knees, includes:

[0027] If the angle between the human body's left lower leg and the infrared image is perpendicular to the direction of the infrared image, or if the angle between the human body's right lower leg and the infrared image is perpendicular to the direction of the infrared image, the current human body's behavior status is normal; otherwise, the current human body's behavior status is abnormal.

[0028] Alternatively, if the coordinates of the left and right knees are in the bottom area of ​​the current human body detection box, then the current human body's behavior status is abnormal; otherwise, the current human body's behavior status is normal.

[0029] Preferably, the second behavior screening based on the angle between the human torso and the infrared image perpendicular to the direction includes:

[0030] If the angle between the human torso and the infrared image perpendicular to the second angle threshold is less than the second angle threshold, then the current human behavior status is normal; otherwise, the current human is lying down.

[0031] Preferably, the third behavioral screening of the lying-down human body based on the body's center of gravity and the boundary of the risk of falling from the bed includes:

[0032] If the body's center of mass exceeds the boundary where there is a risk of falling from the bed, the current body behavior state is judged as a pre-fall warning; otherwise, the current body behavior state is normal.

[0033] Preferably, the continuous bed fall detection includes:

[0034] Every few frames, the changes in the center of gravity of the head and neck and the changes in the center of gravity of the body are compared. If the changes in the center of gravity of the head and neck are greater than the first change threshold and the changes in the center of gravity of the body are greater than the second change threshold, it indicates that a fall from the bed has occurred; otherwise, no fall from the bed has occurred.

[0035] Several alternative methods are provided below, but they are not intended as additional limitations on the overall solution above. They are merely further additions or optimizations. Provided there are no technical or logical contradictions, each alternative method can be combined individually with respect to the overall solution above, or multiple alternative methods can be combined with each other.

[0036] Secondly, an abnormal behavior detection device based on an infrared sensor is provided, including a processor and a memory storing a plurality of computer instructions, wherein the computer instructions, when executed by the processor, implement the steps of the abnormal behavior detection method based on the infrared sensor.

[0037] This invention provides an abnormal behavior detection method and device based on an infrared sensor. Utilizing the characteristic that the human body's high temperature makes it appear brighter in an infrared sensor, the method filters targets by using the average value of all pixels within the detection box during target detection, thus reducing false detections to some extent. After estimating the pose of the detection box and obtaining skeletal key points, the method detects abnormal behavior based on the detection box information and key point information. The detection includes falls, bed falls, and pre-fall warnings, providing comprehensive coverage. Filtering based on different action characteristics eliminates potential false detections such as "picking up things / tying shoelaces," resulting in high detection accuracy. Attached Figure Description

[0038] Figure 1 This is a flowchart of an abnormal behavior detection method based on an infrared sensor according to the present invention;

[0039] Figure 2 This is a schematic diagram of an infrared image of the present invention without a human body.

[0040] Figure 3 This is a schematic diagram of the security area of ​​the present invention;

[0041] Figure 4 This is a schematic diagram of the 26 key points of the human skeleton in this invention;

[0042] Figure 5 This is a schematic diagram of the bottom area of ​​the human body detection frame of the present invention. Detailed Implementation

[0043] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0044] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to limit the invention.

[0045] Example 1:

[0046] like Figure 1 As shown, this embodiment provides an abnormal behavior detection method based on infrared sensors. The abnormal behavior mentioned refers to an abnormal state such as the target object falling, kneeling, or falling out of bed. Additionally, if the room contains a nursing bed, a safe zone for the nursing bed is set. If the target object exceeds this zone and there is a risk of falling out of bed, this is also considered abnormal. The specific steps include:

[0047] Step 1: Determine the safe zones within the monitored area, and the boundaries within the safe zones where there is a risk of falling from the bed.

[0048] Typically, a flat surface, such as the surface of a nursing bed, is placed within the monitoring area to allow a person to lie down. Therefore, this embodiment determines the safety zone based on this flat surface. Specifically, the outer border of the flat surface within the monitoring area is obtained; the center of each side of the outer border is used as the center of the corresponding boundary of the safety zone; and a preset proportion of the length of each side of the outer border is used as the length of the corresponding boundary of the safety zone. If there is no flat surface within the monitoring area to allow a person to lie down, a safety zone may not be set, or a certain area within the monitoring area may be directly designated as the safety zone, and the boundary with the risk of falling from the bed may be indicated as needed.

[0049] Furthermore, the boundary of potential bed fall risk is usually related to the monitoring equipment. A boundary where a person in a lying position might fall from the bed, and where the monitoring equipment has a clear view of the area, is considered a potential bed fall risk boundary. For example, if a person could fall from either side of the nursing bed, but the monitoring equipment can only capture a view of one side, then for the infrared image captured by the monitoring equipment, it is only necessary to determine whether a fall would occur on that side. However, if a person could fall from either side of the nursing bed, and the monitoring equipment can capture a view of both sides, then for the infrared image captured by the monitoring equipment, it is necessary to determine whether a fall would occur on either side.

[0050] Step 2: Acquire infrared images of the monitored area in real time, and extract and filter human detection boxes in the infrared images.

[0051] To reduce the false positive rate, this embodiment acquires infrared images of the monitored area and performs target detection based on the infrared images, including: scaling the infrared images to a fixed size; performing human target detection on the scaled infrared images to obtain human detection boxes; calculating the average pixel value within each human detection box, and retaining the box if the average pixel value is greater than the average threshold; otherwise, discarding the box.

[0052] Scaling the infrared image to a fixed size reduces the difficulty of subsequent image processing. The human target detection algorithm will not be elaborated upon in this embodiment; for example, it could be an R-CNN series network, a YOLO series network, or an SSD series network. Because the human body has a high temperature, the detection boxes filtered based on the pixel mean are considered human detection boxes to eliminate object interference.

[0053] Step 3: Perform key point detection on the image in each human body detection frame to obtain the key points of the human skeleton, and calculate the human body centroid, the angle between the left and right lower legs and the infrared image, and the angle between the human torso and the infrared image.

[0054] The method for detecting key points in the human skeleton will not be elaborated in this embodiment. For example, it can be the AlphaPose algorithm, YOLO-Pose algorithm, Simple3D Pose algorithm, etc., and the number of detected key points can be 14, 17, 22, 25, or 26, etc. Then, the corresponding centroid or angle is calculated based on geometric formulas.

[0055] Step 4: For each human behavioral state without risk of falling from the bed, a second behavioral screening is performed based on the angles of the left and right lower legs perpendicular to the infrared image and the coordinates of the left and right knees to determine whether the human behavioral state is normal or abnormal, including:

[0056] If the aspect ratio of the human body detection frame is Within the specified range, the current human body's behavioral state is considered normal.

[0057] If the aspect ratio of the human body detection frame is Within the specified range, the current human body's behavioral state is considered to be free of the risk of falling out of bed;

[0058] If the aspect ratio of the human body detection frame is Within this range, the current human body's behavioral state is considered to be at risk of falling out of bed, and .

[0059] Step 5: For each human behavioral state without risk of falling from the bed, a second behavioral screening is performed based on the angles of the left and right lower legs perpendicular to the infrared image and the coordinates of the left and right knees to determine whether the human behavioral state is normal or abnormal, including:

[0060] If the angle between the human body's left lower leg and the infrared image is perpendicular to the direction of the infrared image, or if the angle between the human body's right lower leg and the infrared image is perpendicular to the direction of the infrared image, the current human body's behavior status is normal; otherwise, the current human body's behavior status is abnormal.

[0061] Alternatively, if the coordinates of the left and right knees are in the bottom area of ​​the current human body detection box, then the current human body's behavior status is abnormal; otherwise, the current human body's behavior status is normal.

[0062] Step 6: For each human behavior state at risk of falling out of bed, a second behavioral screening is performed based on the angle between the human torso and the infrared image perpendicular to determine if the human behavior state is normal or lying down. Then, a third behavioral screening is performed on the lying-down human body based on the body's center of mass and the boundary of the risk of falling out of bed, determining if the human behavior state is normal or a pre-fall warning, including:

[0063] If the angle between the human torso and the infrared image perpendicular to the second angle threshold is less than the second angle threshold, then the current human behavior status is normal; otherwise, the current human is lying down.

[0064] For a person lying down, if the body's center of mass exceeds the boundary that poses a risk of falling out of bed, the current human behavior state is judged as a pre-fall warning; otherwise, the current human behavior state is considered normal.

[0065] Furthermore, if a fall-before-fall warning exists, continuous fall detection is triggered for the corresponding human target until a fall occurs or no fall-before-fall warning exists. This embodiment performs continuous fall detection based on the head and neck center of mass and the body center of mass. Specifically, every few frames, the changes in the head and neck center of mass and the body center of mass are compared. If the change in the head and neck center of mass is greater than a first threshold and the change in the body center of mass is greater than a second threshold, then a fall has occurred; otherwise, no fall has occurred.

[0066] To visually demonstrate abnormal situations, this embodiment adds abnormal prompts (such as color and text alerts) to human detection boxes with abnormal behavior, and adds a pre-fall warning prompt to human detection boxes with fall-from-bed warnings. If a fall occurs, a fall-from-bed alarm is issued. Additionally, for human detection boxes without abnormal behavior, a no-abnormality prompt can be added, which can also avoid excessive prompts without actually issuing one. If no boundary indicating a risk of falling from a bed is set in step 1, a third behavior screening is unnecessary, and the lying-down human detection box can be in either an abnormal or normal state depending on the specific scenario.

[0067] Compared to fall detection schemes that combine infrared sensors with radar, the advantages of this invention are: it not only includes fall detection, but also bed fall detection and pre-bed fall warning; and it is also less expensive. Compared to fall detection schemes that only use visible light, the advantages of this invention are: it can still perform relevant detection at night, and it includes bed fall detection and pre-bed fall warning.

[0068] Example 2:

[0069] To facilitate understanding, this embodiment provides a specific solution based on embodiment 1 to illustrate in detail the present invention's method for detecting abnormal behavior based on an infrared sensor.

[0070] This invention combines infrared sensors to perform attitude estimation, and then uses the acquired detection frame and key point information to detect and judge behaviors such as falls and bed falls, and also provides pre-fall warnings, thus realizing the detection of abnormal behaviors that pose a danger.

[0071] (1) Determine the safe area.

[0072] The prerequisites for this invention are: the infrared sensor is fixed; if a nursing bed exists in the detection environment, its position relative to the infrared sensor is fixed; the space between the nursing bed and the infrared sensor is large, which may cause the patient to fall off the bed; and at the same time, the infrared sensor can capture images of the four edges of the nursing bed surface.

[0073] Determine the safe area for the nursing bed, specifically as follows:

[0074] The image pre_img is captured when no one is lying on the nursing bed. First, the image pre_img is binarized, preserving the area with the lowest brightness (the temperature is lowest when no one is on the nursing bed, corresponding to the darkest area in the infrared image, such as...). Figure 2 As shown in the image, the binarized image `img_pre_thre` is obtained. Then, line detection is performed on `img_pre_thre`, and the lines are filtered based on their pixel length and position to obtain the four edge lines corresponding to the nursing bed. Based on these four lines, the coordinates of the four intersection points are determined as the four corners of the nursing bed surface, and they are sorted from top to bottom and left to right. That is, the order of the four points is: top left... Top right Bottom left and bottom right In this embodiment, the upper left corner of the infrared image is used as the origin of the coordinate system.

[0075] Based on the four sorting points, determine the centers of the four lines (top line, bottom line, left line, right line), which are respectively ( 、( 、( 、( The calculation formulas are as follows:

[0076]

[0077]

[0078]

[0079]

[0080] Based on the location information of four points, the height of the nursing bed in the image is obtained. Hekuan The calculation formula is:

[0081]

[0082]

[0083] Then, taking the centers of the four lines as the center, the upper and lower lines of the safety zone are used as the width of the nursing bed. 80% of the length; the length of the left and right lines of the safety zone is the height of the nursing bed. 80% of the length is used to determine the safe zone, such as Figure 3 As shown by the yellow line. Figure 3 Safety line numbered 1 is located near the infrared sensor. A fall from the bed is possible on this side, therefore it is identified as a boundary with a risk of falling from the bed, and an image corresponding to this boundary is acquired. The axis value is denoted as Its value is .

[0084] (2) Extract the human body detection frame.

[0085] Read the infrared image img; scale the infrared image img to obtain the scaled image img_resize, with a corresponding size of (640, 640); use the yolov8n network to perform human target detection on the scaled image img_resize to obtain the human detection bounding box rects (containing information such as the coordinates of the top left point of the detection box, the width and height of the detection box). The specific detection process will not be described in detail.

[0086] Because the human body has a higher temperature and therefore appears brighter in infrared images compared to the environment, to reduce false positives, the directly extracted human detection bounding boxes (rects) are filtered. The method is as follows: the average pixel value of the infrared image within the detection box area is calculated. If the average value is greater than 70 (this threshold can be adjusted according to the actual situation), it is considered a human target; otherwise, the detection box is considered a false positive. The final filtered detection boxes are then obtained as rects_new. Typically, one human target is considered to have one human detection box. If multiple human detection boxes are extracted, each human detection box needs to be individually assessed for abnormal behavior.

[0087] (3) Key point detection.

[0088] This invention relates to anomaly detection based on the Alphapose human pose estimation algorithm. The Alphapose human pose estimation algorithm's processing steps are as follows: first, a YOLOv3 network is used for human detection; then, key points of the individual skeleton are detected from the image within the human detection bounding box. Specific processing details are not elaborated here. The 26 key points are distributed as follows: Figure 4 As shown.

[0089] Because the YOLOv3 network is relatively complex and has a long detection time, with the rapid iteration and updates of the YOLO series algorithms, this embodiment uses the lightweight version YOLOv8n network for relevant target detection. At the same time, in order to improve the detection accuracy for infrared image scenes, an infrared human body dataset was first collected, including people lying in bed, lying on the floor, and various standing postures. Then, the data was labeled and the model was trained to obtain the corresponding detection model.

[0090] The aspect ratio (ratio) of the human detection bounding box is obtained based on the `rects_new` information, and the human centroid is calculated based on the `kpts` of the skeletal keypoints. The left and right lower legs of the human body are perpendicular to the image. Angle of axis , The human torso is perpendicular to the image. Angle of axis The calculation method for the human body's center of mass follows these rules: head and neck 0.07, torso 0.43, upper arms 0.06, forearms 0.04, hands 0.02, thighs 0.24, lower legs 0.10, and feet 0.04. The specific calculation methods for each part's center of mass are as follows:

[0091] Head and neck center of mass: ;

[0092] Trunk center of mass: ;

[0093] Left upper arm center of mass: ;

[0094] Right upper arm center of mass: ;

[0095] Center of mass of left forearm: ;

[0096] Right forearm center of mass: ;

[0097] Left-hand center of mass: ;

[0098] Right-hand center of mass: ;

[0099] Left thigh center of gravity: ;

[0100] Right thigh center of mass: ;

[0101] Center of mass of left calf: ;

[0102] Right lower leg center of mass: ;

[0103] Left foot center of gravity: ;

[0104] Right foot center of gravity: ;

[0105] Among them, the above formula Indicates serial number Image coordinates of key points.

[0106] Then the human body's center of mass The calculation formula is:

[0107]

[0108]

[0109] The left and right lower legs of the human body are perpendicular to the image. The angle of the axis is calculated using the following formula:

[0110]

[0111]

[0112] In the formula, The angle between the left lower leg of the human body and the perpendicular direction of the infrared image. The angle between the right lower leg of the human body and the perpendicular direction of the infrared image. It is the arctangent function. To take the absolute value, Here are the coordinates of keypoint number 13 out of the 26 keypoints in the human skeleton. Here are the coordinates of keypoint number 14 out of the 26 keypoints in the human skeleton. Here are the coordinates of keypoint number 15 out of the 26 keypoints in the human skeleton. The coordinates of key point number 16 out of the 26 key points of the human skeleton.

[0113] Human torso perpendicular to the image The angle of the axis is calculated using the following formula:

[0114]

[0115] In the formula, The angle between the human torso and the infrared image perpendicular to the direction of the image. Here are the coordinates of keypoint number 18 out of the 26 keypoints in the human skeleton. The coordinates of key point number 19 out of the 26 key points of the human skeleton.

[0116] (4) Judgment of abnormal behavior:

[0117] Based on the aspect ratio of the detection box and the positions of some key points, abnormal behavior is judged, and a prompt is given in the image: if the judgment state is abnormal, an "attention" abnormal prompt is given on the detection box and the detection box is displayed in red; otherwise, a "Normal" normal prompt is given on the detection box and the detection box is displayed in green. The judgment process is as follows (the following threshold parameters are determined based on actual experiments):

[0118] like exist Within the range, if the target's state is either "walking" or "standing," both of which are considered "normal," then the corresponding human detection box in the infrared image will be displayed in green, and a "Normal" prompt will be given at the upper left corner of the box, indicating that there is no abnormal behavior.

[0119] like exist Within the range, the target's state may be "squatting to tie shoelaces / picking something / kneeling on both knees / sitting," which is classified as having no risk of falling from the bed. The "kneeling on both knees" state is considered "abnormal," while the other states are considered "normal." In the "squatting to tie shoelaces / picking something" state, the angle between the lower leg and the vertical direction of the image is small; an angle threshold of 40 degrees is set. or If the value is below this threshold, the target object is determined to be in a "normal" state, and the corresponding human detection box in the infrared image is displayed in green with a "Normal" prompt at the top left corner of the box; otherwise, the target is determined to be in an "abnormal" state, displayed in red with an "attention" prompt at the top left corner of the box, indicating abnormal behavior (abnormal behavior here does not include pre-fall warnings or falls). Additionally, a judgment is made based on whether key points numbered 14 and 13 (i.e., the coordinates of the left and right knees) are within the bottom area of ​​the current human detection box. Figure 5 As shown, in this embodiment, the bottom 1 / 8 of the human detection box is used as the bottom area. If the coordinates of both the left and right knees are in the bottom area, the target is determined to be in an "abnormal" state. The corresponding human detection box in the infrared image is displayed in red, and an "attention" prompt is given at the top left of the box. Otherwise, the target is considered to be in a "normal" state. The corresponding human detection box in the infrared image is displayed in green, and a "Normal" prompt is given at the top left of the box.

[0120] If ratio is Within the specified range, the target may be in a "lying down / standing up" state, which is classified as having a risk of falling out of bed. The "standing up" state is considered "normal," while the "lying down" state may pose a risk of falling out of bed. Approaching 90 degrees, set the threshold to 85 degrees. First, based on... The value determines the current state of the target. If the target is determined to be in a "normal" state, the corresponding human detection box in the infrared image will be displayed in green, and a "Normal" prompt will be given at the upper left corner of the box; otherwise, the target is in a "lying down" state. Next, a judgment is made based on whether the target's center of mass is within the safe area of ​​the nursing bed. If the human center of mass... If the human body's center of mass is within the boundary posing a risk of falling from the bed, the target is considered "normal," and the corresponding human detection box in the infrared image is displayed in green with a "Normal" message at the top left of the box. Otherwise, if the human body's center of mass exceeds the boundary posing a risk of falling from the bed, it is considered "abnormal," and a fall from the bed may occur. The corresponding human detection box in the infrared image is displayed in red with an "attention" message (pre-fall warning) at the top left of the box, thus triggering a pre-fall warning.

[0121] In addition, if a fall-from-bed warning occurs, the center of gravity of the human head and neck is compared every 10 frames. Changes and the human body's center of mass Changes in the head's center of gravity Changes exceeding 60 and human center of mass If the change exceeds 40 (various postures of a fall can affect the position of the body's center of gravity), the state is determined to be "fall from bed". A detection box is drawn in the infrared image, and the color is red. An "attention" prompt (fall from bed alarm) is given at the upper left corner of the box. Otherwise, the state is determined to be "normal". A detection box is drawn in the infrared image, and the color is green. A "Normal" prompt is given at the upper left corner of the box.

[0122] Example 3:

[0123] This embodiment provides an abnormal behavior detection device based on an infrared sensor, including a processor and a memory storing a number of computer instructions. When the computer instructions are executed by the processor, they implement the steps of the abnormal behavior detection method based on the infrared sensor.

[0124] For specific limitations on the abnormal behavior detection device based on infrared sensors, please refer to the limitations on the abnormal behavior detection method based on infrared sensors mentioned above, which will not be repeated here.

[0125] The memory and processor are electrically connected directly or indirectly to enable data transmission or interaction. For example, these components can be electrically connected to each other via one or more communication buses or signal lines. The memory stores a computer program that can run on the processor, which implements the method of the present invention by running the computer program stored in the memory.

[0126] The memory may be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc. The memory stores the program, and the processor executes the program upon receiving an execution instruction.

[0127] The processor may be an integrated circuit chip with data processing capabilities. The aforementioned processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor.

[0128] It should be noted that, Figures 2-4 The main displays are infrared images without human figures, a safe zone diagram, and a diagram of key points of the human skeleton. Figures 2-4 The graphic text in the text is only an element in the running interface of the software and does not involve the focus of the improvement of this application. Moreover, the clarity of the running interface is related to the pixels and the scaling ratio, so the presentation effect is relatively limited.

[0129] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0130] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the appended claims.

Claims

1. A method for detecting abnormal behavior based on an infrared sensor, characterized in that, The abnormal behavior detection method based on infrared sensors includes: Determine the safe zones within the monitored area, and the boundaries within those safe zones where there is a risk of falling from the bed; Real-time acquisition of infrared images of the monitored area, extraction and filtering of human detection boxes in the infrared images; For each human body detection frame, key point detection is performed to obtain human skeleton key points, and the human body centroid, the angles of the left and right lower legs of the human body with the infrared image and the angle of the human body torso with the infrared image are calculated. Calculate the aspect ratio of each human body detection frame, and perform the first behavior screening based on the aspect ratio to determine whether the human behavior status is normal, no risk of falling from the bed, or risk of falling from the bed. For each human behavior state without the risk of falling from the bed, a second behavior screening is performed based on the angle between the human's left and right lower legs and the infrared image and the coordinates of the left and right knees to determine whether the human behavior state is normal or abnormal. For each human behavior state at risk of falling from the bed, a second behavior screening is performed based on the angle between the human torso and the infrared image perpendicular to determine whether the human behavior state is normal or lying down. A third behavior screening is then performed on the lying down human body based on the human center of mass and the boundary of the risk of falling from the bed, to determine whether the human behavior state is normal or a warning before falling from the bed. If a human behavior state that provides a pre-fall warning exists, then continuous fall detection is triggered for the corresponding human until a fall occurs or there is no pre-fall warning human behavior state.

2. The abnormal behavior detection method based on an infrared sensor according to claim 1, characterized in that, The determination of the safe zone within the monitoring area includes: Obtain the outer border of the plane in the monitoring area used to indicate that the human target is lying down; The center of each side of the outer border is used as the center of the corresponding boundary of the safe area, and the length of each side of the outer border is used as the length of the corresponding boundary of the safe area.

3. The abnormal behavior detection method based on an infrared sensor according to claim 1, characterized in that, The extraction and filtering of human detection boxes in infrared images includes: Scale the infrared image to a fixed size; Human target detection is performed on the scaled infrared image to obtain the human detection box; Calculate the average pixel value within each human detection bounding box. If the average pixel value is greater than the average threshold, retain the human; otherwise, discard the human.

4. The abnormal behavior detection method based on an infrared sensor according to claim 1, characterized in that, The first behavior filtering based on aspect ratio includes: If the aspect ratio of the human body detection frame is Within the specified range, the current human body's behavioral state is considered normal. If the aspect ratio of the human body detection frame is Within the specified range, the current human body's behavioral state is considered to be free of the risk of falling out of bed; If the aspect ratio of the human body detection frame is Within this range, the current human body's behavioral state is considered to be at risk of falling out of bed, and .

5. The abnormal behavior detection method based on an infrared sensor according to claim 1, characterized in that, The second behavioral screening, based on the angles of the left and right lower legs to the infrared image and the coordinates of the left and right knees, includes: If the angle between the human body's left lower leg and the infrared image is perpendicular to the direction of the infrared image, or if the angle between the human body's right lower leg and the infrared image is perpendicular to the direction of the infrared image, the current human body's behavior status is normal; otherwise, the current human body's behavior status is abnormal. Alternatively, if the coordinates of the left and right knees are in the bottom area of ​​the current human body detection box, then the current human body's behavior status is abnormal; otherwise, the current human body's behavior status is normal.

6. The abnormal behavior detection method based on an infrared sensor according to claim 1, characterized in that, The second behavioral screening based on the angle between the human torso and the infrared image perpendicular to the direction includes: If the angle between the human torso and the infrared image perpendicular to the second angle threshold is less than the second angle threshold, then the current human behavior status is normal; otherwise, the current human is lying down.

7. The abnormal behavior detection method based on an infrared sensor according to claim 1, characterized in that, The third behavioral screening of a person lying down, based on the body's center of gravity and the boundary of the risk of falling from the bed, includes: If the body's center of mass exceeds the boundary where there is a risk of falling from the bed, the current body behavior state is judged as a pre-fall warning; otherwise, the current body behavior state is normal.

8. The abnormal behavior detection method based on an infrared sensor according to claim 1, characterized in that, The continuous fall detection includes: Every few frames, the changes in the center of gravity of the head and neck and the changes in the center of gravity of the body are compared. If the changes in the center of gravity of the head and neck are greater than the first change threshold and the changes in the center of gravity of the body are greater than the second change threshold, it indicates that a fall from the bed has occurred; otherwise, no fall from the bed has occurred.

9. An abnormal behavior detection device based on an infrared sensor, comprising a processor and a memory storing a plurality of computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the steps of the abnormal behavior detection method based on an infrared sensor as described in any one of claims 1 to 8.

Images