Fall detection method and system based on machine vision

Abstract

The application discloses a fall detection method and system based on machine vision, and relates to the technical field of machine vision monitoring. The method comprises the following steps: after continuously detecting the presence of a human body, video images and partitioned environment monitoring data are collected; a human body target is detected, tracked and posture key points are extracted, the human body target is projected to a monitoring plane and a corresponding monitoring sub-region is determined; according to key point confidence and a projection region overlap ratio, a center of mass and human body main shaft calculation frame and a lower limb position calculation frame are respectively screened, and the remaining image frames are used for maintaining tracking and region determination; a center of mass vertical velocity is calculated according to a time difference in original collection of adjacent center of mass calculation frames, and rapid falling, ground impact and falling keeping are confirmed in sequence; when the number of confirmed frames and the actual keeping time length are met, and no recovery action is detected in a recovery window, alarm information and corresponding environment monitoring data are output. The application can reduce fall misjudgment caused by posture key point failure and image frame interval change.

Description

Technical Field

[0001] This invention relates to the field of machine vision monitoring technology, and in particular to a fall detection method and system based on machine vision. Background Technology

[0002] Fall detection is widely used in elderly care facilities, home monitoring, and safety monitoring in public places. Current fall detection methods mainly include wearable detection and video image detection. Wearable detection requires the monitored person to continuously wear the sensor device, which is easily affected by wearing conditions and usage habits. Video image detection can determine whether a person has fallen based on the person's external frame, postural key points, and changes in body position in continuous images. It does not require the monitored person to wear additional devices and is suitable for continuous operation within a fixed monitoring area.

[0003] Postural keypoints in video images are easily affected by human occlusion, motion blur, shooting angle, and overlapping of multiple people. For the same human target, some keypoints may show reduced confidence or temporary absence in consecutive images. If the corresponding image frame is still used to calculate the human center of mass position, principal axis tilt angle, and lower limb position, deviations in center of mass displacement, movement velocity, and ground contact distance are likely to occur. Deleting the entire image directly may interrupt cross-frame tracking of the human target, causing jumps in the identity of the same human target and its monitoring area. After keypoint filtering, the image frames actually used for human motion calculation may no longer be continuous. If human movement velocity is still calculated at fixed frame intervals or the duration of the fall is determined solely based on the number of image frames, discrepancies between the calculated velocity value and the actual duration are likely to occur.

[0004] In multi-person monitoring scenarios, it is also necessary to map human targets to monitoring sub-regions within the actual monitoring area. When a human is at a region boundary, partially occluded, or briefly undetected, the region assignment of the human target can easily fluctuate, leading to fall detection results that correspond to environmental monitoring data in incorrect areas. Therefore, while maintaining the continuity of human target tracking and region assignment, it is necessary to distinguish the uses of different image frames in target tracking and human motion calculation, and determine the human movement speed and fall duration based on the actual acquisition time of the image frames involved in the calculation, in order to reduce fall misjudgments caused by short-term distortion of key points and changes in image frame intervals. Summary of the Invention

[0005] The purpose of this invention is to provide a fall detection method and system based on machine vision, in order to solve the following problems: when the posture key points fail temporarily due to occlusion, motion blur or human overlap, directly using the corresponding image frames can easily cause errors in the calculation of the human target centroid position, human principal axis tilt angle and lower limb position; deleting the corresponding image frames can easily interrupt human target tracking and region determination; and when the interval of the filtered image frames changes, using a fixed frame interval to calculate the human movement speed and fall duration can easily lead to misjudgment.

[0006] In a first aspect, the present invention provides a fall detection method based on machine vision, comprising:

[0007] After a human body is continuously detected for a preset dwell time, video acquisition is triggered and environmental monitoring data of the zone is acquired simultaneously.

[0008] Human targets in video images are detected and tracked across frames. The pose key points of the human targets are extracted, the human targets are projected onto the monitoring plane, and the monitoring sub-region to which the human targets belong is determined based on the current projection position, the monitoring sub-region to which the human targets belong in the previous image frame and the tracking trajectory.

[0009] Based on the confidence levels of the pose key points required for the centroid, human body principal axis, and lower limb position, as well as the overlap ratio between the human target projection area and its corresponding monitoring sub-region, it is determined whether each image frame is used for centroid and human body principal axis calculation or lower limb position calculation; image frames that do not meet the corresponding conditions are only used to update the tracking position of the human target and determine its corresponding monitoring sub-region, and are not used to update the corresponding calculation results, nor are they included in the corresponding confirmation frame count.

[0010] Based on the vertical coordinates of the centroid in two adjacent image frames involved in the calculation of the human centroid and the original acquisition time difference, the vertical displacement and vertical velocity of the human centroid are calculated.

[0011] Based on the vertical displacement of the human body's center of mass, the vertical velocity of the human body's center of mass, the tilt angle of the human body's main axis, the shape of the human target's outer frame, and the position of key points of the lower limbs, it is confirmed in sequence whether the human target experiences rapid falling, ground impact, and ground holding.

[0012] When the number of image frames that meet the fall retention conditions reaches a preset number, the actual time difference between the first and last image frames that meet the fall retention conditions reaches a preset retention time, and the recovery amount of the human target bounding box height, the rotation amount of the human body's main axis in the standing direction, and the upward reverse displacement amount of the human body's center of mass in the recovery window do not reach the corresponding recovery threshold, the fall detection result is determined to be valid.

[0013] Based on the original acquisition time of the monitoring sub-area and corresponding image frame of the human target at the time of ground impact, environmental monitoring data of the corresponding monitoring sub-area and time range are extracted, and alarm information is output.

[0014] Furthermore, a calibration mapping relationship is established based on the planar coordinates of the calibration points in the monitoring plane and their image coordinates in the video image. The lowest point among the bottom edge midpoint of the human target bounding box, the midpoint of the double ankle key points, or the visible lower limb key points is determined as the lower projection point. The two endpoints of the lower projection point and the bottom boundary of the human target bounding box are converted into monitoring plane coordinates. The horizontal and vertical dimensions are determined by the distance between the two converted endpoints and the preset projection depth, respectively, using the lower projection point as the center, to construct a rectangular human target projection area.

[0015] Furthermore, when the projected area of ​​a human target overlaps with multiple monitoring sub-regions simultaneously, if the monitoring sub-region to which the human target belongs in the previous image frame still overlaps with the current projected area of ​​the human target and the tracking trajectory is continuous, then the monitoring sub-region to which the human target belongs remains unchanged; otherwise, the monitoring sub-region to which the human target belongs is determined according to the overlap ratio between the projected area of ​​the human target and each monitoring sub-region. When a short-term detection interruption occurs, the position of the human target is predicted based on the position and direction of movement of the human target before the detection interruption, and the corresponding tracking trajectory is maintained or terminated based on the predicted position.

[0016] Furthermore, when the confidence levels of the shoulder and hip key points reach their corresponding thresholds, and the overlap ratio between the projected area of ​​the human target and its corresponding monitoring sub-region reaches a preset overlap threshold, the current image frame is used to calculate the human target's centroid position, principal axis tilt angle, and bounding box shape. When the confidence levels of the lower limb key points reach their corresponding thresholds, and the overlap ratio reaches a preset overlap threshold, the current image frame is used to calculate the distance between the lower limb key points and the ground reference line. Image frames that do not meet the centroid and principal axis calculation conditions are not used to update the human target's centroid position, principal axis tilt angle, and corresponding confirmation frame number; image frames that do not meet the lower limb position calculation conditions are not used to update the distance between the lower limb key points and the ground reference line and corresponding confirmation frame number; the aforementioned image frames are still used to update the human target's tracking position and its corresponding monitoring sub-region.

[0017] Furthermore, the actual acquisition time difference is determined based on the original acquisition timestamps of two adjacent image frames involved in the calculation of the human body's center of mass, and the vertical velocity of the human body's center of mass is calculated using the vertical displacement of the human body's center of mass and the actual acquisition time difference; when the actual acquisition time difference exceeds the preset maximum interval, the vertical velocity of the human body's center of mass is not calculated using the corresponding image frame, and the rapid descent is restarted for confirmation.

[0018] Furthermore, after confirming that the human target has fallen rapidly, the impact on the ground is confirmed based on the distance between the lower limb key points and the ground reference line, the decay of the vertical velocity of the human center of gravity, and the position of the human target's outer frame relative to the ground reference line; the fall retention is confirmed based on the human target's center of gravity displacement amplitude, the human principal axis tilt angle fluctuation amplitude, the average displacement amplitude of the posture key points, the number of confirmation frames, and the actual retention time; and the validity of the fall detection result is determined based on the height recovery of the human target's outer frame, the rotation of the human principal axis in the standing direction, and the upward reverse displacement of the human center of gravity.

[0019] Furthermore, the environmental monitoring data for each monitoring sub-area includes a monitoring sub-area identifier and a sampling timestamp. The monitoring sub-area where the projection point of the lower part of the human target is located at the time of the impact is taken as the fall location, and the original acquisition time of the corresponding image frame is taken as the fall detection time. Data corresponding to the monitoring sub-area identifier and the fall location, and whose sampling time is within a preset time range before and after the fall detection time, are extracted from the environmental monitoring data. The fall location, fall detection time, monitoring image, and extracted environmental monitoring data are written into the alarm information.

[0020] Secondly, the present invention provides a fall detection system based on machine vision, including a data acquisition and triggering module, a human target processing module, an image frame filtering module, a motion calculation module, a fall detection module, an environmental data extraction module, and an alarm output module.

[0021] The acquisition trigger module is used to trigger video acquisition and obtain zoned environmental monitoring data; the human target processing module is used to detect and track human targets, extract posture key points, project the human target onto the monitoring plane, determine the monitoring sub-region to which the human target belongs, and output the key point confidence and the overlap ratio between the human target projection area and the monitoring sub-region; the image frame filtering module is used to determine the centroid and human body principal axis calculation frames and lower limb position calculation frames respectively based on the key point confidence and overlap ratio; the human target processing module is also used to update the tracking position of the human target and the monitoring sub-region to which it belongs using image frames that do not meet the corresponding calculation conditions; the motion calculation module is used for... The vertical displacement and velocity of the centroid are calculated based on the vertical coordinates of the centroid in adjacent centroid calculation frames and the original acquisition time difference. The fall detection module is used to sequentially confirm rapid fall, ground impact, and fall retention, and determines the fall detection result based on the number of confirmed frames, the actual retention time, the height recovery of the human target's bounding box within the recovery window, the rotation of the human body's main axis in the standing direction, and the upward reverse displacement of the human centroid. The environmental data extraction module is used to extract environmental monitoring data based on the monitoring sub-region to which the human target belongs and the original acquisition time of the corresponding image frame when the ground impact is confirmed. The alarm output module is used to output the fall detection result and the corresponding environmental monitoring data.

[0022] Furthermore, the system includes a first processing unit and a second processing unit. The first processing unit is used to receive human presence detection signals and zoned environmental monitoring data, and generates a video acquisition trigger signal when the human body remains present for a preset dwell time. The second processing unit is used to respond to the video acquisition trigger signal, perform human target detection, posture key point extraction, cross-frame tracking, monitoring sub-region determination, image frame filtering, calculation of centroid vertical displacement and centroid vertical velocity, and fall detection. Based on the monitoring sub-region to which the human target belongs at the time of ground impact and the original acquisition time, the system obtains the corresponding environmental monitoring data from the first processing unit and outputs alarm information.

[0023] Compared with the prior art, the present invention has the following beneficial effects:

[0024] First, this invention uses image frames whose attitude key points do not meet the corresponding confidence requirements for human target tracking and monitoring sub-region determination, instead of using them to update the human target centroid position, human principal axis tilt angle, human target bounding box shape, or the distance between lower limb key points and ground reference lines. This ensures that human target tracking and region attribution remain continuous, while reducing the impact of low-reliability attitude key points on the above calculation results.

[0025] Secondly, this invention calculates the vertical velocity of the human body's center of gravity based on the actual acquisition time difference of the image frames involved in the calculation of the human body's center of gravity, and restarts the rapid descent for confirmation when the actual acquisition time difference exceeds the preset maximum interval; at the same time, it uses the number of confirmation frames and the actual holding time to confirm the fall holding, which can reduce the velocity calculation deviation and misjudgment of the fall duration caused by the change in frame interval after image frame screening.

[0026] Third, this invention continuously determines the fall detection results based on rapid descent, ground impact, fall retention, and recovery changes, and extracts environmental monitoring data by confirming the monitoring sub-area to which the human target belongs at the time of ground impact and the original acquisition time. This can reduce false alarms caused by ordinary changes in human posture, and ensure that the fall detection results correspond to environmental monitoring data in the wrong area or at the wrong time. Attached Figure Description

[0027] Figure 1 This is a flowchart of the fall detection method based on machine vision according to the present invention;

[0028] Figure 2 This is a schematic diagram illustrating the projection of a human target onto a monitoring plane and the determination of its corresponding monitoring sub-region according to the present invention.

[0029] Figure 3 This is a schematic diagram of the fall detection system based on machine vision according to the present invention. Detailed Implementation

[0030] The specific embodiments of the present invention will now be described with reference to the accompanying drawings. The following examples are used to illustrate the technical solutions of the present invention and are not intended to limit the scope of protection of the present invention. Where there is no contradiction, the technical features in the various embodiments can be combined with each other.

[0031] Example 1

[0032] Reference Figure 1 and Figure 2 This embodiment provides a fall detection method based on machine vision, including the following steps.

[0033] S1, triggers video capture and acquires environmental monitoring data.

[0034] Human presence detection is performed in the monitored area. When a human presence is continuously detected for a preset dwell time, the video acquisition device is triggered to acquire continuous video images and record the original acquisition timestamp of each image frame. The preset dwell time is determined based on the detection cycle of the human presence detection device and the monitoring scenario.

[0035] After video capture is triggered, environmental monitoring data for each monitoring sub-area is acquired synchronously. The environmental monitoring data includes at least one of illumination data, gas data, temperature data, and humidity data, and is accompanied by a monitoring sub-area identifier and a sampling timestamp.

[0036] S2, detects and tracks human targets, and determines the monitoring sub-area to which the human target belongs.

[0037] Reference Figure 2 Human target detection is performed on video images to obtain the bounding box of the human target; pose key points are extracted from the image region where the human target is located to obtain the position and corresponding confidence of the shoulder key point, hip key point and lower limb key point; cross-frame tracking of human targets in continuous images is performed so that the same human target has the same target label in continuous images.

[0038] After the video acquisition device is installed, multiple calibration points are selected in the monitoring plane, and the planar coordinates of each calibration point and its image coordinates in the video image are obtained. Based on the two types of coordinates, a calibration mapping relationship between the image coordinates and the monitoring plane coordinates is established.

[0039] For a detected human target, the midpoint of the two ankle key points is determined as the lower projection point of the human target; when the two ankle key points cannot be used simultaneously, the lowest point among the visible lower limb key points is determined as the lower projection point; when none of the lower limb key points can be used, the midpoint of the bottom edge of the human target's bounding box is determined as the lower projection point.

[0040] The lower projection point and the two endpoints of the bottom boundary of the human target's bounding box are converted into monitoring plane coordinates using a calibration mapping relationship. The distance between the two endpoints after conversion is calculated and used as the horizontal dimension of the human target's projection area. A rectangular human target projection area is constructed with the monitoring plane coordinates of the lower projection point as the center and a preset projection depth as the vertical dimension.

[0041] The preset projection depth is pre-calibrated based on the installation height of the video acquisition device, the shooting angle, and the footprint of a person standing in the monitoring area.

[0042] Human target projection area and the first Overlap ratio of monitoring sub-regions Determined according to the following formula:

[0043]

[0044] in, The projection area of ​​the human body target. For the first One monitoring sub-area, For the projection area of ​​the human target and the first The overlapping area of ​​the monitoring sub-regions The area of ​​the human target projection region.

[0045] When the lower projection point of a human target is located within a monitoring sub-region, and the overlap ratio between the human target projection region and the monitoring sub-region reaches a preset overlap threshold, the human target is determined to belong to the monitoring sub-region.

[0046] When the projected area of ​​a human target overlaps with multiple monitoring sub-regions simultaneously, the monitoring sub-region to which the same human target belongs in the previous image frame is read. If the monitoring sub-region to which the human target belongs in the previous image frame still overlaps with the current projected area of ​​the human target, and the tracking trajectory of the human target is continuous, the monitoring sub-region to which the human target belongs remains unchanged; otherwise, the human target is determined to belong to the monitoring sub-region with the largest overlap ratio. When the overlap ratio of multiple monitoring sub-regions is the same, the human target is determined to belong to the monitoring sub-region with the smallest distance between the region center and the lower projection point.

[0047] When a human target is briefly detected, the predicted position is determined based on the target's position and direction of movement before the interruption. If the predicted position remains within the monitoring sub-region before the interruption, that sub-region remains unchanged for a preset interruption duration. After the human target is detected again, the distance between the re-detected target position and the predicted position is calculated. If the distance does not exceed a preset connection distance, the re-detected target is continued onto the original tracking trajectory. If the interruption exceeds a preset interruption duration, the original tracking trajectory is terminated, and the vertical displacement and velocity of the centroid are not calculated using image frames from before and after the interruption.

[0048] When multiple human targets exist in the same monitoring sub-area, the target identifier, tracking trajectory and posture key points of each human target are saved separately, and subsequent fall detection is performed separately.

[0049] S3, determine the processing purpose of the image frame based on the key point confidence level and the region overlap ratio.

[0050] For each human target, the confidence levels of the shoulder key points, hip key points, and lower limb key points are obtained, and the overlap ratio between the human target projection area and its corresponding monitoring sub-region is obtained.

[0051] When the confidence levels of the shoulder key points and hip key points reach the corresponding thresholds, and the overlap ratio reaches the preset overlap threshold, the centroid position of the human target, the tilt angle of the human principal axis, and the shape of the human target bounding box are calculated using the current image frame.

[0052] The target center of mass of the human body can be determined based on the midpoint of the shoulder and the midpoint of the hip. Let the midpoint of the key points on the left and right shoulders be... The midpoint of the key points on the left and right hips is Then the first The centroid position of the human target in each image frame involved in the calculation for:

[0053]

[0054] The direction from the midpoint of the shoulder to the midpoint of the hip is determined as the principal axis of the human body. The angle between the principal axis and the perpendicular direction of the image is calculated to obtain the tilt angle of the principal axis. The aspect ratio of the bounding box of the human target is calculated based on its width and height.

[0055] When the confidence level of the lower limb key points reaches the corresponding threshold, and the overlap ratio between the human target projection area and its corresponding monitoring sub-region reaches a preset overlap threshold, the distance between the lower limb key points and the ground reference line is calculated using the current image frame. The ground reference line is pre-calibrated based on the installation location of the video acquisition device and the ground position of the monitoring area.

[0056] Image frames that do not meet the calculation conditions for the human target centroid and human principal axis are not used to update the human target centroid position, human principal axis tilt angle, human target bounding box shape, and corresponding confirmation frame number.

[0057] Image frames that do not meet the conditions for calculating lower limb position are not used to update the distance between lower limb key points and the ground reference line, or the corresponding number of confirmation frames.

[0058] Image frames that do not meet the corresponding calculation conditions are still used to update the bounding box position of the human target, the cross-frame tracking trajectory, and the monitoring sub-region to which they belong.

[0059] S4, calculate the vertical velocity of the human body's center of mass based on the original acquisition time of the image frame.

[0060] The vertical coordinates of the human body's center of mass are derived from the video image coordinates, with the top-left corner of the video image as the origin and the vertically downward direction as the positive direction. The units for the vertical coordinates and displacement of the center of mass are pixels, and the unit for the vertical velocity is pixels per second. A vertical displacement greater than zero indicates that the human body's center of mass is moving downwards. Following the original acquisition time sequence, two adjacent image frames used for calculating the human target's center of mass are read. Let the vertical coordinates of the human body's center of mass in the previous image frame be... The original collection timestamp is The vertical coordinates of the human centroid in the current image frame are: The original collection timestamp is .

[0061] The vertical displacement of the human body's center of mass is:

[0062]

[0063] The actual data collection time difference is:

[0064]

[0065] The vertical velocity of the human body's center of mass is:

[0066]

[0067] When the actual acquisition time difference does not exceed the preset maximum interval, the centroid vertical velocity is used for rapid fall confirmation; when the actual acquisition time difference exceeds the preset maximum interval, the two image frames are not used to calculate the centroid vertical velocity together, the current image frame is used as the new calculation starting point, and the number of confirmation frames accumulated for this rapid fall confirmation is cleared.

[0068] Video capture frame rate The number of consecutive image frames that do not participate in the centroid calculation is allowed to be At that time, the maximum preset interval for:

[0069]

[0070] S5, sequentially confirm rapid descent, ground impact, and fall retention.

[0071] When the vertical displacement of the human body's center of mass is not less than the preset rapid fall displacement threshold, the vertical velocity of the human body's center of mass is not less than the preset rapid fall velocity threshold, and the tilt angle of the human body's main axis increases in the direction of the human body's lateral tilt or the aspect ratio of the human body's outer bounding box increases in the direction of the human body's lateral expansion, the current image frame is determined as the rapid fall confirmation frame.

[0072] Within the preset rapid fall confirmation time limit, when the number of rapid fall confirmation frames reaches a preset number, the rapid fall of the human target is confirmed. If the original acquisition time difference between adjacent image frames participating in the calculation exceeds a preset maximum interval, or if the preset number of confirmation frames is not reached within the rapid fall confirmation time limit, the rapid fall confirmation is cancelled.

[0073] After confirming a rapid descent of the human target, image frames within the impact time window that meet the lower limb position calculation conditions are identified as ground impact candidate frames. Based on the original acquisition timestamp, the image frame whose original acquisition time is closest to that of the ground impact candidate frame is selected from the image frames that meet the conditions for calculating the center of mass and the human body's principal axis as the associated image frame. When the original acquisition time difference between the ground impact candidate frame and the associated image frame does not exceed a preset association time difference, the lower limb key point positions in the ground impact candidate frame are used in conjunction with the human body's vertical velocity of the center of mass, the human target's bounding box position, and the aspect ratio in the associated image frame for the same ground impact judgment. If no associated image frame that meets the preset association time difference is found, the ground impact candidate frame is not used to confirm the ground impact.

[0074] When the shortest distance between the lower limb key points in the ground impact candidate frame and the ground reference line is not greater than a preset ground proximity threshold, the vertical velocity of the human body's center of mass corresponding to the associated image frame decays from not less than a preset rapid fall velocity threshold to an absolute value not greater than a preset low velocity threshold within the impact time window, the shortest distance between the bottom boundary of the human target's bounding box and the ground reference line is not greater than a preset bounding box proximity threshold, and the aspect ratio of the human target's bounding box is not less than a preset lateral expansion threshold, a ground impact is confirmed for the human target. Record the original acquisition time of the ground impact candidate frame and the monitoring sub-region to which the human target belongs.

[0075] After confirming the impact, image frames that meet the calculation conditions for the center of mass and the human body's principal axis are read within a preset static statistics window, and the first image frame in the static statistics window that meets the calculation conditions is used as the reference frame. The maximum distance between the human body's center of mass position in each subsequent image frame and the human body's center of mass position in the reference frame is determined as the human body's center of mass displacement amplitude; the difference between the maximum and minimum values ​​of the human body's principal axis tilt angle within the static statistics window is determined as the human body's principal axis tilt angle fluctuation amplitude; the displacement distance of the same-named posture keypoints in adjacent valid image frames that all reach the corresponding confidence threshold is calculated, and the average value of each displacement distance is determined as the average displacement amplitude of the posture keypoints. When at least two of the above three items do not exceed the corresponding static threshold, the last image frame in the static statistics window is determined as the fall retention confirmation frame.

[0076] When the number of fall hold confirmation frames reaches a preset number, the actual time difference between the first and last fall hold confirmation frames reaches a preset hold duration, and the actual time difference does not exceed a preset confirmation window, the human target is confirmed to be in fall hold.

[0077] After confirming that the human target is in a downed position, record the height of the human target's bounding box, the tilt angle of the human's principal axis, and the vertical coordinate of the human's center of mass at the time of confirmation as the recovery reference height. , Restore reference tilt angle and restore the reference vertical coordinate For the first [item] in the recovery window For an image frame that satisfies the conditions for calculating the centroid and the human body's principal axis, its human target bounding box height, human body principal axis tilt angle, and human centroid vertical coordinates are denoted as follows: , and The height recovery of the human target bounding box is: The rotation of the human body's main axis in the standing direction is The upward displacement of the human body's center of mass is The three changes are compared with their corresponding recovery thresholds.

[0078] If at least one of the above three changes reaches the corresponding recovery threshold within a preset number of image frames, it is determined that the human target has performed a recovery action, and the fall detection result is invalid; if no recovery action is detected at the end of the recovery window, and the human target continues to meet the fall retention condition, the fall detection result is determined to be valid.

[0079] The key point confidence threshold, overlap threshold, preset association time difference, fast fall displacement threshold, fast fall speed threshold, ground proximity threshold, outer frame proximity threshold, low speed threshold, lateral expansion threshold, stationary threshold, recovery threshold, stationary statistics window, and various time parameters are calibrated based on images of standing, walking, sitting, bending, squatting, lying down, and falling under the same camera installation position, image resolution, and acquisition frame rate.

[0080] S6, extract environmental monitoring data and output alarm information.

[0081] The monitoring sub-area where the projection point of the lower part of the human target is located at the time of the ground impact is taken as the fall location, and the original acquisition time of the image frame used to confirm the ground impact is taken as the fall detection time.

[0082] Once the fall detection result is confirmed, data corresponding to the fall location and sampled at a time within a preset time range before and after the fall detection time are extracted from the environmental monitoring data.

[0083] The alarm information includes human target identification, fall location, fall detection time, monitoring images confirming the impact on the ground, monitoring images confirming the fall and retention, and extracted environmental monitoring data. The alarm information is then sent to the monitoring terminal or operation and maintenance management platform.

[0084] Example 2

[0085] Reference Figure 3 This embodiment provides a fall detection system based on machine vision, including a data acquisition and triggering module, a human target processing module, an image frame filtering module, a motion calculation module, a fall detection module, an environmental data extraction module, and an alarm output module.

[0086] The acquisition trigger module is used to trigger video acquisition and obtain zoned environmental monitoring data when a human body remains present for a preset dwell time. The human target processing module is used to detect and track human targets, extract posture key points, project the human target onto the monitoring plane, determine the monitoring sub-region to which the human target belongs, and output the confidence level of key points and the overlap ratio between the human target's projected area and its corresponding monitoring sub-region.

[0087] The image frame filtering module is used to determine the centroid and human body axis calculation frame and the lower limb position calculation frame respectively based on the confidence level and overlap ratio of the key points. The centroid and human body axis calculation frame is provided to the motion calculation module and the fall detection module, and the lower limb position calculation frame is provided to the fall detection module. The human target processing module is also used to update the tracking position and the monitoring sub-region of the human target using image frames that do not meet the corresponding calculation conditions.

[0088] The motion calculation module is used to calculate the vertical displacement and vertical velocity of the centroid based on the vertical coordinates of the centroid in two adjacent image frames involved in the centroid calculation and the original acquisition time difference; when the actual acquisition time difference exceeds the preset maximum interval, the rapid descent is restarted for confirmation.

[0089] The fall detection module sequentially confirms rapid descent, ground impact, and fall retention, determining the fall detection result based on the number of confirmed frames, the actual retention time, the height recovery of the human target's bounding box within the recovery window, the rotation of the human body's main axis in the standing direction, and the upward displacement of the human body's center of mass. The environmental data extraction module extracts environmental monitoring data corresponding to the fall location and fall detection time. The alarm output module outputs alarm information.

[0090] The system includes a first processing unit and a second processing unit. The first processing unit is connected to a human presence detection device and an environmental monitoring sensor. It generates a video acquisition trigger signal and sends it to the second processing unit after a human body has been continuously present for a preset dwell time. It also stores environmental monitoring data according to the monitoring sub-region identifier and sampling timestamp. The second processing unit is connected to a video acquisition device. In response to the video acquisition trigger signal, it controls the video acquisition device to acquire continuous video images and performs human target detection, posture key point extraction, cross-frame tracking, monitoring sub-region determination, image frame filtering, calculation of centroid vertical displacement and centroid vertical velocity, and fall detection.

[0091] After the second processing unit confirms the fall detection result, it sends the monitoring sub-area and original acquisition time at the time of the ground impact to the first processing unit. The first processing unit reads the corresponding environmental monitoring data and sends it to the second processing unit, which then writes alarm information and outputs it.

Claims

1. A fall detection method based on machine vision, characterized in that, include: After a human body is continuously detected for a preset dwell time, video acquisition is triggered and environmental monitoring data of the zone is acquired simultaneously. Detect and track human targets and extract posture key points, project the human target onto the monitoring plane, and determine the monitoring sub-region to which the human target belongs based on the current projection position, the monitoring sub-region to which the previous frame belongs, and the tracking trajectory. Based on the confidence level of key points required for the centroid, human body principal axis, and lower limb positions, and the overlap ratio of the projected regions, determine whether each image frame is used for centroid and human body principal axis calculation or lower limb position calculation; image frames that do not meet the corresponding conditions are only used for tracking and region determination. The vertical velocity of the centroid is calculated based on the original acquisition time difference between two adjacent image frames involved in the centroid calculation. The rapid fall, ground impact, and fall retention are then confirmed sequentially by combining the tilt angle of the human body's main axis, the shape of the outer frame, and the position of key points of the lower limbs. When the number of fall-holding confirmation frames and the actual time difference between the first and last confirmation frames reach the preset number and preset holding time, respectively, and the recovery amount of the inner and outer frame height of the recovery window, the spindle rotation amount, and the centroid reverse displacement amount do not reach the corresponding recovery threshold, the fall detection result is determined to be valid and the corresponding environmental monitoring data and alarm information are output.

2. The fall detection method based on machine vision according to claim 1, characterized in that, Projecting a human target onto a monitoring plane includes: Establish a calibration mapping relationship based on the planar coordinates of the calibration points in the monitoring plane and their image coordinates in the video image; The lowest point among the bottom edge midpoint of the human target bounding box, the midpoint of the two ankle key points, or the lowest point of the visible lower limb key points is determined as the lower projection point; The calibration mapping relationship is used to convert the two endpoints of the lower projection point and the bottom boundary of the human target's outer frame into monitoring plane coordinates; Using the lower projection point as the center, the distance between the two transformed endpoints as the horizontal dimension, and the preset projection depth as the vertical dimension, a rectangular human target projection area is constructed, and the overlap ratio between the projection area and each monitoring sub-area is calculated.

3. The fall detection method based on machine vision according to claim 2, characterized in that, Determine the monitoring sub-region to which the human target belongs, including: When the lower projection point is located within a monitoring sub-region, and the overlap ratio between the human target projection area and the monitoring sub-region reaches a preset overlap threshold, the human target is determined to belong to the monitoring sub-region. When the projection area of ​​a human target overlaps with multiple monitoring sub-regions simultaneously, if the monitoring sub-region to which the human target belongs in the previous image frame still overlaps with the projection area and the tracking trajectory is continuous, then the monitoring sub-region to which the human target belongs remains unchanged; otherwise, the human target is determined to belong to the monitoring sub-region with the largest overlap ratio.

4. The fall detection method based on machine vision according to claim 1, characterized in that, When a brief interruption occurs in the detection of a human target: The predicted location is determined based on the human target's position and direction of movement before the detection was interrupted; When the predicted location is still within the monitoring sub-region that the detection interrupted before the detection was interrupted, the monitoring sub-region to which the human target belongs remains unchanged. If the distance between the re-detected human target location and the predicted location does not exceed the preset connection distance, the re-detected human target will be continued to the original tracking trajectory; If the detection interruption exceeds the preset interruption time and no human target is detected again, the original tracking trajectory is terminated, and the image frames before and after the detection interruption are not used to jointly calculate the vertical displacement and vertical velocity of the centroid.

5. The fall detection method based on machine vision according to claim 1, characterized in that, Determine whether each image frame is used for centroid and human principal axis calculation or lower limb position calculation, including: Obtain confidence scores for shoulder keypoints, hip keypoints, and lower limb keypoints; When the confidence of the shoulder key points and hip key points reaches the corresponding threshold, and the overlap ratio between the human target projection area and the monitoring sub-area reaches the preset overlap threshold, the current image frame is used to calculate the position of the human target centroid, the tilt angle of the human principal axis and the shape of the bounding box. When the confidence level of the lower limb key points reaches the corresponding threshold and the overlap ratio reaches the preset overlap threshold, the distance between the lower limb key points and the ground reference line is calculated using the current image frame. Image frames that do not meet the corresponding conditions are only used to update the tracking position of the human target and the monitoring sub-region to which it belongs, without updating the corresponding calculation results or being included in the corresponding confirmation frame count.

6. The fall detection method based on machine vision according to claim 1, characterized in that, Calculate the vertical velocity of the center of mass and confirm rapid descent, including: Obtain the vertical coordinates of the centroid and the original acquisition timestamp in two adjacent centroid calculation frames. Based on the difference between the vertical coordinates of the centroid and the difference between the original acquisition timestamps, obtain the vertical displacement of the centroid and the actual acquisition time difference, and calculate the vertical velocity of the centroid. When the actual acquisition time difference exceeds the preset maximum interval, the vertical velocity of the centroid is not calculated using the two image frames, and the rapid descent is restarted for confirmation. When the vertical displacement of the centroid is not less than the preset rapid fall displacement threshold, the vertical velocity of the centroid is not less than the preset rapid fall velocity threshold, and the tilt angle of the human body's main axis increases in the direction of the human body's lateral tilt or the aspect ratio of the human body's outer frame increases in the direction of the human body's lateral expansion, the current image frame is determined as the rapid fall confirmation frame. When the number of rapid descent confirmation frames reaches the preset number of confirmation frames within the preset rapid descent confirmation time limit, the rapid descent of the human target is confirmed.

7. The fall detection method based on machine vision according to claim 6, characterized in that, Confirm impact and fall retention, including: After confirming rapid descent, image frames that meet the lower limb position calculation conditions within the impact time window are selected as ground contact candidate frames. Then, from the image frames that meet the calculation conditions of the center of mass and the main axis of the human body, the associated image frame that is closest to its original acquisition time and whose time difference does not exceed the preset associated time difference is selected. When the distance between the lower limb key points in the candidate ground contact frame and the ground reference line, as well as the centroid vertical velocity, the distance between the bottom boundary of the human target bounding box and the ground reference line, and the aspect ratio of the human target bounding box in the associated image frame reach the corresponding ground contact impact thresholds, it is confirmed that the human target has experienced a ground contact impact. After confirming the ground impact, if at least two of the following parameters do not exceed the corresponding static threshold: the displacement amplitude of the human target's center of mass, the fluctuation amplitude of the human target's principal axis tilt angle, and the average displacement amplitude of the attitude key points, the number of confirmation frames reaches the preset number, and the actual time difference between the first and last confirmation frames reaches the preset holding time and does not exceed the preset confirmation window, the human target is confirmed to be in a fallen holding state.

8. The fall detection method based on machine vision according to claim 1, characterized in that: Based on the confirmed height of the human target bounding box, the tilt angle of the human body axis, and the position of the human target centroid when the fall is held, the recovery amount of the human target bounding box height, the rotation amount of the human body axis in the standing direction, and the upward reverse displacement amount of the human target centroid within the recovery window are calculated respectively. When at least one of the three changes reaches the corresponding recovery threshold within a preset number of image frames, the fall detection result is determined to be invalid. When the recovery window ends, if none of the three changes have reached the corresponding recovery threshold and the fall remains valid, the fall detection result is determined to be valid. Based on the original acquisition time of the monitoring sub-area to which the human target belonged at the time of the confirmed ground impact and the corresponding image frame, the corresponding environmental monitoring data was extracted and written into the alarm information.

9. A fall detection system based on machine vision, characterized in that, include: The acquisition trigger module is used to trigger video acquisition and obtain zoned environmental monitoring data after a human body has been present for a preset dwell time. The human target processing module is used to detect and track human targets and extract posture key points, project the human target onto the monitoring plane, determine the monitoring sub-region to which it belongs, and output the confidence level of key points and the overlap ratio of the projected area. The image frame filtering module is used to determine the centroid and human body principal axis calculation frame and the lower limb position calculation frame respectively based on the key point confidence and overlap ratio; The human target processing module is also used to update the tracking position and monitoring sub-region of the human target using image frames that do not meet the corresponding calculation conditions; The motion calculation module is used to calculate the vertical displacement and vertical velocity of the centroid based on the vertical coordinates of the centroid in adjacent centroid calculation frames and the original acquisition time difference. The fall detection module is used to sequentially confirm rapid fall, ground impact, and fall retention, and determines the fall detection result based on the number of confirmed frames, the actual retention time, the recovery amount of the outer frame height within the recovery window, the spindle rotation amount, and the reverse displacement of the center of mass. The environmental data extraction module is used to extract environmental monitoring data based on the monitoring sub-area to which the human target belongs and the original acquisition time of the corresponding image frame at the time of the confirmed ground impact. The alarm output module is used to output fall detection results and corresponding environmental monitoring data.

10. The fall detection system based on machine vision according to claim 9, characterized in that, The system includes a first processing unit and a second processing unit; The first processing unit is connected to the human presence detection device and the environmental monitoring sensor. It is used to generate a video acquisition trigger signal and send it to the second processing unit after the human body has been present for a preset dwell time. It also stores the environmental monitoring data according to the monitoring sub-area identifier and sampling timestamp. The second processing unit is connected to the video acquisition device and is used to respond to the video acquisition trigger signal, control the video acquisition device to acquire continuous video images, and perform human target detection, posture key point extraction, cross-frame tracking, monitoring sub-region determination, image frame filtering, calculation of centroid vertical displacement and centroid vertical velocity, and fall detection. After confirming the fall detection result, the second processing unit sends the monitoring sub-area and original acquisition time at the time of the ground impact to the first processing unit. The first processing unit reads the corresponding environmental monitoring data and returns it to the second processing unit, which then writes the alarm information and outputs it.

Images