Fall behavior recognition method and system for wireless monitoring video

Abstract

This invention relates to the field of video image analysis technology, specifically to a method and system for fall behavior recognition in wireless surveillance video. The method involves acquiring multiple consecutive video frames of the subject, accurately locating target bounding boxes for body parts, extracting key points, and clarifying the positional range of each body part. Since a fall disrupts the body's force line balance, a torso vector is constructed based on the key points, and the angles and differences between the torso and the ground, as well as the body itself, are analyzed to determine a posture tilt index. The significant changes in body contour during a fall are analyzed; the posture tilt index is combined with the contour features of the target bounding boxes for body parts to determine the contour analysis emphasis. Changes in the position of key points reflect the movement trajectory, thus determining the trajectory emphasis. Finally, the trend of changes in contour analysis emphasis and trajectory emphasis is fused and analyzed to eliminate interference such as single-frame occlusion, ensuring continuous and accurate fall behavior recognition results and improving the reliability and practicality of fall detection.

Description

Technical Field

[0001] This invention relates to the field of video image analysis technology, specifically to a method and system for recognizing fall behavior in wireless surveillance videos. Background Technology

[0002] Against the backdrop of accelerating global aging, falls have become one of the leading causes of injury and illness among the elderly. Wireless monitoring technology can enable 24-hour wireless monitoring of elderly people living alone, nursing homes, and other similar settings. It can monitor the elderly's fall behavior in real time, triggering alarms such as APP push notifications and SMS notifications, shortening rescue time, and improving the level of safety and monitoring for the elderly.

[0003] Existing technologies for recognizing fall behavior in personnel using wireless surveillance video typically analyze the foreground contour of the image and detect changes in the aspect ratio of the human target box to determine whether a fall has occurred. However, in real-world scenarios, normal actions such as sitting down or bending over can cause changes in the aspect ratio of the human target box. Therefore, existing technologies that only focus on single geometric or motion features are prone to misjudgment and affect the accuracy of fall behavior recognition. Summary of the Invention

[0004] To address the technical problem that in real-world scenarios, normal actions such as sitting down and bending over can alter the aspect ratio of the human target bounding box, making it easy to misjudge and affect the accuracy of fall behavior recognition by focusing solely on a single geometric or motion feature, this invention aims to provide a fall behavior recognition method and system for wireless surveillance videos. The specific technical solution adopted is as follows: At the current moment, acquire multiple consecutive monitoring video frames of the person under test, locate the target bounding box of the body parts of the person under test in each monitoring video frame, and extract the key points of the human body. In each monitoring video frame, a body torso vector is constructed based on human key points; the angle between the body torso vector and the ground, the angle between body torso vectors, and the differences in angles in different monitoring video frames are analyzed to determine the posture tilt index for each monitoring video frame; based on the contour features of the target boxes of body parts in each monitoring video frame, and in combination with the posture tilt index, the contour analysis importance is determined. Based on the positional changes of key human body points in adjacent surveillance video frames, the intensity of the trajectory of each surveillance video frame is determined. By fusing and analyzing the contours and trajectories of surveillance video frames to identify the falling behavior of the person being tested at the current moment, the system can identify the falling behavior of the person being tested at the current moment.

[0005] Furthermore, the step of locating the target bounding box of the body parts of the person under test in each frame of the surveillance video and extracting key points of the human body includes: The human body target detection box of the person to be tested is obtained in each frame of the surveillance video using a target detection algorithm, and the human body target detection box is used as the target box of the body part of the person to be tested. Multiple human body key points within the target bounding box of the body parts of the person under test are obtained using a real-time pose estimation model. The human body key points include at least head key points, shoulder key points, hip key points, and foot key points.

[0006] Furthermore, the method for obtaining the body trunk vector includes: Within the target bounding box of the body parts of the person being tested, the midpoint of the line connecting two foot key points in the human body key points is taken as the starting point, and the head key point is taken as the ending point to obtain the body centerline vector. The lower body trunk vector is obtained by taking the midpoint of the line connecting the two foot key points as the starting point and the midpoint of the line connecting the two hip key points as the ending point. The upper body trunk vector is obtained by taking the midpoint of the line connecting the two hip key points as the starting point and the head key point as the ending point.

[0007] Furthermore, the method for obtaining the attitude tilt index includes: Select any frame of the surveillance video as the test frame, analyze the angle between the body centerline vector of the person being tested and the ground in the test frame and its variation characteristics in different surveillance video frames, and determine the first tilt factor corresponding to the test frame. In the frame to be tested, the angle between the upper half torso vector and the lower half torso vector of the person to be tested is taken as the second angle corresponding to the frame to be tested. The mean of the second angles of all monitoring video frames before the time sequence of the frame to be tested is calculated as the second mean feature value. The difference between the second included angle corresponding to the frame under test and the second mean feature value corresponding to the frame under test is multiplied by a preset reduction coefficient and then normalized to become the second tilt factor corresponding to the frame under test. The normalized value of the product of the first tilt factor and the second tilt factor corresponding to the frame under test is used as the attitude tilt index of the frame under test.

[0008] Furthermore, the method for obtaining the first dumping factor includes: In the frame to be tested, the angle between the straight line containing the body centerline vector of the person under test and the horizontal line of the ground is obtained as the first angle corresponding to the frame to be tested. The mean of the first angles of all monitoring video frames before the time sequence of the frame to be tested is calculated as the first mean feature value. The difference between the first mean feature value corresponding to the frame under test and the first angle between the frame under test is calculated, and the difference is multiplied by a preset reduction coefficient and normalized to obtain the first tilt factor corresponding to the frame under test.

[0009] Furthermore, the method for obtaining the importance of the contour analysis includes: In each frame of the surveillance video, the ratio of the length of the target box of the body part of the person under test along the horizontal direction to the length along the vertical direction is obtained, which is used as the aspect ratio of each frame of the surveillance video. The ratio of the aspect ratio of each monitoring video frame to the preset aspect ratio is used as the contour deformation factor. The product of the contour deformation factor and the attitude tilt index of each monitoring video frame is used as the contour analysis importance of each monitoring video frame.

[0010] Furthermore, the method for obtaining the trajectory with high intensity includes: In each monitoring video frame, the line connecting two key points on the shoulders in the target frame of the body parts of the person being tested is taken as the shoulder line, and the line connecting two key points on the hips is taken as the waist line. For any frame of surveillance video, calculate the angle between the shoulder line and the waist line in that frame, and use it as the third angle. Calculate the absolute value of the difference between the third angle between the current monitoring video frame and the adjacent previous monitoring video frame, and normalize it to obtain the shoulder and waist instability performance of the monitoring video frame. For any human body key point, calculate the displacement of the human body key point in the current monitoring video frame and the human body key point in the adjacent previous monitoring video frame on the Y-axis in the vertical coordinate system. The ratio of the average positive displacement of all key points in the upper body to the average positive displacement of all key points in the lower body is calculated to obtain the upper and lower body displacement ratio coefficient. The normalized value of the average positive displacement of all human body key points is used as the descent displacement index. The shoulder and waist instability performance is multiplied by the descent displacement index and the upper and lower body displacement ratio coefficient, and the normalized value of the product is used as the trajectory presentation emphasis of the monitoring video frame.

[0011] Furthermore, the fusion analysis of the contour analysis between the surveillance video frames shows a significant trend in the change of importance and trajectory, enabling the identification of the fall behavior of the person being tested at the current moment, including: The contour analysis emphasis of each monitoring video frame is multiplied by the trajectory presentation emphasis and then normalized to obtain the fall tendency degree corresponding to each monitoring video frame. The fall trend of all monitored video frames is fitted with a straight line according to the time sequence to obtain the slope value of the fitted line. The slope value is multiplied by a preset reduction coefficient and then adjusted using the hyperbolic tangent function to obtain the adjustment factor. The sum of the adjustment factor and the preset constant is used as the adjustment index. The product of the adjustment index and the fall tendency of the monitoring video frame at the current moment is normalized and used as the fall behavior recognition index of the person under test at the current moment. If the fall behavior identification index of the person being tested is greater than or equal to the preset fall threshold at the current moment, a fall warning needs to be issued.

[0012] Furthermore, the preset fall threshold value ranges from [0.6, 1].

[0013] A fall behavior recognition system for wireless surveillance video includes a processor and a memory. The memory stores at least one instruction, at least one program, code set, or instruction set. When the processor loads and executes the at least one instruction, at least one program, code set, or instruction set, it implements the steps of the fall behavior recognition method for wireless surveillance video.

[0014] The present invention has the following beneficial effects: Firstly, acquiring multiple consecutive video frames of the person under test at the current moment captures the dynamic changes in their movements over a short period. Simultaneously, accurately locating the bounding boxes of the person's body parts and extracting key human points clarifies the positional range of each body part within the video frames, facilitating the quantification of the person's motion state and providing a more comprehensive data foundation for accurate fall identification. Since, ideally, the human body's force lines are usually relatively balanced, and a fall essentially disrupts this balance, manifesting as a rapid tilt of the body's force lines towards the ground and complex changes in these lines, a torso vector can be constructed based on the key human points. After converting the person's complex body posture into quantifiable vector information, the angle between the torso vector and the ground, the angles between torso vectors, and the differences in these angles across different video frames are analyzed to determine the person's posture tilt index in each frame. During a fall, the person's body contour changes significantly; therefore, this posture tilt index is combined with the contour features of the bounding boxes of body parts to determine the importance of contour analysis. The positional changes of key points on the human body between different frames reflect the movement trajectory of the person. By analyzing these positional changes, it can be determined whether the person is in a normal movement state or has experienced an abnormal fall. Therefore, the emphasis of the trajectory is determined based on this feature. Finally, by fusing and analyzing the trend of contour analysis emphasis and trajectory emphasis between surveillance video frames, interference such as single-frame occlusion is eliminated, ensuring the continuity and accuracy of the fall behavior recognition results. Attached Figure Description

[0015] Figure 1This is a flowchart of a method for recognizing fall behavior in wireless surveillance video, provided in one embodiment of the present invention. Figure 2 This is a system block diagram of a fall behavior recognition system for wireless surveillance video, provided as an embodiment of the present invention. Detailed Implementation

[0016] The following description, in conjunction with the accompanying drawings, details a specific scheme for a fall behavior recognition method and system for wireless surveillance video provided by the present invention.

[0017] Please see Figure 1 The diagram illustrates a method flowchart for fall behavior recognition in wireless surveillance video according to an embodiment of the present invention. The method includes the following steps: Step S1: At the current moment, acquire multiple consecutive monitoring video frames of the person under test, locate the target bounding box of the body parts of the person under test in each monitoring video frame, and extract the key points of the human body.

[0018] Depending on the needs of the monitoring scenario, appropriate wireless surveillance cameras must be selected. For example, in indoor nursing home scenarios, considering changes in lighting and spatial range, cameras with wide-angle lenses, high resolution (such as 1080P or higher), and night vision capabilities can be chosen to ensure clear capture of human activities under different lighting conditions. When monitoring the vicinity of outdoor elderly people living alone, cameras with waterproof, dustproof, and weather-resistant characteristics should be selected, while ensuring sufficient field of view and image clarity.

[0019] Then install the selected camera in a suitable location to obtain the best monitoring angle. Indoors, the camera can be installed in a corner of the room, at a height of about 2.5-3 meters, which can cover a larger area and avoid frequent obstruction by people; outdoors, the camera should be installed at a high place, such as under the eaves of the house or on a dedicated monitoring pole, to ensure that it can monitor areas where people may be active, such as doorways and courtyards.

[0020] Next, set an appropriate video frame acquisition frequency. To accurately capture the detailed changes in the movements of the person being monitored, the acquisition frequency should be no less than 15 frames per second. In scenarios requiring higher precision monitoring, the acquisition frequency can be increased to 30 frames per second or higher. The camera acquires monitoring video frames in real time according to the set acquisition frequency, thus obtaining multiple consecutive monitoring video frames of the person being monitored at the current moment.

[0021] It should be noted that the acquisition period of the monitoring video frame in this embodiment of the present invention can be set to the current time as the cutoff point and back to the historical time by 5 seconds. The specific duration of the acquisition period can be adjusted according to the implementation scenario and is not limited here. The person to be tested refers to the person who needs to be identified for fall behavior. In this embodiment of the present invention, a single person to be tested is used for explanation.

[0022] In the embodiments of the present invention, the collection and acquisition of personal information data are authorized by the relevant users, and the process does not violate relevant laws and regulations, nor does it violate public order and good morals.

[0023] The most obvious sign of a fall is a sudden change in the human body from an upright position to a lying position. Therefore, the target bounding boxes of the body parts of the person being tested can be located in each frame of the monitoring video, and key points of the human body can be extracted at the same time. This helps to quantify the dynamic changes of the person being tested's body in subsequent processes and achieve more accurate fall behavior recognition.

[0024] Preferably, in one embodiment of the present invention, the method for obtaining the target bounding box of the body part and key points of the human body includes: The YOLOv8 object detection algorithm is used to obtain the human body target bounding box of the person under test in each frame of the surveillance video, and the human body target bounding box is used as the target bounding box of the body parts of the person under test. At the same time, the OpenPose real-time pose estimation model is used to obtain multiple human key points within the target bounding box of the body parts of the person under test. In this embodiment of the invention, the human key points include at least one head key point, two shoulder key points, two hip key points, and two foot key points. Among them, the head key point and shoulder key point belong to the human key points of the upper body, and the hip key point and foot key points belong to the human key points of the lower body.

[0025] It should be noted that the YOLOv8 object detection algorithm and the OpenPose real-time pose estimation model are both well-known technologies and will not be elaborated upon here.

[0026] Step S2: In each monitoring video frame, construct a body torso vector based on human key points; analyze the angle between the body torso vector and the ground, the angle between body torso vectors, and the differences in angles in different monitoring video frames to determine the posture tilt index for each monitoring video frame; based on the contour features of the body part target boxes in each monitoring video frame, and combined with the posture tilt index, determine the importance of contour analysis.

[0027] The changes in human posture can be described by the relative positions and angles of different parts of the body. The body trunk vector can represent the main supporting structure of the human body. Therefore, in this step, the body trunk vector can be constructed based on the key points of the human body in each frame of the monitoring video for use in the subsequent analysis process.

[0028] Preferably, in one embodiment of the present invention, the method for obtaining the body torso vector includes: When the human body is standing or moving normally, the lower body (including the legs and hips) is the main supporting part, while the upper body (from the hips to the head) is the moving part that is supported by the lower body.

[0029] Therefore, in each frame of the monitoring video, within the target bounding box of the body parts of the person under test, the midpoint of the line connecting two foot key points is used as the starting point, and the midpoint of the line connecting two hip key points is used as the ending point to obtain the lower body trunk vector, which is used to reflect the posture of the lower body; the midpoint of the line connecting two hip key points is used as the starting point, and the head key point is used as the ending point to obtain the upper body trunk vector, which is used to represent the posture of the upper body; the midpoint of the line connecting two foot key points is used as the starting point, and the head key point is used as the ending point to obtain the body centerline vector, which covers the overall body center trend from the support of the lower body to the extension of the upper body.

[0030] Considering that in an ideal posture, the human body's force lines maintain relative balance, the angle between the body's center line and the ground is relatively large. A fall essentially disrupts this balance, manifesting as the body's center line rapidly tilting towards the ground, causing a sudden drop in the angle between the center line and the ground. Furthermore, in an ideal posture, the collinearity of the upper and lower torso tilt angles is high. A fall causes the body's center of gravity to shift to one side of the tilt, prompting the body to instinctively adjust its balance quickly, resulting in poor collinearity of the upper and lower torso in the short term. Therefore, in each frame of the monitoring video, we analyze the angle between the body's torso vector and the ground, the angles between body's torso vectors, and the differences in these angles across different monitoring video frames to determine the posture tilt index for each frame.

[0031] Preferably, in one embodiment of the present invention, the method for obtaining the attitude tilt index includes: For ease of explanation and illustration, we will select one frame from the surveillance video as the frame to be tested.

[0032] Analyze the angle between the body centerline vector of the person under test and the ground in the test frame and its variation characteristics in different monitoring video frames to determine the first tilt factor corresponding to the test frame: In the test frame, obtain the angle between the straight line containing the body centerline vector of the person under test and the horizontal line of the ground, as the first angle corresponding to the test frame. When there is no tendency to fall, the first angle should be close to 90 degrees, while when the person tilts or falls, the first angle will be significantly reduced.

[0033] The mean of the first included angle of all monitoring video frames preceding the test frame is calculated as the first mean feature value. This first mean feature value represents the average level of the first included angle of all monitoring video frames preceding the test frame and can provide a reference for abnormalities in the first included angle of the test frame. The difference between the first mean feature value corresponding to the test frame and the first included angle of the test frame is calculated. If this difference is positive, it indicates that the first included angle of the test frame is smaller compared to previous monitoring video frames, meaning the closer the body is to the ground, and therefore the more obvious the tendency to fall. Therefore, this difference is multiplied by a preset reduction factor and normalized, and this value is used as the first tilting factor corresponding to the test frame. Based on the previous analysis, a larger first tilting factor indicates a higher tendency to fall. Since the difference here may be positive or negative, the normalization here can use... The function is configured, and a preset scaling factor, such as 0.1, is set to map the values ​​to the function's sensitive area.

[0034] When the human body is in an ideal posture, such as standing, the spine maintains its physiological curvature, and the head, shoulders, and hips are basically vertically aligned, resulting in the upper and lower torsos remaining vertically aligned. However, when a fall occurs, the body's postural balance is disrupted, leading to a short-term decrease in the collinearity of the upper and lower torsos. Therefore, in the test frame, the angle between the vectors of the upper and lower torsos of the person being tested is taken as the second angle corresponding to the test frame. The second angle reflects the degree of relative twisting or bending between the upper and lower torsos in the test frame; a larger value indicates a greater degree of twisting or bending. Similarly, the mean of the second angles of all monitoring video frames preceding the test frame is calculated as the second mean feature value. The second mean feature value characterizes the average level of body twisting posture in all monitoring video frames preceding the test frame, providing a reference for abnormalities in the second angle of the test frame. The difference between the second included angle corresponding to the test frame and the second mean characteristic value corresponding to the test frame is calculated. If this difference is positive, it indicates that the degree of posture distortion in the test frame has increased compared to previous moments, thus the falling trend will be more obvious. Therefore, this difference is multiplied by a preset reduction factor and normalized, and this normalized value is used as the second tilt factor corresponding to the test frame. Based on the aforementioned analysis, the larger the second tilt factor, the higher the tendency to fall. Since the difference here may be positive or negative, the normalization here can use... The function is configured, and a preset scaling factor, such as 0.1, is set to map the values ​​to the function's sensitive area.

[0035] Finally, since both the first and second tilt factors are positively correlated with the probability of the person falling in the test frame, the normalized product of the first and second tilt factors corresponding to the test frame is used as the attitude tilt index of the test frame. This normalization can be achieved using maximum-minimum normalization, where the maximum and minimum values ​​can be obtained based on a large amount of experimental data.

[0036] It should be noted that in this embodiment of the invention, the included angle can be calculated using vector method and straight line slope method, both of which are well known technologies and will not be elaborated here. In this embodiment of the invention, the coordinate system is set with the horizontal direction to the right as the positive X-axis and the vertical direction downward as the positive Y-axis.

[0037] Considering that in normal human states such as standing, walking, and sitting, the vertical height of the body is much greater than the horizontal width, when a fall occurs, the body quickly tilts to the ground from an upright position. The shoulders will shift significantly in the tilting direction, resulting in a significant increase in the horizontal distance between the acromion and the hip joint. The horizontal width of the body will increase significantly, while the vertical height will decrease sharply, resulting in a large aspect ratio of the external outline rectangle. Therefore, we can also analyze the outline features of the target boxes of body parts in each frame of the surveillance video, and combine them with the posture tilt index obtained above to determine the importance of outline analysis.

[0038] Preferably, in one embodiment of the present invention, the method for obtaining the importance of contour analysis includes: In each frame of the surveillance video, the ratio of the horizontal length to the vertical length of the target bounding box of the body part of the person under test is obtained as the aspect ratio of each frame. The smaller the aspect ratio, the closer the posture of the person under test is to an upright state in that frame. Conversely, the larger the aspect ratio, the closer the posture of the person under test is to a tilted or even fallen posture in that frame.

[0039] Then, the ratio of the aspect ratio of each monitoring video frame to the preset aspect ratio is calculated as the contour deformation factor. In this embodiment of the present invention, the preset aspect ratio can be obtained based on historical experience data or prior values ​​of a normal human body standing. Therefore, the larger the contour deformation factor, the higher the probability of the person being tested falling.

[0040] Based on the logic in the aforementioned steps, it can be seen that both the contour deformation factor and the posture tilt index are directly proportional to the probability of the person being tested falling. Therefore, in each frame of the monitoring video, the product of the contour deformation factor and the posture tilt index is used as the contour analysis importance of each frame of the monitoring video. At this time, the greater the contour analysis importance, the higher the probability that the person being tested has lost balance and is likely to fall.

[0041] It should be noted that since the first monitoring video frame in the time sequence does not contain any previous monitoring video frames, its corresponding contour analysis importance can be set as the average of the contour analysis importance of all remaining monitoring video frames.

[0042] Step S3: Based on the positional changes of key human body points in adjacent monitoring video frames, determine the emphasis of the trajectory of each monitoring video frame.

[0043] During normal activities such as walking or sitting, the shoulder line and waist line of the human body usually remain relatively parallel and move in coordination, with relatively gentle and consistent changes in direction. However, when a fall occurs, the body loses its balance, and the upper body shoulder line and lower body waist line often twist or tilt in different directions. Furthermore, when a fall is imminent, the body's center of gravity and waistline will exhibit a matching behavior. For example, the body's center of gravity drops sharply, and the higher the body's key points, the faster the vertical descent speed. For instance, the head descends faster vertically than the legs. Therefore, it is also possible to analyze the positional changes of human key points in adjacent monitoring video frames to determine the emphasis of the trajectory of each monitoring video frame.

[0044] Preferably, in one embodiment of the present invention, the method for obtaining the intensity of the trajectory includes: First, in each monitoring video frame, the line connecting two key points on the shoulders in the target frame of the body parts of the person being tested is taken as the shoulder line, and the line connecting two key points on the hips is taken as the waist line.

[0045] For any given frame of a surveillance video, calculate the angle between the shoulder line and the waistline in that frame, and use this angle as the third angle. During normal walking or movement, although the shoulders and hips swing, they remain largely parallel and coordinated. However, when falling, the body loses balance and attempts to regain balance, often resulting in a violent, asymmetrical twisting of the upper and lower body. Therefore, the larger the angle between the shoulder line and the waistline, the more likely a fall is to occur.

[0046] The absolute value of the difference between the third angle between the current surveillance video frame and the adjacent previous surveillance video frame is calculated. This absolute value represents the change in the third angle between two adjacent surveillance video frames. The larger the value, the more significant the instability, and the more likely a fall has occurred. Therefore, this absolute value is normalized to obtain the shoulder and waist instability performance of the surveillance video frame. Based on the aforementioned logic, a larger shoulder and waist instability performance is considered a higher probability of a fall. The normalization here can use maximum and minimum value normalization, and the maximum and minimum values ​​can be obtained based on a large amount of experimental data.

[0047] Furthermore, in a fall, the body, such as the head, torso, and limbs, essentially begins to accelerate its descent relative to the ground. This is reflected in an aggravated sudden drop at the center of each key part. Therefore, for any key point on the human body, the displacement on the Y-axis in the vertical coordinate system between the key point in the current monitoring video frame and the key point in the adjacent previous monitoring video frame is calculated. The value of the displacement is equal to the difference between the vertical coordinate value of the key point in the current monitoring video frame and the vertical coordinate value of the key point in the adjacent previous monitoring video frame. In this embodiment of the invention, the positive direction of the Y-axis is taken as vertically downward. At this time, the displacement is positive, indicating that the position of the key point on the human body has fallen. The closer to the ground, the higher the probability of a fall.

[0048] Because higher key points on the human body experience faster vertical descent during a fall or tilt, the average positive displacement of all key points in the upper body was calculated, and then the ratio of this average to the average positive displacement of all key points in the lower body was determined. This ratio yields the upper-lower body displacement ratio coefficient. A larger coefficient indicates a more significant displacement of the upper body key points relative to the lower body key points, suggesting a higher probability that the person being tested is in the process of falling or tilting. If the denominator is zero when calculating the upper-lower body displacement ratio coefficient, a very small positive number (e.g., 10) is added to the denominator to prevent this error. -6 .

[0049] The overall downward displacement of all human body keypoints is calculated by averaging the positive displacement values ​​of all keypoints and then normalizing the result to obtain a downward displacement index. Based on the previous analysis, a larger downward displacement index indicates a higher probability of a fall. Normalization here can be achieved by dividing the independent variable by the vertical length of the body part's bounding box, thus eliminating the influence of distance and camera resolution.

[0050] Since the shoulder and waist instability performance, descent displacement index, and upper and lower body displacement ratio coefficient are all positively correlated with the probability of falling, the product of these parameters is multiplied, and the normalized product is used as the trajectory emphasis of that monitoring video frame. A higher trajectory emphasis indicates a higher probability of the person being tested falling in that frame, thus requiring closer attention. The normalization here can use maximum-minimum value normalization, which can be extracted from a large database of experimental data.

[0051] Step S4: By fusing and analyzing the contour analysis between surveillance video frames to identify the trend of changes in the intensity of the contour analysis and the trajectory showing a trend of significant changes, the fall behavior of the person being tested at the current moment can be identified.

[0052] Based on the aforementioned steps, the contour analysis importance and trajectory presentation importance corresponding to each monitoring video frame can be obtained, and both are positively correlated with the probability of the person being tested falling. Furthermore, if the tendency of the person being tested to fall is more significant in the short term, the warning level needs to be further increased. Therefore, in this step, the changing trends of contour analysis importance and trajectory presentation importance between monitoring video frames can be integrated and analyzed to identify the falling behavior of the person being tested at the current moment.

[0053] Preferably, in one embodiment of the present invention, the process of identifying the fall behavior of the person being tested at the current moment includes: Since both the emphasis on contour analysis and the emphasis on trajectory presentation are positively correlated with the fall posture, the emphasis on contour analysis and the emphasis on trajectory presentation are multiplied and normalized in each frame of the surveillance video to obtain the fall posture tendency for each frame. A higher fall posture tendency indicates a higher probability of a fall and a greater need for warning. Similarly, the normalization here can use maximum-minimum value normalization, where the maximum and minimum values ​​can be obtained from a database of extensive experimental data.

[0054] Then, the fall tendency of all monitoring video frames is fitted with a straight line according to the time sequence (the horizontal axis is the sequence number of the monitoring video frame, and the vertical axis is the fall tendency), and the slope value of the fitted line is obtained. This slope value can characterize the change trend of the fall tendency of the test subject over a period of time. When the slope value is positive and larger, it indicates that the real-time tilt of the test subject is more significant. Since the slope value may be positive or negative, the slope value is multiplied by the preset reduction coefficient and then the value is adjusted by the hyperbolic tangent function (tanh function) to obtain the adjustment factor. The value of the adjustment factor is between [-1, 1]. The larger the value, the higher the real-time tilt. The preset reduction coefficient can be 0.1, which is used to map the value to the function sensitivity zone.

[0055] The sum of the adjustment factor and the preset constant is used as the adjustment index. In order to prevent overshoot, the preset constant is set to 1 in this embodiment of the invention. At this time, the value of the adjustment index is a non-negative number. The larger the value, the higher the probability that the person being tested will fall.

[0056] Therefore, the normalized value of the product of the adjustment index and the fall tendency of the current monitoring video frame is used as the fall behavior recognition index for the person under test at the current moment. Based on the aforementioned logic, the larger the fall behavior recognition index, the more significant the possibility of the person under test falling at the current moment, and the more timely the warning is needed. The normalization here can be achieved using maximum and minimum value normalization, where the maximum and minimum values ​​can be extracted from a large amount of experimental data into a database.

[0057] Finally, if the fall behavior identification index of the person being tested is greater than or equal to the preset fall threshold at the current moment, a fall warning needs to be issued.

[0058] It should be noted that the least squares method is a well-known technique, and the specific process will not be described in detail here; the preset fall threshold can be set to the range of [0.6, 1), and in this embodiment of the present invention it is set to 0.65. The specific value can be adjusted according to the implementation scenario, and will not be described in detail here.

[0059] In summary, firstly, acquiring multiple consecutive video frames of the person under test at the current moment can capture the dynamic changes in their movements over a short period. Simultaneously, accurately locating the bounding boxes of the person's body parts and extracting key human points clarifies the positional range of each body part within the video frames, helping to quantify the person's motion state and providing a more comprehensive data foundation for accurate fall identification. Since, ideally, the human body's force lines usually maintain relative balance, and a fall essentially disrupts this balance, manifesting as a rapid tilt of the body's force lines towards the ground and complex changes in these lines, a torso vector can be constructed based on the key human points. After converting the person's complex body posture into quantifiable vector information, the angle between the torso vector and the ground, the angles between torso vectors, and the differences in these angles across different video frames are analyzed to determine the person's posture tilt index in each frame. During a fall, the person's body contour changes significantly; therefore, combining this posture tilt index with the contour features of the bounding boxes of body parts determines the importance of contour analysis. The positional changes of key points on the human body between different frames reflect the movement trajectory of the person. By analyzing these positional changes, it can be determined whether the person is in a normal movement state or has experienced an abnormal fall. Therefore, the emphasis of the trajectory is determined based on this feature. Finally, by fusing and analyzing the trend of contour analysis emphasis and trajectory emphasis between surveillance video frames, interference such as single-frame occlusion is eliminated, ensuring the continuity and accuracy of the fall behavior recognition results.

[0060] This invention also provides a fall behavior recognition system for wireless surveillance video. Please refer to [link to relevant documentation]. Figure 2 The diagram shows a system block diagram, including a data acquisition module 201 for implementing step S1 in the above method embodiment; a motion state analysis module 202 for implementing steps S2 and S3 in the above method embodiment; and a fall behavior recognition module 203 for implementing step S4 in the above method embodiment.

[0061] It should be noted that the system architecture includes a processor, a memory, a bus, and a communication interface. The processor, communication interface, and memory are connected via the bus. The memory may include high-speed random access memory, and the bus may be an ISA bus, PCI bus, or EISA bus, etc. The processor may be an integrated circuit chip with signal processing capabilities. The memory stores at least one instruction, at least one program, code set, or instruction set. When the processor loads and executes the at least one instruction, at least one program, code set, or instruction set, it implements the steps in a fall behavior recognition method for wireless surveillance video.

[0062] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A fall behavior recognition method for wireless monitoring of video, characterized by, The method includes: At the current moment, acquire multiple consecutive monitoring video frames of the person under test, locate the target bounding box of the body parts of the person under test in each monitoring video frame, and extract the key points of the human body. In each monitoring video frame, a body torso vector is constructed based on human key points; the angle between the body torso vector and the ground, the angle between body torso vectors, and the differences in angles in different monitoring video frames are analyzed to determine the posture tilt index for each monitoring video frame; based on the contour features of the target boxes of body parts in each monitoring video frame, and in combination with the posture tilt index, the contour analysis importance is determined. Based on the positional changes of key human body points in adjacent surveillance video frames, the intensity of the trajectory of each surveillance video frame is determined. By fusing and analyzing the contours and trajectories of surveillance video frames to identify the falling behavior of the person being tested at the current moment, the system can identify the falling behavior of the person being tested at the current moment. 2.The fall behavior recognition method for wireless monitoring video of claim 1, wherein, The step of locating the target bounding box of the body parts of the person under test in each frame of the surveillance video and extracting key points of the human body includes: The human body target detection box of the person to be tested is obtained in each frame of the surveillance video using a target detection algorithm, and the human body target detection box is used as the target box of the body part of the person to be tested. Multiple human body key points within the target bounding box of the body parts of the person under test are obtained using a real-time pose estimation model. The human body key points include at least head key points, shoulder key points, hip key points, and foot key points. 3.The fall behavior recognition method for wireless monitoring video of claim 2, wherein, The method for obtaining the body trunk vector includes: Within the target bounding box of the body parts of the person being tested, the midpoint of the line connecting two foot key points in the human body key points is taken as the starting point, and the head key point is taken as the ending point to obtain the body centerline vector. The lower body trunk vector is obtained by taking the midpoint of the line connecting the two foot key points as the starting point and the midpoint of the line connecting the two hip key points as the ending point. The upper body trunk vector is obtained by taking the midpoint of the line connecting the two hip key points as the starting point and the head key point as the ending point.

4. The fall behavior recognition method for wireless monitoring video according to claim 3, characterized in that, The method for obtaining the attitude tilt index includes: Select any frame of the surveillance video as the test frame, analyze the angle between the body centerline vector of the person being tested and the ground in the test frame and its variation characteristics in different surveillance video frames, and determine the first tilt factor corresponding to the test frame. In the frame to be tested, the angle between the upper half torso vector and the lower half torso vector of the person to be tested is taken as the second angle corresponding to the frame to be tested. The mean of the second angles of all monitoring video frames before the time sequence of the frame to be tested is calculated as the second mean feature value. The difference between the second included angle corresponding to the frame under test and the second mean feature value corresponding to the frame under test is multiplied by a preset reduction coefficient and then normalized to become the second tilt factor corresponding to the frame under test. The normalized value of the product of the first tilt factor and the second tilt factor corresponding to the frame under test is used as the attitude tilt index of the frame under test.

5. The fall behavior recognition method for wireless monitoring video according to claim 4, characterized in that, The method for obtaining the first dumping factor includes: In the frame to be tested, the angle between the straight line containing the body centerline vector of the person under test and the horizontal line of the ground is obtained as the first angle corresponding to the frame to be tested. The mean of the first angles of all monitoring video frames before the time sequence of the frame to be tested is calculated as the first mean feature value. The difference between the first mean feature value corresponding to the frame under test and the first angle between the frame under test is calculated, and the difference is multiplied by a preset reduction coefficient and normalized to obtain the first tilt factor corresponding to the frame under test.

6. The method for fall behavior recognition in wireless surveillance video according to claim 1, characterized in that, The method for obtaining the importance of the contour analysis includes: In each frame of the surveillance video, the ratio of the length of the target box of the body part of the person under test along the horizontal direction to the length along the vertical direction is obtained, which is used as the aspect ratio of each frame of the surveillance video. The ratio of the aspect ratio of each monitoring video frame to the preset aspect ratio is used as the contour deformation factor. The product of the contour deformation factor and the attitude tilt index of each monitoring video frame is used as the contour analysis importance of each monitoring video frame.

7. A method for recognizing fall behavior in wireless surveillance video according to claim 2, characterized in that, The methods for obtaining the intensity of the trajectory include: In each monitoring video frame, the line connecting two key points on the shoulders in the target frame of the body parts of the person being tested is taken as the shoulder line, and the line connecting two key points on the hips is taken as the waist line. For any frame of surveillance video, calculate the angle between the shoulder line and the waist line in that frame, and use it as the third angle. Calculate the absolute value of the difference between the third angle between the current monitoring video frame and the adjacent previous monitoring video frame, and normalize it to obtain the shoulder and waist instability performance of the monitoring video frame. For any human body key point, calculate the displacement of the human body key point in the current monitoring video frame and the human body key point in the adjacent previous monitoring video frame on the Y-axis in the vertical coordinate system. The ratio of the average positive displacement of all key points in the upper body to the average positive displacement of all key points in the lower body is calculated to obtain the upper and lower body displacement ratio coefficient. The normalized value of the average positive displacement of all human body key points is used as the descent displacement index. The shoulder and waist instability performance is multiplied by the descent displacement index and the upper and lower body displacement ratio coefficient, and the normalized value of the product is used as the trajectory presentation emphasis of the monitoring video frame.

8. A method for fall behavior recognition in wireless surveillance video according to claim 1, characterized in that, The fusion analysis of the contour analysis between the surveillance video frames shows a trend of significant changes in importance and trajectory, enabling the identification of the fall behavior of the person being tested at the current moment, including: The contour analysis emphasis of each monitoring video frame is multiplied by the trajectory presentation emphasis and then normalized to obtain the fall tendency degree corresponding to each monitoring video frame. The fall trend of all monitored video frames is fitted with a straight line according to the time sequence to obtain the slope value of the fitted line. The slope value is multiplied by a preset reduction coefficient and then adjusted using the hyperbolic tangent function to obtain the adjustment factor. The sum of the adjustment factor and the preset constant is used as the adjustment index. The product of the adjustment index and the fall tendency of the monitoring video frame at the current moment is normalized and used as the fall behavior recognition index of the person under test at the current moment. If the fall behavior identification index of the person being tested is greater than or equal to the preset fall threshold at the current moment, a fall warning needs to be issued.

9. A method for recognizing fall behavior in wireless surveillance video according to claim 8, characterized in that, The preset fall threshold value ranges from [0.6, 1].

10. A fall behavior recognition system for wireless surveillance video, characterized in that, The method includes a processor and a memory, the memory storing at least one instruction, at least one program, code set, or instruction set, wherein when the at least one instruction, at least one program, code set, or instruction set is loaded and executed by the processor, it implements the steps of the fall behavior recognition method for wireless surveillance video as described in any one of claims 1-9.

Images