A vision-based fall behavior recognition and alarm method for outdoor fitness running scenes

Abstract

The present invention belongs to the technical field of outdoor sports safety monitoring, and is specifically a fall behavior recognition and alarm method for outdoor fitness running scenes based on vision, comprising step 1: collecting RGB image information of human body through a monocular camera arranged on an outdoor public fitness trail; step 2: performing human body detection on the picture, and obtaining the detection frame x, y, w, h of each person in the picture, wherein (x, y) represents the upper left corner point of the detection frame, w represents the width of the detection frame, and h represents the height of the detection frame; step 3: performing human posture estimation on each detection frame, and obtaining 24 skeleton key points {x1, y1, ..., x 24 ,y 24}, which has a reasonable structure and can be used in outdoor fitness tracks. Based on a monocular camera, a fall model database is established. The visual image of the monocular camera recognizes the runner's accidental fall or accidental fall, and implements a system alarm.

Description

Technical Field

[0001] The present invention relates to the technical field of outdoor sports safety monitoring, and in particular to a vision-based fall behavior recognition and alarm method in an outdoor fitness running scene. Background Art

[0002] Vision-based fall detection technology is becoming the mainstream technology for fall detection because of its high recognition rate, low impact on external factors such as light and temperature, and video recording function for easy verification afterwards. For example, in retirement communities, the use of vision-based fall detection alarm systems instead of manual patrols is very suitable for real-time monitoring of various emergencies, and can promptly detect the fall behavior of the elderly, ensuring the safety of the elderly, and the monitoring success rate is higher than that of systems based on wearable devices.

[0003] Application (patent) number: CN 112115827 A. Patent description: A method for identifying fall behavior based on dynamic characteristics of human posture, characterized by: processing the RGB image information of the human body collected by a monocular camera to obtain key points of human posture, constructing posture vectors representing the main trunk and legs of the human body: calculating the offset angle and body length ratio of the main trunk and legs of the human body, using them as key dynamic quantities to characterize the tilted posture of the human body, integrating multiple key dynamic quantities of the human body into human body inclination posture feature symbols and human body stable posture feature symbols to describe human body state changes: using reference template vector comparison, lower limb stability energy value and state score to form a feature vector, applying SVM to complete the binary classification of fall identification, and combining the threshold judgment of the dynamic characteristics of the human body's falling posture to improve the accuracy and precision of human fall behavior identification.

[0004] Application (patent) number: CN 112489368 A. Patent description: A method and system for intelligent fall recognition and detection alarm, the method comprising: establishing a fall data set, building a fall recognition model, and training the fall recognition model based on the fall data set; inputting the collected motion image information of people in the monitoring area into the fall recognition model, analyzing the collected motion image information, and judging whether there is a person in a fall posture; marking and tracking the target person with a fall posture, and judging whether to alarm according to the fall analysis and tracking detection results. Application (patent) number: CN 112669568A. Patent description: A multi-mode human fall detection method, comprising: collecting acceleration data of the detection object; judging the state of the detection object according to the acceleration data; collecting the surface electromyography signal of the detection object when judging that the detection object is in a fall state; extracting electromyography signal features from the surface electromyography signal; identifying the electromyography signal features through a support vector machine; triggering an alarm system when the support vector machine recognizes that the detection object is in a fall state. Application (patent) number: CN 112435440 A. Patent description: A non-contact indoor person fall identification method based on the Internet of Things platform, which transmits the captured infrared thermal image including the thermal information of the indoor person to the cloud server in real time through the NB-IoT narrowband wireless communication module, and at the same time, returns the alarm information of the person fall to the monitoring device installed in the indoor monitoring area, realizing two-way wireless connection, no local gateway, no central network, and has industrial-grade system reliability. In this way, there is no need to connect to the home broadband network, and a wide coverage of non-contact indoor person fall identification can be realized. Application (patent) number: CN 112346050 A. Patent description: A fall detection method based on a Wi-Fi device, the fall detection method based on a Wi-Fi device comprising: determining a human body velocity feature based on channel state information of a wireless signal acquired in a target area; inputting the human body velocity feature into a fall detection model to obtain a human fall recognition result output by the fall detection model; the fall detection model is trained based on a fall / non-fall sample data set; the non-fall samples in the fall / non-fall sample data set are from samples of daily activities of people that are actually collected, and the fall samples in the fall / non-fall sample data set are obtained by preprocessing the input real fall samples based on a data expansion model.

[0005] The above technologies for identifying human falls are mainly based on the following two methods: The first is based on wearable devices or acceleration sensors for identification. This identification method requires people with a risk of falling to actively wear sensor-related equipment before they can identify the fall event. If they are not worn, they cannot be identified. Moreover, when the human body is in motion, the sensor-related equipment will hinder the movement of the athlete to a certain extent. The second is a method based on computer vision. This identification method mainly captures the human motion picture through a monocular, binocular or multi-eye camera, inputs its motion image information into the fall recognition model, and analyzes the collected motion image information. On the one hand, this method relies on the detection accuracy of the human posture points, and on the other hand, the collected human motion images are easily affected by the camera angle, making the motion image information inaccurate and the recognition accuracy low. At the same time, the current focus of identifying the population of falls is on the elderly, and most of them are indoor fall recognition. This single population recognition method design cannot meet more outdoor environments. At present, the limitations of the methods or technologies for identifying falls are large and there are few application scenarios.

[0006] To this end, we propose a new vision-based fall behavior recognition and alarm method in outdoor fitness running scenarios to solve the above problems. Summary of the invention

[0007] The purpose of this section is to summarize some aspects of the embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the specification abstract and the invention title of this application to avoid blurring the purpose of this section, the specification abstract and the invention title, and such simplifications or omissions cannot be used to limit the scope of the present invention.

[0008] In view of the problems existing in the existing vision-based fall behavior recognition and alarm methods in outdoor fitness running scenes, the present invention is proposed.

[0009] Therefore, the purpose of the present invention is to provide a vision-based fall behavior recognition and alarm method in an outdoor fitness running scene, which can be implemented with an outdoor fitness track as the application scenario, based on a monocular camera, to establish a fall model database, the visual image of the monocular camera can recognize the runner's accidental fall or accidental fall, and implement a system alarm.

[0010] To solve the above technical problems, according to one aspect of the present invention, the present invention provides the following technical solutions:

[0011] A vision-based fall behavior recognition and alarm method in an outdoor fitness running scene comprises the following steps:

[0012] Step 1: Collect human RGB image information through a monocular camera placed on an outdoor public fitness trail;

[0013] Step 2: Perform human body detection on the image and obtain the detection frame x, y, w, h of each person in the image, where (x, y) represents the upper left corner of the detection frame, w represents the width of the detection frame, and h represents the height of the detection frame;

[0014] Step 3: Perform human posture estimation for each detection frame and obtain 24 skeleton key points {x1, y1, …, x 24 ,y 24};

[0015] Step 4: Encode each person's 24 skeletal key points through the motion posture coding model to form a 128-dimensional floating-point normalized feature vector;

[0016] Step 5: Select various falling postures N, extract features from these postures through steps 1-3, and make a falling feature library F f =(f f1 ,…,f fN ). Select various postures M that are similar to falling down, such as lying down or sitting down, extract features from these postures through steps 1-3, and make a non-falling down feature library F n =(f n1 ,…,f nM );

[0017] Step 6: Obtain a video stream from a common surveillance camera and extract each frame of the video stream. Extract feature f from each frame through steps 1-3, and compare the feature with the features in the falling feature library and the non-falling feature library. If the maximum comparison score of feature f and the feature in the falling feature library exceeds the threshold τ f , at the same time, the maximum score of feature f compared with the feature in the non-falling feature library is lower than the threshold τ n , it is considered to be in a falling state;

[0018] Step 7: If the falling state lasts for a period of time t, which can be set to 5 seconds, it can be determined that someone has fallen and an alarm can be sounded.

[0019] As a preferred solution of the vision-based fall behavior recognition and alarm method in an outdoor fitness running scene described in the present invention, an outdoor fitness track is used as the application scene.

[0020] As a preferred solution of the vision-based fall behavior recognition and alarm method in the outdoor fitness running scene described in the present invention, wherein: a video stream is obtained from an ordinary surveillance camera, each frame of the video stream is extracted, and features are extracted from each frame of the image through the above steps 2-4.

[0021] As a preferred solution of the vision-based fall behavior recognition and alarm method in the outdoor fitness running scene described in the present invention, the system alarm prevention includes monitoring terminal reminders, mobile terminal communication alarms and 120 emergency calls.

[0022] As a preferred solution of the vision-based fall behavior recognition and alarm method in the outdoor fitness running scene described in the present invention, the camera is a panoramic camera and has an automatic portrait tracking system.

[0023] Compared with the prior art, the beneficial effects of the present invention are: the runner's accidental fall or accidental fall is recognized through the visual image of the monocular camera, and the system alarm is implemented. This method is a vision-based fall behavior recognition and alarm method for outdoor fitness running scenes. The video stream of the runner is obtained through an ordinary camera on the outdoor fitness trail, each frame of the picture is extracted from the video stream, and the skeleton key points for evaluating the human posture are extracted from the picture to form a motion posture code, which is compared with the established fall feature library and non-fall feature library, thereby realizing the recognition of the accidental fall or fall state of the runner. At the same time, by using the data improvement of the feature library, one can solve the problem of deformation of the human posture geometry diagram obtained due to the camera angle; two can realize the recognition of the runner's forward fall, side fall, sitting, shoe lacing and other postures; three can realize the recognition of overlapping visual images of multiple people, thereby greatly improving the recognition rate of fall events in outdoor fitness running scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the present invention will be described in detail below in combination with the accompanying drawings and detailed embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. Among them:

[0025] Figure 1 It is a schematic diagram of the step flow structure of the present invention. DETAILED DESCRIPTION

[0026] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0027] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0028] Secondly, the present invention is described in detail with reference to schematic diagrams. When describing the embodiments of the present invention in detail, for the sake of convenience, the cross-sectional diagrams showing the device structure will not be partially enlarged according to the general scale, and the schematic diagrams are only examples, which should not limit the scope of protection of the present invention. In addition, in actual production, the three-dimensional dimensions of length, width and depth should be included.

[0029] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0030] Example 1

[0031] A vision-based fall behavior recognition and alarm method in an outdoor fitness running scene comprises the following steps:

[0032] Step 1: Collect human RGB image information through a monocular camera placed on an outdoor public fitness trail;

[0033] Step 2: Perform human body detection on the image and obtain the detection frame x, y, w, h of each person in the image, where (x, y) represents the upper left corner of the detection frame, w represents the width of the detection frame, and h represents the height of the detection frame;

[0034] Step 3: Perform human posture estimation for each detection frame and obtain 24 skeleton key points {x1, y1, …, x 24 ,y 24};

[0035] Step 4: Encode each person's 24 skeletal key points through the motion posture coding model to form a 128-dimensional floating-point normalized feature vector;

[0036] Step 5: Select various falling postures N, extract features from these postures through steps 1-3, and make a falling feature library F f =(f f1 ,…,f fN ). Select various postures M that are similar to falling down, such as lying down or sitting down, extract features from these postures through steps 1-3, and make a non-falling down feature library F n =(fn1,…,f nM );

[0037] Step 6: Obtain a video stream from a common surveillance camera and extract each frame of the video stream. Extract feature f from each frame through steps 1-3, and compare the feature with the features in the falling feature library and the non-falling feature library. If the maximum comparison score of feature f and the feature in the falling feature library exceeds the threshold τ f , at the same time, the maximum score of feature f compared with the feature in the non-falling feature library is lower than the threshold τ n, it is considered to be in a falling state;

[0038] Step 7: If the falling state lasts for a period of time t, which can be set to 5 seconds, it can be determined that someone has fallen and an alarm can be sounded.

[0039] Specifically, the outdoor fitness track is used as the application scenario.

[0040] Specifically, a video stream is obtained from a common surveillance camera, each frame of the image is extracted from the video stream, and features are extracted from each frame of the image through the above steps 2-4.

[0041] Specifically, the system alarm prevention includes monitoring terminal reminders, mobile terminal communication alarms and 120 emergency calls.

[0042] Specifically, the camera is a panoramic camera and has an automatic portrait tracking system.

[0043] Working principle: During the use of the present invention, the visual image of the monocular camera is used to identify the runner's accidental fall or accidental fall, and the system alarm is implemented. This method is a vision-based fall behavior recognition and alarm method for outdoor fitness running scenes. The video stream of the runner is obtained through an ordinary camera on an outdoor fitness trail, each frame of the picture is extracted from the video stream, and the skeletal key points for evaluating the human posture are extracted from the picture to form a motion posture code, which is compared with the established fall feature library and non-fall feature library, thereby realizing the recognition of the runner's accidental fall or fall state. At the same time, by using the data improvement of the feature library, one can solve the problem of deformation of the human posture geometry diagram obtained due to the camera angle; second, it can realize the recognition of the runner's forward fall, side fall, sitting, shoe lacing and other postures; third, it can realize the recognition of overlapping visual images of multiple people, thereby greatly improving the recognition rate of fall events in outdoor fitness running scenes.

[0044] Although the present invention has been described above with reference to the embodiments, various modifications may be made thereto and parts thereof may be replaced by equivalents without departing from the scope of the present invention. In particular, as long as there is no structural conflict, the various features in the embodiments disclosed in the present invention may be used in combination with each other in any manner, and the fact that these combinations are not exhaustively described in this specification is only for the sake of omitting space and saving resources. Therefore, the present invention is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A vision-based fall behavior recognition and alarm method in an outdoor fitness running scene, characterized in that: The steps include: Step 1: Collect human RGB image information through a monocular camera placed on an outdoor public fitness trail; Step 2: Perform human body detection on the image and obtain the detection frame of each person in the image ,in Indicates the upper left corner of the detection box. Indicates the width of the detection box. Indicates the height of the detection box; Step 3: Estimate the human posture for each detection frame and obtain 24 skeleton key points for each person ; Step 4: Encode each person's 24 skeletal key points through the motion posture coding model to form a 128-dimensional floating-point normalized feature vector; Step 5: Select various falling postures , extract features from these postures through steps 1 to 3, and make them into a falling feature library , select various postures similar to falling , including lying and sitting, extract features from these postures through steps 1 to 3, and make a non-falling feature library ; Step 6: Obtain the video stream from the ordinary surveillance camera, extract each frame of the video stream, and extract features from each frame through steps 1 to 3 , compare the feature with the features in the falling feature library and the non-falling feature library respectively. If the feature The maximum score of the feature comparison with the feature library exceeds the threshold , while features The maximum score of feature comparison with the non-falling feature library is lower than the threshold , it is considered to be in a falling state; Step 7: If the downed state lasts for a period of time ,set up If the time is less than 5 seconds, it is judged that someone has fallen to the ground and the police are alerted.

2. The method for identifying and alarming a fall in an outdoor fitness running scene based on vision according to claim 1, characterized in that: The application scenario is outdoor fitness track.

3. The method for identifying and alarming a fall in an outdoor fitness running scene based on vision according to claim 1, characterized in that: Obtain a video stream from a common surveillance camera, extract each frame of the video stream, and extract features from each frame of the video through steps 2 to 4 above.

4. The method for identifying and alarming a fall in an outdoor fitness running scene based on vision according to claim 1, characterized in that: The system alarm modes include monitoring terminal reminders, mobile terminal communication alarms and 120 emergency calls.

5. The method for identifying and alarming a fall in an outdoor fitness running scene based on vision according to claim 1, characterized in that: The camera is a panoramic camera with an automatic portrait tracking system.

Images