Drowning behavior analysis method and drowning prevention system based on multi-target tracking and recognition

Abstract

The present invention discloses a drowning behavior analysis method and a drowning prevention system based on multi-target tracking and identification. A multi-target identification algorithm is first used to identify human targets, locate human positions and mark human key points, and then a multi-target tracking algorithm is used to track human targets, human positions and human key points. Then, a dynamic information library of human targets is established based on human positions and human key points. Finally, a drowning behavior analysis algorithm is used to analyze the dynamic information library to generate a swimmer calibration screen and early warning information. The present invention uses computer vision technology to quickly identify the potential drowning risk of swimmers, prevent accidents before they happen, has the advantages of accuracy and stability, can achieve full-time and all-round monitoring, effectively reduces labor costs and resource consumption, is conducive to reducing the incidence of drowning accidents, and improves the safety of swimming.

Description

Technical Field

[0001] The invention relates to a drowning behavior analysis method and a drowning prevention system based on multi-target tracking and identification, belonging to the drowning prevention safety management technology of a swimming venue. Background Art

[0002] Nowadays, new application models and solutions formed by computer vision technology have broad application prospects in improving the safety performance and economic benefits of swimming venues. AI algorithms based on visual monitoring are trained on a large number of data sets and have sufficiently high accuracy. They can implement 24-hour uninterrupted monitoring, intelligent identification and real-time warning of managed waters, and can accurately identify whether there are people entering outside of opening hours, swimmers drowning, etc. Summary of the invention

[0003] Purpose of the invention: In order to overcome the deficiencies in the prior art, the present invention provides a drowning behavior analysis method and an anti-drowning system based on multi-target tracking and identification, which provides a solution for future swimming venue safety management through the collaborative work of multiple modules including data sensing equipment, back-end processing platform, front-end display and front-end alarm, combined with multi-target identification and tracking algorithm and drowning behavior analysis algorithm.

[0004] Technical solution: To achieve the above purpose, the technical solution adopted by the present invention is:

[0005] A drowning behavior analysis method comprises the following steps:

[0006] Step 1: Build a dynamic information database to store the human body position and human body key point information of all target human bodies at multiple times; the dynamic information database uses the globally unique ID number of each target human body as the primary key, and establishes a dynamic information queue for each target human body. The dynamic information queue stores the human body position label and human body key point label of the target human body in the order of timestamps; enter Step 2;

[0007] Step 2: Compare the latest dynamic information entry in the dynamic information queue of each human target with the dynamic information entry before the set time length, and calculate the displacement value according to the human position tag: if the displacement value is less than the set displacement value, it is determined that the target human body has not undergone obvious displacement within the set time length, and enter step 3; otherwise, repeat step 2;

[0008] Step 3: Detect the target human body posture and obtain the target human body key point label KP from the latest dynamic information entry in the dynamic information queue. ls ,Y ls ),(X rs ,Y rs ),(X lh ,Y lh),(X rh ,Y rh )], where: (X ls ,Y ls ) is the left shoulder coordinate of the target body, (X rs ,Y rs ) is the right shoulder coordinate of the target body, (X lh ,Y lh ) is the left hip coordinate of the target body, (X rh ,Y rh ) is the right hip coordinate of the target human body; go to Step 4;

[0009] Step 4: Calculate the direction vector of the target human posture That is, the vector from the midpoint of the left and right shoulders of the target human body to the midpoint of the left and right hips; go to Step 5;

[0010] Step 5: Let the ground horizontal vector be Calculate the angle between the target human body and the horizontal direction of the ground If 0°<θ<60°, the target human body is judged to be in a non-standing state and has a drowning risk; otherwise, return to Step 2.

[0011] A drowning prevention system based on multi-target tracking and recognition comprises a back-end processing platform. The back-end processing platform first adopts a multi-target recognition algorithm to identify human targets, locate human positions and mark human key points in a video stream of a swimming pool, then adopts a multi-target tracking algorithm to track human targets, human positions and human key points, then establishes a dynamic information library of human targets based on human positions and human key points, and finally adopts a drowning behavior analysis algorithm to analyze the dynamic information library to generate a swimmer calibration picture and warning information.

[0012] Specifically, the above-mentioned anti-drowning system based on multi-target tracking and recognition has the following steps in its workflow:

[0013] S01: Process the video stream by framing, mark the target human body, human body position and human body key points in the video frame to form an original frame, pre-process the original frame and construct a training data set;

[0014] S02: constructing a multi-target recognition and tracking module, and updating the parameters of the multi-target recognition and tracking module by a gradient optimization method; the multi-target recognition and tracking module includes a multi-target recognition unit and a multi-target tracking unit;

[0015] The multi-target recognition unit inputs the video frames in the training data set and outputs the human body positions and human body key point prediction results of the multi-targets;

[0016] The multi-target tracking unit inputs the human body positions and original frames of multiple targets at multiple times, and matches the trajectory information of each human body target in different video frames;

[0017] S03: Input the real-time video stream into the trained multi-target recognition and tracking module to recognize and track each human target and the corresponding human position and human key points in each video frame in real time;

[0018] S04: Build a dynamic information database to store the human body position and key point information of multiple targets at multiple times;

[0019] S05: Construct a drowning behavior analysis module to analyze the dynamic information database and generate a swimmer calibration image and warning information.

[0020] Specifically, in step S01, the target human body, human body position and human body key points are marked in the video frame, and the process of constructing the training sample is: first, the video stream is divided into frames, and then each video frame is marked respectively, and the marking tool adopts the open source marking tool coco-annotator; marking the target human body means using a rectangular position detection frame to frame the swimmer in the video frame, the human body position label is composed of the upper left corner coordinates and the lower right corner coordinates of the position detection frame, and the human body key point label is composed of the left shoulder coordinates, right shoulder coordinates, left hip coordinates and right hip coordinates of the swimmer.

[0021] Specifically, in step S02, the multi-target recognition unit is composed of a convolutional neural network based on a feature pyramid, and the third-layer features, fourth-layer features and fifth-layer features output by the feature pyramid are subjected to network fusion encoding to obtain multi-scale fusion features, and the multi-scale fusion features are input into an output head composed of a fully connected layer to output the human body positions and human body key point prediction results of multiple targets.

[0022] Specifically, in step S02, the multi-target tracking unit associates the position detection frame in the previous video frame with the position detection frame in the current video frame based on the Kalman filtering method, extracts the appearance features of the target human body, thereby locating the position of the same target human body in consecutive video frames, and establishes a globally unique ID number for each target human body to achieve continuous tracking of the target human body.

[0023] Specifically, in step S04, a dynamic information library is constructed to store the human body position and human body key point information of all target human bodies at multiple times; the dynamic information library uses the globally unique ID number of each target human body obtained in the multi-target tracking unit as the primary key, and establishes a dynamic information queue for each target human body. After the multi-target recognition and tracking module processes each frame of video, it stores a dynamic information entry in the dynamic information queue corresponding to the target human body appearing in the video frame; the dynamic information entry includes the human body position label and human body key point label of the target human body in the video frame, and the timestamp of the video frame. Each dynamic information queue stores dynamic information entries of a set time length at most. When the timestamp difference between the first and last two dynamic information entries in the dynamic information queue is greater than the set time length, the dynamic information entry with the earliest timestamp in the dynamic information queue is deleted.

[0024] Specifically, in step S05, the drowning behavior analysis algorithm compares the first and last two dynamic information items in the dynamic information queue of each human target. If it is found that the calculated displacement value of the target human body in the two dynamic information items is less than the set displacement value, it is determined that the target human body has not undergone obvious displacement within the set time length. At this time, the target human body posture is detected, and the human body key point label KP of the target human body is obtained from the latest dynamic information item in the dynamic information queue:

[0025] KP=[(X ls ,Y ls ),(X rs ,Y rs ),(X lh ,Y lh ),(X rh ,Y rh )]

[0026] Where: (X ls ,Y ls ) is the left shoulder coordinate of the target body, (X rs ,Y rs ) is the right shoulder coordinate of the target body, (X lh ,Y lh ) is the left hip coordinate of the target body, (X rh ,Y rh ) is the right hip coordinate of the target body;

[0027] Calculate the direction vector of the target human posture That is, the vector from the midpoint of the left and right shoulders of the target human body to the midpoint of the left and right hips:

[0028]

[0029] Let the ground horizontal vector be Calculate the angle θ between the target human body and the horizontal direction of the ground:

[0030]

[0031] If 0°<θ<60°, the target human body is judged to be in a non-standing state and is at risk of drowning, and a warning message is generated and sent to the front-end alarm.

[0032] Specifically, it also includes a front-end display, which uses the WebSocket protocol to receive the swimmer calibration picture sent by the back-end processing platform. The front-end display is designed based on the Django framework and displays the swimmer calibration picture in real time in the form of video stream.

[0033] Specifically, it also includes a front-end alarm device, which is a smart bracelet, an early warning loudspeaker or an early warning screen. The back-end processing platform sends the early warning information to the front-end alarm device through the Wi-Fi communication protocol. The early warning information includes the target human body with drowning risk and its body position.

[0034] Beneficial effects: The swimming pool anti-drowning safety management system based on multi-target tracking and recognition provided by the present invention can quickly identify the potential drowning risk of swimmers through computer vision technology to prevent accidents before they happen. It has the advantages of accuracy and stability, and can achieve full-time and all-round monitoring, effectively reducing labor costs and resource consumption, which is conducive to reducing the incidence of drowning accidents and improving the safety of swimming. It is an important development direction for future safety management of swimming pools. Specifically, the present invention has the following advantages over the prior art: 1. Energy saving and environmental protection: by optimizing algorithms and hardware configurations, the energy consumption of the system can be reduced, which is in line with the concept of green and environmentally friendly sustainable development; 2. Reduction of labor costs: traditional swimming pool monitoring requires a large number of lifeguards to conduct uninterrupted patrols, while the present invention can automatically monitor and warn, greatly reducing the workload of lifeguards and reducing labor costs; 3. Efficient real-time monitoring and early warning: the system of the present invention can monitor the situation in the swimming pool in real time through advanced deep learning algorithms, quickly identify drowning behaviors, and effectively reduce missed reports and false alarms in manual monitoring; 4. Data-based management: the system of the present invention can store and analyze monitoring data to provide data support for the management of swimming pools; 5. Flexibility and scalability: the system of the present invention can be customized and expanded according to the specific needs of swimming pools to adapt to venues of different sizes and functions; the system of the present invention can also be integrated with other smart devices and services to achieve more intelligent management and services. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 The structural block diagram of the drowning prevention system based on multi-target tracking and identification of the present invention;

[0036] Figure 2 It is a schematic diagram of the implementation process of the system of the present invention;

[0037] Figure 3 It is a schematic diagram of the structure of a multi-target tracking unit;

[0038] Figure 4 Flowchart of the drowning behavior analysis algorithm. DETAILED DESCRIPTION

[0039] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0040] An anti-drowning system based on multi-target tracking and recognition can monitor the situation in the swimming pool in real time, accurately identify the drowning behavior of swimmers, and issue an alarm at the first time to notify lifeguards to carry out timely rescue, thereby improving the safety of swimming. The anti-drowning system includes a data sensing device, a back-end processing platform, a front-end display and a front-end alarm; the data sensing device is used to collect the video stream of the swimming pool in real time and send it to the back-end processing platform; the back-end processing platform first uses a multi-target recognition algorithm to identify human targets, locate the human position and mark the key points of the human body, and then uses a multi-target tracking algorithm to track the human target, the human position and the key points of the human body, and then establishes a dynamic information library of the human target based on the human position and key points of the human body, and finally uses a drowning behavior analysis algorithm to analyze the dynamic information library, generate a swimmer calibration picture and warning information, send the swimmer calibration picture to the front-end display, and send the warning information to the front-end alarm. Figure 1 As shown, the data perception device composed of a camera cluster transmits the monitoring image to the back-end processing platform deployed on the back-end server. The back-end processing platform transmits the swimmer's calibration image to the front-end display at the manager's place and transmits the warning information to the smart bracelet worn by the lifeguard.

[0041] like Figure 2 As shown, this is a workflow diagram of the above-mentioned anti-drowning system based on multi-target tracking and recognition, and each step is described in detail below.

[0042] S01: The video stream of the swimming pool is collected in real time through the data sensing device and sent to the backend processing platform.

[0043] The data sensing device is composed of a camera group consisting of several surveillance cameras deployed around the swimming pool. The camera group can capture the complete swimming pool image and pass these images to the back-end processing platform.

[0044] S02: Perform frame processing on the video stream, mark the target human body, human body position and human body key points in the video frame to form an original frame, and then pre-process the original frame to construct a training data set.

[0045] First, the video stream is divided into frames, and then each video frame is annotated. The annotation tool uses the open source annotation tool coco-annotator. Annotating the target human body means using a rectangular position detection frame to frame the swimmer in the video frame. The human position label is composed of the upper left corner coordinates and the lower right corner coordinates of the position detection frame. The human key point label is composed of the swimmer's left shoulder coordinates, right shoulder coordinates, left hip coordinates and right hip coordinates. The annotated video frame is used as the original frame, and then the original frame is preprocessed, including data enhancement methods such as random translation, random rotation, random scaling, random horizontal flipping or random cropping, so as to expand the training data set and enhance the robustness of the training.

[0046] S03: constructing a multi-target recognition and tracking module, and updating the parameters of the multi-target recognition and tracking module by a gradient optimization-based method; the multi-target recognition and tracking module includes a multi-target recognition unit and a multi-target tracking unit.

[0047] The multi-target recognition unit inputs the video frames in the training data set and outputs the prediction results of the human body positions and key points of the multi-targets. The multi-target recognition unit is composed of a convolutional neural network based on a feature pyramid, performs network fusion encoding on the third layer features, the fourth layer features and the fifth layer features output by the feature pyramid, obtains multi-scale fusion features, inputs the multi-scale fusion features into an output head composed of a fully connected layer, and outputs the prediction results of the human body positions and key points of the multi-targets.

[0048] The multi-target tracking unit inputs the human body positions and original frames of multiple targets at multiple times, and matches the trajectory information of each human body target in different video frames. Figure 3 As shown, the multi-target tracking unit, based on the Kalman filtering method, associates the position detection frame of the current video frame output by the multi-target recognition algorithm with the trajectory information of the previous frame to model the association, calibrates the same target body according to the appearance features, thereby locating the position of the same target body in consecutive video frames, and establishes a globally unique ID number for each target body in the target tracking algorithm to achieve continuous tracking of the target body.

[0049] In order to model the motion of the target in the plane image, a discrete Kalman filter with a constant velocity model is usually used. Tracking with a Kalman filter first requires the establishment of a system model of the target motion, including the state variables, state transition equations, and process noise of the target. The state variables usually include information such as the position and velocity of the target. The state transition equation describes how the target transitions from one state to the next in time. Process noise represents the uncertainty and noise in the state transition process.

[0050] In the multi-target tracking algorithm, in order to describe the target state, a state vector is defined, which contains 8 elements, which can be expressed as follows:

[0051]

[0052] Where: (x c (k),y c (k)) represents the center coordinate of the position detection frame of the current state, and (w(k), h(k)) represents the width and height of the position detection frame of the current state; Indicates the center point coordinates of the position detection box of the next state, Indicates the width and height of the position detection box of the next state.

[0053] The observation vector is represented as in, Indicates the observed values ​​of the horizontal and vertical coordinates of the center point coordinates of the position detection frame in the current state, z w (k),z h (k) represents the observed values ​​of the width and height of the position detection box in the current state.

[0054] Before starting tracking, the initial target state needs to be obtained. The initial target state is obtained by the human position information output by the multi-target recognition unit. In each time step, the target state at the next moment is predicted based on the state estimate of the target at the previous moment and the human position information at the current moment output by the multi-target recognition unit. In the multi-target tracking algorithm, the process noise covariance Q k and the measurement noise covariance matrix R k Calculated according to the following formula:

[0055]

[0056]

[0057] Among them, σ p , σ v and σ m is the noise factor, and is the algorithm's predicted value of the width and height of the position detection box at time k-1, and It is the predicted value of the width and height of the position detection box at time k by the algorithm at time k-1.

[0058] Based on the observation vector and the predicted target state, the target state estimate is updated. The update step uses the observation vector to update the predicted target state with the observation vector z kThe prediction and update steps are repeated to continuously update the target state estimate over time. Each iteration provides a new state estimate, along with the estimated state covariance, which describes the accuracy and uncertainty of the estimate.

[0059] When the parameters of the multi-target recognition and tracking module are updated based on the gradient optimization method, the loss function L is composed of the classification loss function L cls , IoU regression loss function L CloU and the human key point regression loss L kp constitute:

[0060] L=L cls +L CloU +L kp

[0061]

[0062]

[0063]

[0064] Where: Classification loss function L cls is the VFL loss function, q is the IoU of the predicted position detection box and the annotated position detection box (i.e., the intersection of the predicted rectangular box and the annotated rectangular box divided by the union of the two rectangular boxes), p is the predicted value, α and γ are weight coefficients; IoU regression loss function L CloU In, b, b gt Respectively represent the center points of the predicted rectangular box and the annotated rectangular box, ρ represents the Euclidean distance between the two rectangular boxes, c represents the diagonal distance of the closed area of ​​the two rectangular boxes, v is used to measure the consistency of the relative proportions of the two rectangular boxes, and α is the weight coefficient; the human key point regression loss L kp In the above example, M represents the number of key points, x m and m Predict coordinates for key points, and Label the coordinates of key points.

[0065] S04: Input the real-time video stream into the trained multi-target recognition and tracking module to recognize and track each human target and the corresponding human position and human key points in each video frame in real time.

[0066] In the specific implementation, for the video stream sent by the data perception device, video frames are extracted from it at a rate of 5 frames per second, and are input into the multi-target recognition and tracking module in sequence, and the target human body appearing in each video frame and the corresponding human body position and human body key points are output.

[0067] S05: Build a dynamic information database to store multi-time and multi-target human body positions and human body key point information.

[0068] The dynamic information library is used to store the body position and body key point information of all target bodies at multiple times; the dynamic information library uses the globally unique ID number of each target body obtained in the multi-target tracking unit as the primary key, and establishes a dynamic information queue for each target body. After the multi-target recognition and tracking module processes each frame of video, it stores a dynamic information entry in the dynamic information queue corresponding to the target body appearing in the video frame; the dynamic information entry includes the body position label and body key point label of the target body in the video frame, as well as the timestamp of the video frame. Each dynamic information queue stores dynamic information entries of up to 10s. When the timestamp difference between the first and last two dynamic information entries in the dynamic information queue is greater than 10s, the dynamic information entry with the earliest timestamp in the dynamic information queue is deleted.

[0069] S06: Construct a drowning behavior analysis module to analyze the dynamic information database and generate a swimmer calibration image and warning information.

[0070] like Figure 4 As shown in the figure, the drowning behavior analysis algorithm first compares the first and last two dynamic information items in the dynamic information queue of each human target. If the calculated displacement value of the target human body in the two dynamic information items does not exceed 0.2 position detection frames, it is judged that the target human body has no obvious displacement within 10s. At this time, the target human body posture is detected, and the human key point label KP of the target human body is obtained from the latest dynamic information item in the dynamic information queue:

[0071] KP=[(X ls ,Y ls ),(X rs ,Y rs ),(X lh ,Y lh ),(X rh ,Y rh )]

[0072] Where: (X ls ,Y ls ) is the left shoulder coordinate of the target body, (X rs ,Y rs ) is the right shoulder coordinate of the target body, (X lh ,Y lh ) is the left hip coordinate of the target body, (X rh ,Y rh ) is the right hip coordinate of the target body;

[0073] Calculate the direction vector of the target human posture That is, the vector from the midpoint of the left and right shoulders of the target human body to the midpoint of the left and right hips:

[0074]

[0075] Let the ground horizontal vector be Calculate the angle θ between the target human body and the horizontal direction of the ground:

[0076]

[0077] If 0°<θ<60°, the target human body is judged to be in a non-standing state and is at risk of drowning, and a warning message is generated and sent to the front-end alarm.

[0078] S07: Use the Django framework to build a front-end display device, receive the swimmer calibration images sent by the back-end processing platform, and display them on the front-end display device in the form of a video stream.

[0079] The front-end display is built using the Django framework. The back-end processing platform uses the multi-target recognition and tracking module to track and detect the positions of all swimmers in each frame. The detected swimmers are judged to be drowning through the drowning behavior analysis module. The results obtained by the back-end processing platform are transmitted to the front-end display via the WebSocket protocol. The display interface of the front-end display is implemented using technologies such as HTML, CSS and JavaScript, and the swimmer calibration screen is displayed in real time in the form of video streaming.

[0080] S08: The back-end processing platform sends warning information to the front-end alarm.

[0081] The front-end alarm is a smart bracelet, an early warning loudspeaker or an early warning screen. In addition to displaying the information of the swimmers in the pool on the front-end display, it is also necessary to promptly feedback the drowning situation to the lifeguard so that the lifeguard can respond in time. In addition to detecting the location information of all swimmers in the pool, the back-end processing platform will also identify the detected swimmers as drowning. Once it is determined to be drowning, the back-end processing platform will immediately send a drowning alarm and send the location information of the drowning person to the smart bracelet worn by the lifeguard via a Wi-Fi connection. In this way, the lifeguard can receive the drowning alarm and location information in time, so that he can respond quickly and ensure the personal safety of the swimmer.

[0082] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the above embodiments do not limit the present invention in any form, and any technical solution obtained by equivalent replacement or equivalent transformation falls within the protection scope of the present invention.

Claims

1. A drowning behavior analysis method, characterized in that: The steps include: Step 1: Build a dynamic information database to store the human body position and human body key point information of all target human bodies at multiple times; the dynamic information database uses the globally unique ID number of each target human body as the primary key, and establishes a dynamic information queue for each target human body. The dynamic information queue stores the human body position label and human body key point label of the target human body in the order of timestamps; proceed to Step 2; Step 2: Compare the latest dynamic information entry in the dynamic information queue of each human target with the dynamic information entry before the set time length, and calculate the displacement value according to the human position tag: if the displacement value is less than the set displacement value, it is determined that the target human body has not undergone obvious displacement within the set time length, and enter step 3; otherwise, repeat step 2; Step 3: Detect the target human body posture and obtain the target human body key point label KP from the latest dynamic information entry in the dynamic information queue. ls ,Y ls ),(X rs ,Y rs ),(X lh ,Y lh ),(X rh ,Y rh )], where: (X ls ,Y ls ) is the left shoulder coordinate of the target body, (X rs ,Y rs ) is the right shoulder coordinate of the target body, (X lh ,Y lh ) is the left hip coordinate of the target body, (X rh ,Y rh ) is the right hip coordinate of the target human body; go to Step 4; Step 4: Calculate the direction vector of the target human posture That is, the vector from the midpoint of the left and right shoulders of the target human body to the midpoint of the left and right hips; go to Step 5; Step 5: Let the ground horizontal vector be Calculate the angle between the target human body and the horizontal direction of the ground If 0°<θ<60°, the target human body is judged to be in a non-standing state and has a drowning risk; otherwise, return to Step 2.

2. A drowning prevention system based on multi-target tracking and recognition, characterized in that: The invention comprises a back-end processing platform, which first adopts a multi-target recognition algorithm to identify human targets, locate human positions and mark human key points in the video stream of the swimming pool, then adopts a multi-target tracking algorithm to track human targets, human positions and human key points, then establishes a dynamic information library of human targets based on human positions and human key points, and finally adopts the drowning behavior analysis method described in claim 1 to analyze the dynamic information library to generate a swimmer calibration picture and warning information.

3. The anti-drowning system based on multi-target tracking and identification according to claim 2 is characterized in that: The workflow includes the following steps: S01: Process the video stream by framing, mark the target human body, human body position and human body key points in the video frame to form an original frame, pre-process the original frame and construct a training data set; S02: construct a multi-target recognition and tracking module, and update the parameters of the multi-target recognition and tracking module based on a gradient optimization method; The multi-target recognition and tracking module includes a multi-target recognition unit and a multi-target tracking unit; The multi-target recognition unit inputs the video frames in the training data set and outputs the human body positions and human body key point prediction results of the multi-targets; The multi-target tracking unit inputs the human body positions and original frames of multiple targets at multiple times, and matches the trajectory information of each human body target in different video frames; S03: Input the real-time video stream into the trained multi-target recognition and tracking module to recognize and track each human target and the corresponding human position and human key points in each video frame in real time; S04: Build a dynamic information database to store the human body position and key point information of multiple targets at multiple times; S05: Construct a drowning behavior analysis module to analyze the dynamic information database and generate a swimmer calibration image and warning information.

4. The anti-drowning system based on multi-target tracking and identification according to claim 3 is characterized in that: In the step S01, the target human body, human body position and human body key points are marked in the video frame, and the process of constructing the training sample is as follows: first, the video stream is divided into frames, and then each video frame is marked respectively, and the marking tool adopts the open source marking tool coco-annotator; marking the target human body means using a rectangular position detection frame to frame the swimmer in the video frame, the human body position label is composed of the upper left corner coordinates and the lower right corner coordinates of the position detection frame, and the human body key point label is composed of the left shoulder coordinates, right shoulder coordinates, left hip coordinates and right hip coordinates of the swimmer.

5. The anti-drowning system based on multi-target tracking and identification according to claim 3 is characterized in that: In step S02, the multi-target recognition unit is composed of a convolutional neural network based on a feature pyramid, and the third-layer features, fourth-layer features and fifth-layer features output by the feature pyramid are subjected to network fusion encoding to obtain multi-scale fusion features, and the multi-scale fusion features are input into an output head composed of a fully connected layer to output the human body positions and human body key point prediction results of the multiple targets.

6. The anti-drowning system based on multi-target tracking and identification according to claim 3 is characterized in that: In step S02, the multi-target tracking unit associates the position detection frame in the previous video frame with the position detection frame in the current video frame based on the Kalman filtering method, extracts the appearance features of the target body, thereby locating the position of the same target body in consecutive video frames, and establishes a globally unique ID number for each target body to achieve continuous tracking of the target body.

7. The anti-drowning system based on multi-target tracking and identification according to claim 3 is characterized in that: In the step S04, a dynamic information library is constructed to store the human body position and human body key point information of all target human bodies at multiple times; the dynamic information library uses the globally unique ID number of each target human body obtained in the multi-target tracking unit as the primary key, and establishes a dynamic information queue for each target human body. After the multi-target recognition and tracking module processes each frame of video, it stores a dynamic information entry in the dynamic information queue corresponding to the target human body appearing in the video frame; the dynamic information entry includes the human body position label and human body key point label of the target human body in the video frame, and the timestamp of the video frame. Each dynamic information queue stores dynamic information entries of a set time length at most. When the timestamp difference between the first and last two dynamic information entries in the dynamic information queue is greater than the set time length, the dynamic information entry with the earliest timestamp in the dynamic information queue is deleted.

8. The drowning prevention system based on multi-target tracking and identification according to claim 3 is characterized in that: In step S05, the drowning behavior analysis module compares the first and last two dynamic information items in the dynamic information queue of each human target. If it is found that the calculated displacement value of the target human body in the two dynamic information items is less than the set displacement value, it is determined that the target human body has not undergone obvious displacement within the set time length. At this time, the target human body posture is detected, and the human body key point label KP of the target human body is obtained from the latest dynamic information item in the dynamic information queue: KP=[(X ls ,Y ls ),(X rs ,Y rs ),(X lh ,Y lh ),(X rh ,Y rh )] Where: (X ls ,Y ls ) is the left shoulder coordinate of the target body, (X rs ,Y rs ) is the right shoulder coordinate of the target body, (X lh ,Y lh ) is the left hip coordinate of the target body, (X rh ,Y rh ) is the right hip coordinate of the target body; Calculate the direction vector of the target human posture That is, the vector from the midpoint of the left and right shoulders of the target human body to the midpoint of the left and right hips: Let the ground horizontal vector be Calculate the angle θ between the target human body and the horizontal direction of the ground: If 0°<θ<60°, the target person is judged to be in a non-standing state and is at risk of drowning, and a warning message is generated and sent to the front-end alarm.

9. The drowning prevention system based on multi-target tracking and identification according to claim 2, characterized in that: It also includes a front-end display, which uses the WebSocket protocol to receive the swimmer calibration picture sent by the back-end processing platform. The front-end display is designed based on the Django framework and displays the swimmer calibration picture in real time in the form of video stream.

10. The drowning prevention system based on multi-target tracking and identification according to claim 2, characterized in that: It also includes a front-end alarm, which is a smart bracelet, an early warning loudspeaker or an early warning screen. The back-end processing platform sends early warning information to the front-end alarm via a Wi-Fi communication protocol. The early warning information includes a target human body with a drowning risk and its body position.

Images