A method for visual prediction of drowning prevention based on pose estimation

By constructing a visual prediction method for drowning prevention based on posture estimation, and utilizing a lightweight network and an optimized YOLOv7 module, the high labor cost and missed detection problems of drowning detection in swimming pools are solved, achieving efficient and low-cost drowning risk identification.

CN117726988BActive Publication Date: 2026-11-10HEFEI INSTITUTE OF PHYSICAL SCIENCE CHINESE ACADEMY OF SCIENCES
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202311751288.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-12-19
Publication Date
2026-11-10
Estimated Expiration
2043-12-19

AI Technical Summary

Technical Problem

In the current technology, drowning detection in swimming pools relies on manual monitoring, resulting in high labor costs and a high rate of missed detections. There is a lack of effective automated drowning risk identification and early warning systems.

Method used

A visual prediction method for drowning prevention based on attitude estimation is adopted. By constructing a lightweight attitude estimation network and an optimized YOLOv7 module, combined with the lightweight module and an adaptive loss function, the accuracy of swimmer position and motion state detection is improved. Inference is accelerated by using the Ascend Atlas 500 smart station.

Benefits of technology

It achieves low-cost and efficient drowning detection, reduces the incidence of safety accidents, simplifies the dataset creation and annotation process, and improves the recognition rate and model inference speed.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117726988B_ABST
    Figure CN117726988B_ABST
Patent Text Reader

Abstract

The application discloses a kind of based on posture estimation's anti-drowning visual prediction method, comprising:1 data acquisition and processing;2 construct the improved data set light weight network based on posture estimation, for extracting human posture skeleton diagram;3 construct the optimized YOLOV7 inference model prediction network, for multi-scale feature extraction to data and prediction result;4 training model obtains best weight;5 inference acceleration module based on Ascend Atlas 500 intelligent small station is used to improve model inference speed.The application can real-time drowning risk early warning, realize drowning detection, intelligent alarm and other functions, so as to guarantee the life safety of swimmer, also significantly reduce the occurrence of safety accident.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of visual inspection technology and relates to a visual prediction method for drowning prevention based on posture estimation. Background Technology

[0002] Currently, swimming faces significant safety hazards, with frequent drowning accidents. These accidents are caused by unforeseen circumstances and various unpredictable factors, such as accidental bumps, sudden loss of consciousness or coma due to pre-existing medical conditions. Even experienced swimmers are susceptible to drowning due to these factors. A drowning accident not only endangers the life of the victim but also causes economic losses to the swimming pool. Therefore, in the continuous construction of swimming pools, greater emphasis is placed on pool safety.

[0003] Swimming pools have installed numerous cameras, primarily for surveillance purposes. For drowning detection, the venues rely on manual monitoring, but this method has significant drawbacks: the manpower required is prohibitively high, and the workload on monitoring personnel is immense, increasing the risk of missed detections. According to relevant data, to prevent most drowning accidents, it's crucial not only to actively promote drowning prevention education and swimming skills but also to strengthen the safety measures at swimming pools. From a technical perspective, utilizing a fully automated visual recognition and early warning monitoring system for swimming pool drowning risks to enhance venue safety is highly meaningful. Summary of the Invention

[0004] This invention aims to address the shortcomings of existing technologies by proposing a visual recognition method for drowning prevention based on skeleton extraction. The method utilizes a lightweight network-improved dataset and an optimized YOLOv7 module to make the model more accurate in detecting the movement status and position of swimmers in swimming pools, thereby ensuring swimmers' safety and significantly reducing the occurrence of accidents.

[0005] To achieve the above-mentioned objectives, the present invention adopts the following technical solution:

[0006] The present invention provides a visual prediction method for drowning prevention based on attitude estimation, characterized by the following steps:

[0007] Step 1: After acquiring a swimming video dataset and performing frame extraction and preprocessing, the resulting swimming stroke image frame sequence is denoted as F. ori ={F ori,1 ,F ori,2 ,...,F ori,n ,...,F ori,N}, where F ori,n This represents the nth swimming image frame; n = 1, 2, ..., N, where N represents the total number of image frames after frame extraction.

[0008] For F ori,n The position, movement state, and key points of each swimmer's posture are labeled to obtain the nth labeled swimming image frame, denoted as F. n Thus, the labeled swimming stroke image frame sequence F = {F1, F2, ..., F...} is obtained. n ,...,F N}, F n The location, movement status, and key points of the human posture of the m-th swimmer are denoted as follows:

[0009] The motion states of the labeled information are divided into three categories: swimming, standing, and drowning.

[0010] Step 2: Construct a lightweight pose estimation network for F n After processing, the human posture skeleton diagram F is obtained. OP,n ;

[0011] Step 3: Construct the optimized YOLOv7 prediction network for F OP,n Processing yields F n The prediction box and prediction category probability of the m-th swimmer;

[0012] Step 4: Construct the loss function, including: location loss Confidence loss With category loss The lightweight pose estimation network and the optimized YOLOv7 prediction network are trained using gradient descent, and the position loss, confidence loss and class loss are calculated to update the network parameters until the position loss, confidence loss and class loss converge, thereby obtaining the trained and optimized YOLOv7 inference model.

[0013] Step 5: Port the optimized YOLOV7 inference model to the Atlas 500 smart station, and convert the weight file of the optimized YOLOV7 inference model into the om file format adapted to the Atlas 500 smart station. The NPU embedded neural network processor on the Atlas 500 smart station uses the optimized YOLOV7 inference model to predict swimming video data, thereby obtaining the target detection results corresponding to the human posture skeleton map.

[0014] The drowning prevention visual prediction method based on attitude estimation described in this invention is characterized in that the lightweight attitude estimation network in step 2 sequentially includes: a ResNet18 network and T stage modules, wherein each stage module includes: a top branch ρ and a bottom branch ρ.

[0015] Step 2.1: The first few residual layers of different scales in the ResNet18 network are applied to the labeled nth swimming image frame F. n Feature extraction is performed to obtain the feature map F of the key points of the nth human body. feat,n Thus, the feature map set F of key points of the human body is obtained. feat ={F feat,1 ,F feat,2 ,...,F feat,n ,...,F feat,N};

[0016] Step 2.2: When t=1, the top branch ρ in the t-th stage module t With bottom branch Using depthwise convolution and pointwise convolution operations respectively to apply F feat,n After processing, the heatmap of the t-th joint is obtained. affinity domain with the t-th relation

[0017] When t = 2, 3, ..., T, F feat,n Joint heatmap output by the (t-1)th stage module and relationship affinity domain After splicing, it serves as the input to the t-th stage module, and the t-th stage module outputs the heatmap of the t-th joint. Affinity domain with the t-th relation Thus, the heatmap of the Tth joint is output by the Tth stage module. With the Tth relation affinity domain

[0018] F feat,n and and The keypoints in the stitched feature map are paired and connected to obtain the nth human body keypoint feature map F. feat,n The corresponding human skeletal diagram is denoted as F. OP,n Thus, the human posture skeleton diagram sequence F is obtained. OP ={F OP,1 ,F OP,2 ,...,F OP,n ,...,F OP,N}

[0019] The optimized YOLOV7 prediction network in step 2 includes: the feature extraction EfficientNet network, the YOLOV7 neck module, and the head detection head;

[0020] Step 3.1: The feature extraction EfficientNet network consists of Q cascaded CBS basic units and 3 combined modules. Each combined module contains an MPConv module and an improved ELAN module. The q-th CBS basic unit is composed of a convolutional layer, a batch normalization layer, and a SiLU activation layer. The upper branch of the MPConv module includes a max pooling layer (MaxPool) and a CBS module, while the lower branch consists of several CBS modules. The upper branch of the improved ELAN module is a 1×1 convolutional layer, and the lower branch sequentially includes a 1×1 convolutional layer, several CBS modules, and a SAConv module. The SAConv module includes upper, middle, and lower branches.

[0021] F OP,n The input is fed into the EfficientNet feature extraction module and processed sequentially through Q cascaded CBS basic units to obtain the nth coarse feature map S. n,CBS ;

[0022] When j=1, S n,CBS The input is given to the j-th combination module, and after processing by the upper and lower branches of the j-th MPConv module, the two results are concatenated and used as the output of the j-th MPConv module, denoted as .

[0023] The upper branch pair of the j-th improved ELAN module After processing, the upper branch processing result of the j-th improved ELAN module is obtained.

[0024] The 1×1 convolutional layer in the lower branch of the j-th improved ELAN module, and several CBS modules sequentially... After processing, the j-th CBS feature map is obtained.

[0025] Input the j-th SAConv module and use Equation (1) to obtain the j-th SAConv feature map.

[0026]

[0027] In equation (1), w represents the weight parameters to be learned in the convolutional layer, Conv represents the convolution operation, r is the hyperparameter, Δw is the weight to be trained, and G represents the switching function.

[0028] Will and After concatenation, the j-th enhanced feature extraction map output by the j-th improved ELAN module is obtained.

[0029] When j = 2, 3 The j-th combined module is processed to obtain the j-th enhanced feature extraction map. Thus, the output of the feature extraction EfficientNet network is obtained, denoted as... Thus, F is obtained. OP The enhanced feature extraction map set S = {S1, S2, ..., S...} after passing through the EfficientNet network is S = {S1, S2, ..., S...} n ,...,S N};

[0030] Step 3.2: The YOLOv7 neck module includes: a multi-scale spatial pyramid pooling SPPCSPC module, two CBS basic units, a depthwise separable convolutional DSC3 module, and a C3 module. The multi-scale spatial pyramid pooling SPPCSPC module includes upper and lower branches. The upper branch contains several CBS basic units and P parallel pooling layers with different pooling kernels. The lower branch contains several CBS basic units. The DSC3 module contains two structurally identical upper and lower branches. Each branch contains two depthwise separable processing blocks, each of which includes: a depthwise separable convolutional layer, a batch normalization layer, and an activation layer SiLU. The C3 module contains upper and lower branches, each with several CBS basic units.

[0031] Will The input is processed by the SPPCSPC module and then by P parallel pooling layers in the upper branch. The parallel pooling results are concatenated and then processed by several CBS basic units in the upper branch to obtain the upper branch processing result SC of the SPPCSPC module. n,up ;at the same time, After processing by several CBS basic units in the lower branch of the SPPCSPC module, the lower branch processing result SC of the SPPCSPC module is obtained. n,down ;

[0032] SC n,up and SC n,down After stitching, the nth multi-scale spatial pyramid pooling result SC output by the SPPCSPC module is obtained. n Thus, the multi-scale spatial pyramid pooling sequence SC={SC1,SC2,...,SC3} is obtained. n ,...,SC N};

[0033] and SC n The two results obtained after processing by the first CBS basic unit are concatenated and denoted as the fusion feature.

[0034] After processing by the first depth-separable processing block in the upper and lower branches of the DSC3 module, the results from the upper and lower branches are concatenated and then input into the second depth-separable processing block for further processing, resulting in the MidDSC processing result from the DSC3 module. n ;

[0035] and MidDSC n After processing by the second CBS basic unit, the two results are concatenated and denoted as the fusion feature.

[0036] After processing by the first depth-separable processing block in the upper and lower branches of the DSC3 module, the results from the upper and lower branches are concatenated and then input into the second depth-separable processing block for further processing to obtain... Corresponding fused feature results

[0037] After downsampling, compared with MidDSC n The data is then stitched together. The stitched result is processed by the first depth-separable processing block in the upper and lower branches of the DSC3 module. The resulting upper and lower branch results are then stitched together and input into the second depth-separable processing block for further processing. Corresponding fused feature results

[0038] After downsampling, the result is compared with the nth multi-scale spatial pyramid pooling result SC. n The data is then concatenated, and the concatenated result is processed through the upper and lower branches of module C3. The results of the upper and lower branches are then concatenated and used as the final output. Corresponding fused feature results Thus, the fused feature map output by the YOLOv7 neck module is denoted as: This leads to the fused feature map set Neck = {Neck1, Neck2, ..., Neck}. n ,...,Neck N};

[0039] Step 3.3: The Head detection head is used to inspect the Neck. n Processing yields F nPrediction box for the m-th swimmer With predicted category probability in, These represent the coordinates of the top-left and bottom-right corners of the prediction box for the m-th swimmer, respectively, and the prediction category label is...

[0040] The loss function in step 4 is established as follows:

[0041] Step 4.1: Construct F using equation (2) n Position loss of the m-th swimmer

[0042]

[0043] In equation (2), For the true frame of the m-th swimmer The coordinates of the top left and bottom right corners;

[0044] Step 4.2: Construct F using equation (3) n Category loss of the m-th swimmer

[0045]

[0046] In equation (3), F represents n The true category label for the m-th swimmer in the data;

[0047] Step 4.3: Construct F using equation (4) n Confidence loss of the m-th swimmer

[0048]

[0049] In equation (4), F represents n The intersection-union (IoU) ratio of the predicted bounding box to the actual bounding box of the m-th swimmer. and F n The center point ρ of the predicted bounding box and the ground truth bounding box of the m-th swimmer. n,m c represents the Euclidean distance between the center points of the predicted bounding box and the ground truth bounding box. n,m Indicates enclosing F n The diagonal length of the smallest closed rectangle between the predicted bounding box and the true bounding box of the m-th swimmer, v n,m It is F nThe aspect ratio of the predicted bounding box to the true bounding box of the m-th swimmer is obtained by equation (5), and α is a parameter used to balance the aspect ratio column, which is obtained by equation (6).

[0050]

[0051] In equation (5), W n m and F represents n The width and height of the actual frame of the m-th swimmer. and F represents n The width and height of the prediction frame for the m-th swimmer;

[0052]

[0053] The present invention provides an electronic device, including a memory and a processor, wherein the memory is used to store a program that supports the processor in executing the drowning prevention visual prediction method, and the processor is configured to execute the program stored in the memory.

[0054] The present invention discloses a computer-readable storage medium on which a computer program is stored, wherein the computer program, when executed by a processor, performs the steps of the drowning prevention visual prediction method.

[0055] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0056] 1. This invention is simple to operate. For the prediction of drowning in different types of swimming pools, the hardware parameters can be reset, which is convenient to operate and can be applied to the prediction of most drowning. It has the characteristics of simple structure, convenient operation, high recognition rate and easy product update.

[0057] 2. This invention employs an improved lightweight dataset module based on pose estimation methods, which can optimize and improve the dataset, avoiding the tedious problems caused by creating and labeling large datasets, and improving the quality of the dataset.

[0058] 3. This invention employs an optimized YOLOv7 inference model prediction module and uses a few-sample learning method to avoid situations where training is impossible due to an insufficient number of bad samples. Furthermore, it improves the recognition rate of drowning detection by adaptively adjusting the receptive domain and using an improved loss function.

[0059] 4. This invention employs an inference acceleration module based on the Ascend Atlas 500 intelligent station, using the NPU microprocessor in the Atlas 500 intelligent station for optimized YOLOv7 model inference, thereby improving the model's inference speed. Attached Figure Description

[0060] Figure 1 This is a flowchart of a visual prediction method for drowning prevention based on attitude estimation according to the present invention. Detailed Implementation

[0061] In this embodiment, a posture estimation-based visual prediction method for drowning prevention is used to improve the detection accuracy of the YOLOv7 inference model prediction module to detect whether drowning is occurring in a swimming pool. This posture estimation-based visual prediction method for drowning prevention includes the following steps:

[0062] Step 1: After acquiring a swimming video dataset and performing frame extraction and preprocessing, the resulting swimming stroke image frame sequence is denoted as F. ori ={F ori,1 ,F ori,2 ,...,F ori,n ,...,F ori,N}, where F ori,n This represents the nth swimming image frame; n = 1, 2, ..., N, where N represents the total number of image frames after frame extraction.

[0063] Step 1.1: For F ori,n The position, movement state, and key points of each swimmer's posture are labeled to obtain the nth labeled swimming image frame, denoted as F. n Thus, the labeled swimming stroke image frame sequence F = {F1, F2, ..., F...} is obtained. n ,...,F N}, F n The location, movement status, and key points of the human posture of the m-th swimmer are denoted as follows:

[0064] The movement status of the labeled information is divided into three categories: swimming, standing, and drowning.

[0065] Step 2: Construct a lightweight pose estimation network, which includes: a ResNet18 network and T stage modules. Each stage module includes: a top branch ρ and a bottom branch ρ.

[0066] Step 2.1: The first few residual layers of different scales in the ResNet18 network are applied to the labeled nth swimming image frame F. n Feature extraction is performed to obtain the feature map F of the key points of the nth human body. feat,n Thus, the feature map set F of key points of the human body is obtained. feat ={F feat,1 ,F feat,2 ,...,Ffeat,n ,...,F feat,N};

[0067] Step 2.2: When t=1, the top branch ρ in the t-th stage module t With bottom branch Using depthwise convolution and pointwise convolution operations respectively to apply F feat,n After processing, the heatmap of the t-th joint is obtained. Affinity domain with the t-th relation The use of depthwise convolution and pointwise convolution operations reduces the computational cost of convolution in the replaced branch prediction network to only 1 / 9 of the computational cost of the traditional convolution.

[0068] When t = 2, 3, ..., T, F feat,n Joint heatmap output by the (t-1)th stage module and relationship affinity domain After splicing, it serves as the input to the t-th stage module, and the t-th stage module outputs the heatmap of the t-th joint. Affinity domain with the t-th relation Thus, the heatmap of the Tth joint is output by the Tth stage module. With the Tth relation affinity domain Specifically, as shown in equations (1) and (2):

[0069]

[0070]

[0071] F feat,n and and The keypoints in the stitched feature map are paired and connected to obtain the nth human body keypoint feature map F. feat,n The corresponding human skeletal diagram is denoted as F. OP,n Thus, the human posture skeleton diagram sequence F is obtained. OP ={F OP,1 ,F OP,2 ,...,F OP,n ,...,F OP,N}

[0072] Step 3: Construct the optimized YOLOv7 prediction network, including: the feature extraction EfficientNet network, the YOLOv7 neck module, and the head detection head;

[0073] Step 3.1: Feature Extraction. The EfficientNet network consists of Q cascaded CBS basic units and three combined modules. Each combined module contains an MPConv module and an improved ELAN module. The q-th CBS basic unit is composed of a convolutional layer, a batch normalization layer, and a SiLU activation layer. The upper branch of the MPConv module includes a max pooling layer (MaxPool) and a CBS module, while the lower branch consists of several CBS modules. The upper branch of the improved ELAN module is a 1×1 convolutional layer, and the lower branch includes a 1×1 convolutional layer, several CBS modules, and one SAConv module. The SAConv module has three branches: upper, middle, and lower. The SAConv module can adaptively adjust the receptive field according to changes in the object's scale and pose.

[0074] Step 3.1.1: F OP,n The input is fed into the EfficientNet feature extraction module and processed sequentially through Q cascaded CBS basic units to obtain the nth coarse feature map S. n,CBS ;

[0075] Step 3.1.2: When j = 1, S n,CBS The input is given to the j-th combination module, and after processing by the upper and lower branches of the j-th MPConv module, the two results are concatenated and used as the output of the j-th MPConv module, denoted as .

[0076] The upper branch pair of the j-th improved ELAN module After processing, the upper branch processing result of the j-th improved ELAN module is obtained.

[0077] The 1×1 convolutional layer in the lower branch of the j-th improved ELAN module, and several CBS modules sequentially... After processing, the j-th CBS feature map is obtained.

[0078] Input the j-th SAConv module and use Equation (3) to obtain the j-th SAConv feature map.

[0079]

[0080] In equation (3), w represents the weight parameters to be learned in the convolutional layer, Conv represents the convolution operation, r is the hyperparameter, Δw is the weight to be trained, and G represents the switching function, which can adaptively adjust the switch to handle different objects.

[0081] Will and After concatenation, the j-th enhanced feature extraction map output by the j-th improved ELAN module is obtained.

[0082] When j = 2, 3 The j-th combined module is processed to obtain the j-th enhanced feature extraction map. Thus, the output of the feature extraction EfficientNet network is obtained, denoted as... Thus, F is obtained. OP The enhanced feature extraction map set S = {S1, S2, ..., S...} after passing through the EfficientNet network is S = {S1, S2, ..., S...} n ,...,S N}

[0083] Step 3.2: The YOLOv7 neck module includes: a multi-scale spatial pyramid pooling SPPCSPC module, two CBS basic units, a depthwise separable convolutional DSC3 module, and a C3 module. The multi-scale spatial pyramid pooling SPPCSPC module contains upper and lower branches. The upper branch contains several CBS basic units and P parallel pooling layers with different pooling kernels. The lower branch contains several CBS basic units. The DSC3 module contains two structurally identical upper and lower branches. Each branch contains two depthwise separable processing blocks. Each depthwise separable processing block contains a depthwise separable convolutional layer, a batch normalization layer, and an activation layer SiLU. The C3 module contains upper and lower branches, each with several CBS basic units.

[0084] Step 3.2.1: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation would require the full context.] The input is processed by the SPPCSPC module and then by P parallel pooling layers in the upper branch. The parallel pooling results are concatenated and then processed by several CBS basic units in the upper branch to obtain the upper branch processing result SC of the SPPCSPC module. n,up ;at the same time, After processing by several CBS basic units in the lower branch of the SPPCSPC module, the lower branch processing result SC of the SPPCSPC module is obtained. n,down ;

[0085] SC n,up and SC n,down After stitching, the nth multi-scale spatial pyramid pooling result SC output by the SPPCSPC module is obtained. n Thus, the multi-scale spatial pyramid pooling sequence SC={SC1,SC2,...,SC3} is obtained. n ,...,SCN The SPPC module can obtain different receptive field results for the same feature map.

[0086] Step 3.2.2: and SC n The two results obtained after processing by the first CBS basic unit are concatenated and denoted as the fusion feature.

[0087] After processing by the first depth-separable processing block in the upper and lower branches of the DSC3 module, the results from the upper and lower branches are concatenated and then input into the second depth-separable processing block for further processing, resulting in the MidDSC processing result from the DSC3 module. n ;

[0088] and MidDSC n After processing by the second CBS basic unit, the two results are concatenated and denoted as the fusion feature.

[0089] After processing by the first depth-separable processing block in the upper and lower branches of the DSC3 module, the results from the upper and lower branches are concatenated and then input into the second depth-separable processing block for further processing to obtain... Corresponding fused feature results

[0090] Step 3.2.3: After downsampling, compared with MidDSC n The data is then stitched together. The stitched result is processed by the first depth-separable processing block in the upper and lower branches of the DSC3 module. The resulting upper and lower branch results are then stitched together and input into the second depth-separable processing block for further processing. Corresponding fused feature results

[0091] After downsampling, the result is compared with the nth multi-scale spatial pyramid pooling result SC. n The data is then concatenated, and the concatenated result is processed through the upper and lower branches of module C3. The results of the upper and lower branches are then concatenated and used as the final output. Corresponding fused feature results Thus, the fused feature map output by the YOLOv7 neck module is denoted as: This leads to the fused feature map set Neck = {Neck1, Neck2, ..., Neck}. n ,...,NeckN}

[0092] Step 3.3: Head detection head to Neck n Processing yields F n Prediction box for the m-th swimmer With predicted category probability in, These represent the coordinates of the top-left and bottom-right corners of the prediction box for the m-th swimmer, respectively, and the prediction category label is...

[0093] Step 4: Construct the loss function, including location loss, confidence loss, and class loss;

[0094] Step 4.1: Construct F using equation (4) n Position loss of the m-th swimmer

[0095]

[0096] In equation (4), For the true frame of the m-th swimmer The coordinates of the top left and bottom right corners.

[0097] Step 4.2: Construct F using equation (5) n Category loss of the m-th swimmer

[0098]

[0099] In equation (5), F represents n The true category label of the m-th swimmer.

[0100] Step 4.3: Construct F using equation (6) n Confidence loss of the m-th swimmer

[0101]

[0102] In equation (6), F represents n The intersection-union (IoU) ratio of the predicted bounding box to the actual bounding box of the m-th swimmer. and F n The center point ρ of the predicted bounding box and the ground truth bounding box of the m-th swimmer. n,m c represents the Euclidean distance between the center points of the predicted bounding box and the ground truth bounding box. n,m Indicates enclosing F nThe diagonal length of the smallest closed rectangle between the predicted bounding box and the true bounding box of the m-th swimmer, v n,m It is F n The aspect ratio of the predicted bounding box of the m-th swimmer to the actual bounding box is obtained by equation (7), and α is a parameter used to balance the aspect ratio column, which is obtained by equation (8).

[0103]

[0104] In equation (7), W n m and F represents n The width and height of the actual frame of the m-th swimmer. and F represents n The width and height of the prediction frame for the m-th swimmer.

[0105]

[0106] Loss of the position of the nth swimming stroke image n,pos Category Loss n,label Loss on confidence n,CIoU Specifically, as shown in equations (9), (10), and (11):

[0107]

[0108] In equation (9), M represents the total number of people in the nth swimming stroke image;

[0109]

[0110]

[0111] The three losses for the nth swimming stroke image are weighted and summed to obtain the total loss (Loss) of the nth swimming stroke image after optimization of the YOLOv7 model. n Specifically, as shown in equation (12):

[0112] Loss n =λ1Loss n,pos +λ2Loss n,label +λ3Loss n,CIoU (12)

[0113] Wherein, λ1, λ2, and λ3 are balance coefficients.

[0114] Step 5: Port the optimized YOLOv7 inference model to the Atlas 500 smart station. The smart station can accelerate processing by combining the AclLite module and utilizing DVPP hardware capabilities. Convert the weight file of the optimized YOLOv7 inference model to the .om file format adapted to the Atlas 500 smart station. The NPU embedded neural network processor on the Atlas 500 smart station uses the optimized YOLOv7 inference model to predict swimming video data, thereby obtaining the target detection results corresponding to the human posture skeleton map.

[0115] In this embodiment, an electronic device includes a memory and a processor. The memory stores a program that supports the processor in executing the above-described method, and the processor is configured to execute the program stored in the memory.

[0116] In this embodiment, a computer-readable storage medium stores a computer program, which is executed by a processor to perform the steps of the above method.

Claims

1. A visual prediction method for drowning prevention based on attitude estimation, characterized in that, Includes the following steps: Step 1: After acquiring a swimming video dataset and performing frame extraction and preprocessing, the resulting swimming stroke image frame sequence is denoted as... ,in, Indicates the first One swimming image frame; , This indicates the total number of image frames after frame extraction; right The position, movement status, and key posture points of each swimmer are marked to obtain the marked number. A swimming image frame, denoted as Thus, the labeled swimming stroke image frame sequence is obtained. ,Will The Middle The location, movement status, and key points of the human posture of each swimmer are denoted as follows: ; The motion states of the labeled information are divided into three categories: swimming, standing, and drowning. Step 2: Construct a lightweight pose estimation network pair The process yields a human skeletal diagram. ; Step 3: Construct the optimized YOLOv7 prediction network. Processing is performed to obtain The Middle swimmers The predicted bounding box and the predicted class probability; The optimized YOLOV7 prediction network includes: an EfficientNet feature extraction network, a YOLOV7 neck module, and a head detection head; Step 3.1: The feature extraction EfficientNet network consists of sequentially connected... There are one CBS basic unit and three combined modules. Each combined module contains one MPConv module and one improved ELAN module. Any of the following... Each CBS basic unit consists of a convolutional layer, a batch normalization layer, and a SiLU activation layer in sequence; the upper branch of the MPConv module includes a max pooling layer MaxPool and a CBS module, and the lower branch of the MPConv module consists of several CBS modules; the upper branch of the improved ELAN module is... The convolutional layer, and the lower branches of the improved ELAN module, include the following in sequence: The system comprises a convolutional layer, several CBS modules, and one SAConv module; wherein the SAConv module includes three branches: upper, middle, and lower. Will The input is fed into the feature extraction module of the EfficientNet network, and then passes through... After feature extraction processing of the cascaded CBS basic units, the nth coarse feature map is obtained. ; when hour, Enter the first In the combined module, and respectively through the first... After processing the upper and lower branches of the MPConv module, the two results are concatenated and used as the first result. The output of each MPConv module is denoted as... ; No. An improved ELAN module's upper branch pair After processing, the first result is obtained. The results of the branch processing of an improved ELAN module ; No. In the lower branch of an improved ELAN module Convolutional layers, several CBS modules sequentially After processing, the first result is obtained. CBS feature map ; Enter the first In the SAConv module, and using equation (1) to obtain the first... SAConv feature map : (1) In equation (1), This represents the weight parameters to be learned in the convolutional layer. This represents the convolution operation. For hyperparameters, The weights to be trained. Represents the switching function; Will and After splicing, we get the first... The first improved ELAN module output Enhanced feature extraction map ; when hour, Enter the first The process is carried out in the first combination module to obtain the first... Enhanced feature extraction map Thus, the output of the feature extraction EfficientNet network is obtained, denoted as... And thus obtain Enhanced feature extraction map set after passing through EfficientNet network ; Step 3.2: The YOLOv7 neck module includes: a multi-scale spatial pyramid pooling SPPCSPC module, two CBS basic units, a depthwise separable convolutional DSC3 module, and a C3 module. The multi-scale spatial pyramid pooling SPPCSPC module contains upper and lower branches. The upper branch contains several CBS basic units and different pooling kernels. The DSC3 module contains two identical upper and lower branches and a shared depthwise separable processing block. Each branch contains a depthwise separable processing block, which in turn includes: a depthwise separable convolutional layer, a batch normalization layer, and an activation layer SiLU. The C3 module contains upper and lower branches, each of which contains several CBS basic units. Will Input into the SPPCSPC module and pass through the previous branch. After processing by each parallel pooling layer, the parallel pooling results are concatenated and then input into several CBS basic units of the upper branch for processing to obtain the upper branch processing result of the SPPCSPC module. ;at the same time, After processing by several CBS basic units in the lower branch of the SPPCSPC module, the processing result of the lower branch of the SPPCSPC module is obtained. ; and After stitching, the nth multi-scale spatial pyramid pooling result output by the SPPCSPC module is obtained. Thus, a multi-scale spatial pyramid pooling sequence is obtained. ; and The two results obtained after processing by the first CBS basic unit are concatenated and denoted as the fusion feature. ; After processing by the first depth-separable processing block in the upper and lower branches of the DSC3 module, the results from the upper and lower branches are concatenated and then input into the second depth-separable processing block for further processing to obtain the final processing result of the DSC3 module. ; and After processing by the second CBS basic unit, the two results are concatenated and denoted as the fusion feature. ; After processing by the first depth-separable processing block in the upper and lower branches of the DSC3 module, the results from the upper and lower branches are concatenated and then input into the second depth-separable processing block for further processing to obtain... Corresponding fused feature results ; After downsampling, and The data is then stitched together. The stitched result is processed by the first depth-separable processing block in the upper and lower branches of the DSC3 module. The resulting upper and lower branch results are then stitched together and input into the second depth-separable processing block for further processing. Corresponding fused feature results ; After downsampling, the result is compared with the nth multi-scale spatial pyramid pooling result. The data is then concatenated, and the concatenated result is processed through the upper and lower branches of module C3. The results of the upper and lower branches are then concatenated and used as the final output. Corresponding fused feature results Thus, the fused feature map output by the YOLOv7 neck module is denoted as... This leads to the fusion feature map set. ; Step 3.3: The Head detection head pair Processing is performed to obtain The Middle swimmers Prediction box With predicted category probability ,in, , They represent the first The coordinates of the top left and bottom right corners of the prediction box for each swimmer, and the predicted category label are: ; Step 4: Construct the loss function, including: location loss Confidence loss With category loss The lightweight pose estimation network and the optimized YOLOv7 prediction network are trained using gradient descent, and the position loss, confidence loss and class loss are calculated to update the network parameters until the position loss, confidence loss and class loss converge, thereby obtaining the trained and optimized YOLOv7 inference model. Step 5: Port the optimized YOLOV7 inference model to the Atlas 500 smart station, and convert the weight file of the optimized YOLOV7 inference model into the om file format adapted to the Atlas 500 smart station. The NPU embedded neural network processor on the Atlas 500 smart station uses the optimized YOLOV7 inference model to predict swimming video data, thereby obtaining the target detection results corresponding to the human posture skeleton map.

2. The drowning prevention visual prediction method based on attitude estimation according to claim 1, characterized in that, The lightweight pose estimation network in step 2 includes, in sequence: a ResNet18 network, ... There are several stage modules, each of which includes: a top branch. With bottom branch ; Step 2.1: The first few residual layers of different scales in the ResNet18 network are compared with the labeled first... A swimming image frame Feature extraction is performed to obtain the first... Feature map of key points of an individual This allows us to obtain a feature set of key points on the human body. combine ; Step 2.2: When At that time, the first Top branch in each stage module With bottom branch Using depthwise convolution and pointwise convolution operations respectively Processing is performed to obtain the corresponding result. Joint heatmap With the Affinity domain ; when At that time, With the The joint heatmap output by each stage module and relationship affinity domain After splicing, as the first The input of the stage module, and by the first stage module The output of the stage module is the first one. Joint heatmap With the Affinity domain ; and thus by the first The output of the stage module is the first one. Joint heatmap With the Affinity domain ; Will and and The keypoints in the spliced ​​feature map are paired and connected to obtain the first keypoint. Personal body key feature map The corresponding human skeletal diagram is denoted as Thus, a sequence of human posture skeleton diagrams is obtained. .

3. The drowning prevention visual prediction method based on attitude estimation according to claim 2, characterized in that, The loss function in step 4 is established as follows: Step 4.1: Construct using equation (2) The Middle The position loss of a swimmer : (2) In equation (2), , For the first A real frame of a swimmer The coordinates of the top left and bottom right corners; Step 4.2: Construct using equation (3) The Middle Category of losses for individual swimmers : (3) In equation (3), express The Middle The actual category label for each swimmer; Step 4.3: Construct using equation (4) The Middle Confidence loss of individual swimmers : (4) In equation (4), express The Middle The intersection-union (IoU) of the predicted bounding boxes and the actual bounding boxes for each swimmer. and They represent The Middle The center point of the predicted bounding box and the corresponding ground truth bounding box for each swimmer. This represents the Euclidean distance between the center points of the predicted bounding box and the ground truth bounding box. Indicates encirclement The Middle The diagonal length of the smallest closed rectangle between the predicted bounding box and the true bounding box of a swimmer. yes The Middle The aspect ratio of the predicted bounding box to the actual bounding box of each swimmer is obtained by equation (5). It is a parameter used to balance the aspect ratio, and is obtained from equation (6); (5) In equation (5), and express The Middle The width and height of a swimmer's actual frame. and express The Middle The width and height of the prediction frame for each swimmer; (6)。 4. An electronic device, comprising a memory and a processor, characterized in that, The memory is used to store a program that supports the processor in executing any of the drowning prevention visual prediction methods according to claims 1-3, and the processor is configured to execute the program stored in the memory.

5. A computer-readable storage medium storing a computer program thereon, characterized in that, The computer program, when run by a processor, performs the steps of any of the drowning prevention visual prediction methods described in claims 1-3.