Multi-scale drowning target detection method based on improved YOLOv7

Abstract

The invention provides a multi-scale drowning target detection method based on improved YOLOv7, and relates to the technical field of drowning detection. The multi-scale drowning target detection method based on the improved YOLOv7 specifically comprises the following steps: S1, input end data processing; s2, carrying out backbone network and feature fusion; s3, training and performance optimization; and S4, system integration and practical application. According to the invention, through Mosaic data enhancement, adaptive anchor point calculation and image scaling, the detection effect of a small target is improved, the adaptability of the model is enhanced, the feature pyramid neural network fusing the attention mechanism can more effectively process the all-weather scene change and shielding problems, the overall detection precision is improved, and the method is suitable for large-scale popularization and application. Through fusion of feature maps of different scales, in combination with up-sampling, down-sampling and an attention mechanism, the adaptability of the algorithm to global information is enhanced, and important features can be effectively learned and highlighted.

Description

Technical Field

[0001] The present invention relates to the technical field of drowning detection, and in particular to a multi-scale drowning target detection method based on improved YOLOv7. Background Art

[0002] According to the World Health Organization (WHO), drowning is the third leading cause of accidental injury worldwide, with 320,000 people drowning each year, accounting for 7% of all injury-related deaths [1]. Drowning can occur in a variety of environments, especially in swimming pools, reservoirs, and large water play areas. Currently, the traditional detection and rescue of drowning in seaside bathing beaches, swimming pools, and water play areas in some scenic spots still relies heavily on the human eye of lifeguards. Lifeguards rely solely on human eyes for observation, lacking more effective auxiliary equipment to help them overcome the decline in observation effect caused by long-term high concentration and empiricism. Detecting and identifying different movements of the target human body through artificial intelligence technology is an emerging and promising research direction in the field of computer vision. Applying artificial intelligence technology to the field of drowning detection will significantly outperform traditional methods that rely on human eye observation by lifeguards in terms of detection efficiency, cost, and scope of use.

[0003] Vision-based systems and wearable sensor-based systems are existing drowning detection technologies. Al shbatat et al. proposed a vision-based integrated monitoring system consisting of a Raspberry Pi, two Pixy cameras, and an Arduino Nano board. They used two cameras to detect and monitor swimmers by measuring their positions, and swimmers had to wear passive yellow vests. NEPTUNE is another unique technology that uses statistical image processing of video sequences to detect drowning people as quickly as possible. The equations used to detect approaching drowning people are based on variables generated by statistical image processing. Another system called VI BE uses background extraction to detect and track drowning people and uses the VIBE algorithm to obtain frame differences to update the motion region, which mainly evaluates the position of the swimmer when making judgments. How-Lung et al. studied some of the difficulties in finding drowning people in water environments and provided an automatic detection monitoring system. According to the authors, the main obstacles in aquatic environments are ripples and splashes of water, as well as background motion in reflective areas. Occlusion is also considered a challenging difficulty when it comes to identifying swimmers. The solution they proposed is an algorithm that takes all these issues into account and detects water crises in complex aquatic environments. Upulie Handalage et al. proposed a drowning detection system, which mainly includes three functions: detecting drowning people, sending drones to victims, and detecting dangerous activities. The drowning detection component detects drowning victims through a custom CNN model, which detects drowning in three stages and immediately notifies the user through an audio alarm. The second component is a rescue drone, which is launched and sent to the victim's location coordinates according to the drowning detection command. This program uses a custom x and y coordinate block system to link to the ground GPS coordinates. At the same time, potential dangerous activities, including running around the swimming pool and drinking, will be notified to the on-duty personnel in the venue through the mobile alarm of the danger detection component. This will prompt the on-duty personnel (including lifeguards) to make responsible decisions. The above-mentioned existing technologies, such as ripples, splashes and background motion in the reflective area of ​​the water, may interfere with the accuracy of the visual system, such as having to wear a passive yellow vest, which limits the universality and practicality of the system. Occlusion (such as blocking each other when multiple people swim) is a challenge when identifying swimmers, which affects the efficiency of the detection system.

[0004] Therefore, the present invention proposes a multi-scale drowning target detection method based on improved YOLOv7 to solve the problems raised by the background technology. Summary of the invention

[0005] 1. Technical issues to be resolved

[0006] In view of the shortcomings of the prior art, the present invention provides a multi-scale drowning target detection method based on improved YOLOv7, which solves the problems of low detection efficiency and detection accuracy of the existing detection methods.

[0007] (II) Technical solution

[0008] To achieve the above objectives, the present invention is implemented by the following technical scheme: a multi-scale drowning target detection method based on improved YOLOv7 specifically includes the following steps:

[0009] S1. Input data processing

[0010] Perform Mosaic data enhancement, adaptive anchor point calculation and image scaling;

[0011] S2. Backbone network and feature fusion

[0012] Set up a backbone network based on YOLOv7 and use the feature pyramid network and attention mechanism to integrate it;

[0013] S3. Training and performance optimization

[0014] Customize CNN model training with real-time feedback and parameter tuning;

[0015] S4. System integration and practical application

[0016] Deploy drowning detection components, integrate drone systems, and use mobile alarms to detect dangerous activities around swimming pools, such as running and drinking, and promptly notify on-duty personnel within the venue.

[0017] Preferably, the oMosaic data enhancement in the input data processing of step S1 is to enhance the input data using Mosaic technology, the purpose of which is to improve the network's detection effect on small targets by synthesizing parts of multiple pictures, and to enhance the model's adaptability to various scene changes.

[0018] Preferably, the adaptive anchor point calculation and image scaling in the input data processing of step S1 are to introduce adaptive anchor point and image scaling function blocks, which helps to adjust the anchor point size and image scaling ratio of the detection algorithm according to targets of different scales, thereby improving detection accuracy.

[0019] Preferably, the backbone network based on YOLOv7 in the backbone network and feature fusion in step S2 is an efficient backbone network structure that follows YOLOv7, which has been proven to have a good balance between speed and accuracy.

[0020] Preferably, the feature pyramid network and attention mechanism in the backbone network and feature fusion of step S2 are a combination of feature pyramid network (FPN) and attention mechanism, which fuses feature maps of different depths to enhance the model's adaptability and detection accuracy to occlusion and all-weather changing scenes, and uses the attention mechanism to enhance important features and suppress unimportant information, thereby improving detection accuracy.

[0021] Preferably, the training of the customized CNN model in step S3 training and performance optimization is to train the customized CNN model on specific data sets, and these data sets include drowning posture images taken in swimming pool and beach scenes.

[0022] Preferably, the real-time feedback and parameter tuning in step S3 training and performance optimization is to adjust the parameters in the training process by real-time monitoring of changes in the loss function (Loss), so as to ensure that the model can converge quickly and achieve high accuracy.

[0023] Preferably, the system integration and practical application of step S4 includes deploying the trained model as a drowning detection component, which can detect in real time and notify relevant personnel through audio alarms, and combining with the drone system to automatically dispatch drones to designated coordinates for rescue based on detected drowning events, and using mobile alarms to detect dangerous activities around the swimming pool, such as running and drinking, and promptly notify on-duty personnel in the venue.

[0024] (III) Beneficial effects

[0025] The present invention provides a multi-scale drowning target detection method based on improved YOLOv7.

[0026] Beneficial effects:

[0027] The invention provides a multi-scale drowning target detection method based on improved YOLOv7. The method designs a feature pyramid neural network integrating an attention mechanism. This structure enables the algorithm to process targets of different scales, especially in complex aquatic environments, which is particularly important for identifying drowning people at a long distance or partially occluded. By using Mosaic data enhancement and adaptive anchor point calculation, as well as functional blocks of adaptive image scaling, the algorithm can improve the detection effect of small targets, which is crucial for detecting drowning incidents that may occur in swimming pools or beaches. Especially in outdoor environments with changing conditions, the algorithm can effectively cope with scene changes and occlusion detection accuracy problems in all-weather time periods. The introduction of the attention mechanism helps the model focus on key features and ignore irrelevant background interference, such as water surface reflection and splashing. The invented system can not only detect drowning postures, but also remind lifeguards to pay attention in real time through voice. Such a design makes up for the shortcoming that human eyes cannot achieve 24-hour continuous high-precision observation, and improves the timeliness of rescue. Through these designs, the improved YOLOv7 algorithm can effectively improve the accuracy and response speed of drowning monitoring, which is of great significance for rescuers to promptly discover and deal with potential drowning accidents. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a structural diagram of the improved yolov7 algorithm of the present invention;

[0029] Figure 2 This is a detailed image of the UF upsampling feature layer of the present invention;

[0030] Figure 3 It is the detail image of the DF downsampling feature layer of the present invention;

[0031] Figure 4 A schematic diagram of deep learning and computer vision of the present invention;

[0032] Figure 5 This is an excerpt from the data set taken at the swimming pool of the present invention;

[0033] Figure 6 An excerpt from the dataset taken for the beach of the present invention;

[0034] Figure 7 It is a curve chart of the experimental results analysis of the present invention;

[0035] Figure 8 It is the training feedback parameter diagram a of the present invention;

[0036] Fig. 9 This is the training feedback parameter diagram b of the present invention;

[0037] Fig.10 This is the training feedback parameter diagram c of the present invention;

[0038] Fig.11 This is a test result diagram of the present invention. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] Example:

[0041] like Figure 1-11 As shown in FIG. 1 , the multi-scale drowning target detection method based on the improved YOLOv7 specifically includes the following steps:

[0042] S1. Input data processing

[0043] Perform Mosaic data enhancement, adaptive anchor point calculation, and image scaling;

[0044] S2. Backbone network and feature fusion

[0045] Set up a backbone network based on YOLOv7 and use the feature pyramid network and attention mechanism to integrate it;

[0046] S3. Training and performance optimization

[0047] Customize CNN model training with real-time feedback and parameter tuning;

[0048] S4. System integration and practical application

[0049] Deploy drowning detection components, integrate drone systems, and use mobile alarms to detect dangerous activities around swimming pools, such as running and drinking, and promptly notify on-duty personnel within the venue.

[0050] Based on the above method steps, the details are as follows:

[0051] Fast drowning posture detection system based on YOLOv7

[0052] In existing research, target detection methods are mainly divided into one-stage detection algorithms and two-stage detection algorithms. Speed ​​is undoubtedly the most important thing for dangerous scenarios such as drowning. It is necessary to achieve high accuracy while being fast. The neural network model used has high requirements in terms of parameter quantity and processing time. Therefore, it is necessary to use a network structure such as YOLO that can combine the two types of tasks of "search-recognition" into one. At the same time, the accuracy must also be guaranteed to a certain extent.

[0053] Based on YOLOv7, the present invention designs a multi-scale target detection algorithm based on a feature pyramid neural network with a fusion attention mechanism. I have fused the feature maps of all the output scales of the detector. In this way, the deep semantic feature information and the shallow fine-grained information can be fully integrated. At the same time, in order to fuse features of different scales, the present invention first uses upsampling layers and downsampling layers to unify the scales of feature maps of different scales, and then uses a series operation to fuse feature layers of different scales. Finally, in order to enable the neural network to automatically learn the correlation and importance between feature map channels, an attention mechanism is introduced in the feature fusion process to control the contribution of feature maps of different channels when learning weights. Finally, a 1×1 convolution is used to output the detection result. The improved network structure of the present invention is as follows: Figure 1 shown.

[0054] The main design ideas and working steps of the improved feature pyramid neural network with integrated attention mechanism of the present invention are as follows:

[0055] (1) Input: This part of the input mainly performs Mosaic data enhancement on the input data, the main purpose of which is to improve the detection effect of small targets. In addition, the present invention also designs and adds functional blocks for adaptive anchor point (AutoAnchor) calculation and adaptive image scaling.

[0056] (2) Backbone network,The backbone network of YOLOv5 mainly consists of four modules,,specifically: Focus, CSP1_X, CBL, and SPP.

[0057] (3) Feature pyramid network integrating attention mechanism (Cross layer integrating attention mechanism) - Considering that the information around the drowning person is also useful, for example, the same action of slapping the water surface with the wrist is more likely to be swimming in shallow water than in deep water. A swimmer making similar actions in deep water means that he is in danger and struggling for help rather than swimming. Therefore, the information around the detected target is also an important part of neural network learning. If other CNN-based target detection algorithms are used, the current common feature fusion method is to concatenate features of different scales in the channel dimension. This method cannot reflect the correlation and importance of features between different channels.

[0058] like Figure 1 The improved yolov7 algorithm structure diagram is shown.

[0059] Inspired by the up-down net, the present invention fuses the feature maps of all output scales in the network. In order to fuse features of different scales, the number of channels of feature maps of different dimensions extracted by the backbone network is first unified to 256 using 1x1 convolution.

[0060] like Figure 2 The UF upsampling feature layer details shown

[0061] T i =Conv(C i +Upsample(T i+1 )) (1.1)

[0062] C i+1 Upsampling is performed using bilinear nearest neighbor upsampling, followed by a 1x1 convolution operation to reduce the blur caused by the upsampling, and then combined with C i Add pixel-by-pixel points at the pixel level, and then pass through a 1x1 convolution kernel to obtain the output feature T i , T i The calculation formula is as follows. The details of the UF upsampling feature layer are as follows Figure 2 shown.

[0063] In the DF module, the feature layer T with shallow texture information i-1 Upsample and combine with the deep feature layer T i Pixel summation is performed to make the image have a richer multi-scale feature map, and then a 1x1 convolution operation is performed. The DF downsampling feature layer calculation method is as follows: Figure 3 Detailed image of the DF downsampling feature layer shown.

[0064] Images with higher resolution tend to contain more fine-grained features, while images with lower resolution contain richer semantic information. Therefore, by fusing different feature layers, the accuracy of target detection can be improved.

[0065] (4) Prediction output - This part of the invention basically retains the neck network in the original YOLOv7 network. I added the FPN (Feature Pyramid Networks) upsampling structure to the original neck network, generated a tensor with a dimension of 76×76×255, and then performed two downsamplings to obtain tensors with dimensions of 38×38×255 and 19×19×255 respectively. This is mainly done to solve the multi-scale problem in target detection. Then calculate the Softmax Loss (classification loss function) and CIOU Loss (boundary regression loss function) under multiple dimensions, obtain the prediction results and output them.

[0066] After the image passes through the network, some object features will be lost due to the existence of the pooling layer. Considering that after the information is fused across layers, the fusion of semantic features and position information enables the feature map to obtain more information, but it cannot reflect the relevance and importance of the feature map space and different dimensions. Inspired by CBAM, the present invention adds a convolutional attention mechanism to each fusion feature layer to enhance the algorithm's adaptability to the global information of the image. Spatial attention mechanism such as Figure 4 Shown is CBAM integrated with a YOLOBlock in YOLO.

[0067] Channel attention uses the relationship between feature map channels and uses global average pooling and maximum pooling to aggregate feature map spatial information to reduce information loss. The generated channel attention map focuses on extracting the contour information features of the detection target. The calculation formula of the channel attention map is as follows:

[0068]

[0069] After the channel attention module, the spatial attention module is connected. The input of the spatial attention module is obtained by multiplying the attention feature map with the initial input. The module aggregates the position information through convolution operation and also uses two pooling methods, global average pooling and maximum pooling. After splicing, it is connected by convolution to generate the two-dimensional spatial attention of the present invention. Finally, it is multiplied with the channel attention feature map to obtain a new feature map.

[0070]

[0071] represents the sigmoid function, f 7×7 represents the convolution operation with a filter size of 7×7; F s avg is the average pooling feature and F s max is the maximum pooling feature.

[0072] Constructing a dataset of common drowning help postures

[0073] Due to privacy concerns and insufficient academic attention to this field, there are currently very few datasets on human drowning postures on the Internet. Therefore, under the guidance of a swimming coach in a scenic area, the present invention simulated and photographed a set of drowning posture datasets. The present invention took a large number of photos in the swimming pool and the beach for dataset establishment. The dataset taken in the swimming pool is excerpted as follows: Figure 5 , an excerpt from the beach photography dataset is as follows Figure 6 .

[0074] Table 1. Target objects and corresponding labels of constructed datasets

[0075] Pool Side Label Beach Label struggle Struggle Struggling state 1 Warning01 cramp Warning Struggling State 2 Warning02 Sink Underwater Drowning Struggle Drowned Rescued and safe Safe Drowning Underwater

[0076] When constructing the dataset, the required images were mainly sourced from selfies. It was planned to collect about 300 images of each type. After taking a total of about 2,400 images for the entire dataset, the dataset was expanded using the metadata enhancement method, including adjusting image exposure, scaling, translation, random cropping, random combination and other image processing methods. Finally, a total of 8 category labels were produced.

[0077] Experimental results analysis

[0078] During the training process, the dynamic process of training can be intuitively observed by drawing the loss curve. Figure 7 It is the corresponding Loss curve during the training process. The horizontal axis represents the number of training iterations, and the vertical axis represents the Loss value during the training process. The Loss value decreases relatively quickly during the training process, and the curve stabilizes after 1000 iterations of training. When the iterative training is performed 2000 times, the curve converges to 0.23, and the training ends.

[0079] In order to enable this method to better solve the drowning safety problem caused by poor detection accuracy due to scene changes and occlusion in all weather conditions, and thus improve the drowning detection effect, the present invention conducts comparative experiments on multiple detection methods, such as Figure 7 shown.

[0080] Table 2 is a comparison of the performance of the method of the present invention and other detection algorithms on a data set simulating drowning. Precision accuracy. Note: This section needs to be supplemented with analysis experiments.

[0081] With 199 rounds of training, the loss function decreases from 0.058 to0.002 and the val classification decreases from 0.057 to 0.002, a continuous decrease in the value of the loss function indicates that the training process is better surface does not appear overfitting condition.

[0082] Table 2 Comparison of the performance of multiple detection methods

[0083] Ethod of detection Precision AP% VFPS / (frame·s-1) YOLOv5 YOLOv7 72.3％ FasterRCNN Articlemethod 83.6 76.3 28

[0084] After 199 rounds of training, the loss function dropped from 0.058 to 0.002, and the val classification dropped from 0.057 to 0.002. The continuous decrease in the loss function value indicates that the training process is good and there is no overfitting.

[0085] like Figure 8 and Fig. 9 The training feedback parameters are shown in the figure.

[0086] Precision: It refers to the ratio of the correct positive class to all the positive classes found; Recall: It is the actual positive accuracy, that is, how many samples are found correctly.

[0087] Parameter mAP@0.5:0.95: represents the average mAP at different IoU thresholds from 0.5 to 0.95 with a step size of 0.05. mAP@0.5: represents the average mAP at thresholds greater than 0.5.

[0088] After 199 rounds of training, the training indicators fed back showed that the final recall rate reached 0.94, the precision reached 0.97, and the mAP@0.5 reached 0.97mAP@0.5:0.95 reached 0.65. The training indicator data is good, and in theory, it can identify more obvious drowning and drowning help gestures.

[0089] like Fig.10 and Fig.11 As shown, the final training results after training with 2200 images.

[0090] in conclusion

[0091] Applying YOLOv7 to drowning sites and large water entertainment venues can effectively reduce drowning deaths. Aiming at the problems of small targets, all-weather environmental changes, reflections and swimmer occlusion, a multi-scale target detection algorithm based on a feature pyramid neural network with fusion attention mechanism was designed on the basis of the YOLOv7 algorithm, and a comparative test was conducted on the selfie dataset and the swimming pool monitoring dataset. The experiment shows that the method of the invention greatly improves the detection effect of small targets, all-weather environmental changes, reflections and swimmer occlusion, and increases the detection rate of drowning people from 72.3% to 83.6%, with a speed of 30frame· -1 , which meets the requirements of real-time monitoring, but the algorithm of the present invention is still insufficient in detecting drowning persons in large areas with heavy flow of people, and is still some distance away from being put into use.

[0092] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A multi-scale drowning target detection method based on improved YOLOv7, characterized in that: The specific steps include: S1. Input data processing Perform Mosaic data enhancement, adaptive anchor point calculation, and image scaling; S2. Backbone network and feature fusion Set up a backbone network based on YOLOv7 and use the feature pyramid network and attention mechanism to integrate it; S3. Training and performance optimization Customize CNN model training with real-time feedback and parameter tuning; S4. System integration and practical application Deploy drowning detection components, integrate drone systems, and use mobile alarms to detect dangerous activities around swimming pools, such as running and drinking, and promptly notify on-duty personnel within the venue.

2. The multi-scale drowning target detection method based on improved YOLOv7 according to claim 1, characterized in that: The oMosaic data enhancement in the input data processing of step S1 is to enhance the input data using Mosaic technology, the purpose of which is to improve the network's detection effect on small targets by synthesizing parts of multiple pictures, and to enhance the model's adaptability to various scene changes.

3. The multi-scale drowning target detection method based on improved YOLOv7 according to claim 1, characterized in that: The adaptive anchor point calculation and image scaling in the input data processing of step S1 are to introduce adaptive anchor point and image scaling function blocks, which help to adjust the anchor point size and image scaling ratio of the detection algorithm according to targets of different scales, thereby improving the detection accuracy.

4. The multi-scale drowning target detection method based on improved YOLOv7 according to claim 1, characterized in that: The backbone network based on YOLOv7 in the backbone network and feature fusion of step S2 is an efficient backbone network structure that follows YOLOv7, which has been proven to have a good balance between speed and accuracy.

5. The multi-scale drowning target detection method based on improved YOLOv7 according to claim 1, characterized in that: The feature pyramid network and attention mechanism in the backbone network and feature fusion in step S2 are to combine the feature pyramid network (FPN) and the attention mechanism, and to enhance the model's adaptability and detection accuracy to occlusion and all-weather changing scenes by fusing feature maps of different depths. The attention mechanism is used to enhance important features and suppress unimportant information, thereby improving the accuracy of detection.

6. The multi-scale drowning target detection method based on improved YOLOv7 according to claim 1, characterized in that: The training of the customized CNN model in step S3 training and performance optimization is to train the customized CNN model on specific data sets, and these data sets include drowning posture images taken in swimming pool and beach scenes.

7. The multi-scale drowning target detection method based on improved YOLOv7 according to claim 1, characterized in that: The real-time feedback and parameter tuning in step S3 training and performance optimization is to adjust the parameters in the training process by real-time monitoring the changes in the loss function (Loss) to ensure that the model can converge quickly and achieve high accuracy.

8. The multi-scale drowning target detection method based on improved YOLOv7 according to claim 1, characterized in that: The system integration and practical application of step S4 includes deploying the trained model as a drowning detection component, which can detect in real time and notify relevant personnel through audio alarms, combining with the drone system, automatically dispatching drones to designated coordinates for rescue based on detected drowning events, and using mobile alarms to detect dangerous activities around the swimming pool, such as running and drinking, and promptly notifying on-duty personnel in the venue.

Images