Open water drowning prevention dangerous behavior identification and model training method and early warning system

Abstract

The application belongs to the technical field of computer vision and intelligent security and protection, and particularly relates to an open water drowning prevention dangerous behavior recognition and model training method and an early warning system. The application constructs a collaborative recognition model comprising a backbone network, a semantic segmentation network and a target detection network, takes a water area segmentation mask as a spatial attention weight to be integrated into a detection feature map, and adopts differentiated data enhancement and a composite loss function to complete model training; in an inference stage, an edge side architecture is used to accurately calculate the distance between a person and a water area shoreline and realize hierarchical early warning. The early warning system is composed of a camera device, an edge server and a cloud platform, and realizes end-edge-cloud collaborative operation. The application effectively reduces the false detection rate of drowning prevention dangerous behaviors in a complex open water scene, improves early warning accuracy and response real-time performance, and is suitable for open water drowning prevention intelligent monitoring and emergency early warning.

Description

Technical Field

[0001] This application belongs to the field of computer vision and intelligent security technology, specifically relating to a method and early warning system for identifying and training models of dangerous behaviors that could lead to drowning in open waters. Background Technology

[0002] Public safety in open waters has long relied on traditional "human + physical" methods such as manual patrols and physical barriers, which suffer from technical bottlenecks such as limited coverage, delayed response, and difficulty in adapting to complex, all-weather environments. With the development of artificial intelligence technology, intelligent drowning prevention systems based on video surveillance have become a research direction for solving these problems. The core of these systems lies in automatically and in real-time identifying dangerous behaviors in waterways through computer vision algorithms.

[0003] Existing intelligent drowning prevention technologies are mainly based on general target detection models (such as the YOLO series) or simple "segmentation + detection" multi-task frameworks. However, these solutions still have a series of key technical shortcomings that urgently need to be addressed when dealing with complex and ever-changing real-world outdoor water scenarios: First, the models suffer from insufficient scene adaptability and recognition accuracy. Existing AI early warning models are mostly trained on standardized datasets, and their network architectures are typically simple stacking or independent parallel processing of segmentation and detection tasks. This architecture struggles to focus on true water risk areas when processing images with complex backgrounds (such as pedestrians and vegetation on the shore) and under extreme weather conditions. This results in significant interference in feature extraction, poor model generalization ability, and a high false detection rate—the root cause of the poor cross-scene generalization ability of existing AI early warning models. Furthermore, for dangerous behaviors such as drowning and near-water accidents, which have low probability of occurrence but are extremely harmful, there is a lack of targeted data preprocessing strategies to alleviate sample class imbalance, further limiting the model's ability to identify key risk features.

[0004] Second, the early warning judgment mechanism is crude, and the spatial relationship quantification is inaccurate. Existing early warning systems mostly rely on the overlap between the target detection box and a preset fixed area or simple Euclidean distance for judgment, failing to quantify the precise spatial relationship between personnel targets and the irregular, dynamically changing shoreline contours. This crude judgment logic directly leads to inaccurate early warnings—it easily misses real dangers and may also produce false alarms due to shoreline interference, making reliable risk classification difficult. Essentially, this reflects the algorithmic inadequacy of the early warning system's adaptation to the spatiotemporal risk characteristics of water areas.

[0005] Third, the system suffers from high response latency and weak closed-loop management capabilities. To meet real-time requirements, existing solutions typically trade off between "lightweight inference on front-end devices" and "centralized processing on cloud servers." The former has limited computing power and is prone to sacrificing accuracy, while the latter suffers from high network transmission and computation latency, resulting in slow response. This system-level architectural flaw leads to excessively high end-to-end latency from the occurrence of an action to the triggering of effective intervention, resulting in low efficiency in early warning and response, and an inability to form an efficient management closed loop of "monitoring-analysis-early warning-response-feedback."

[0006] In summary, existing technologies in the field of intelligent identification and early warning of dangerous behaviors in open waters have long been hampered by three interconnected technical challenges: high model false detection rate, inaccurate early warning judgment, and large system response delay. There is a lack of a comprehensive technical solution that can systematically solve the above-mentioned defects by coordinating and optimizing the algorithm model, application logic, and system architecture. Summary of the Invention

[0007] This application proposes an intelligent drowning prevention solution for open water areas, aiming to systematically solve the technical defects in the prior art, such as high false detection rate due to insufficient model architecture coordination, inaccurate early warning judgment due to crude early warning judgment mechanism, and large system response delay due to unreasonable system architecture.

[0008] The first aspect of this application provides a training method for a collaborative identification model of dangerous behaviors for preventing drowning in open water, including: A dataset of outdoor open water surveillance images is obtained, wherein the images are labeled with human targets and their behavior categories; For image samples in the dataset labeled as dangerous behaviors, a multi-scale enhancement strategy is used for differential augmentation; An initial model is constructed, which includes a backbone network, a semantic segmentation network for pixel-level segmentation of water areas and non-water areas, and a target detection network for detecting and identifying human behavior within the water area. During the training of the initial model, the water area segmentation mask output by the semantic segmentation network is used as a spatial attention weight and applied to the feature map of the target detection network. The initial model is jointly trained using a composite loss function consisting of a variable focus loss function, a generalized intersection-union loss function, and a distributed focus loss function to obtain a trained collaborative identification model for drowning prevention hazards in open water.

[0009] In some embodiments of this application, the differential augmentation is used to mitigate class imbalance of dangerous behavior samples in the dataset.

[0010] In some embodiments of this application, the multi-scale enhancement strategy includes one or more of the following operations: Image resizing, automatic enhancement, fill, random cropping, random flipping, normalization, and random erasure.

[0011] In some embodiments of this application, the backbone network is a PP-HGNetV2 network, the semantic segmentation network is a DeepLabV3 network, and the object detection network is a YOLOv7 network.

[0012] In some embodiments of this application, the composite loss function is calculated using the following formula: , in, For variable focus loss, For generalized intersection and comparison of losses, For the distribution focus loss, These are the weighting coefficients. The number of positive samples. For target labels.

[0013] A second aspect of this application provides a method for identifying and issuing early warnings of dangerous behaviors that could lead to drowning in open water, executed by an edge server, including: Acquire water monitoring video streams from camera equipment; Extract the current video frame from the video stream and input the current video frame into the collaborative recognition model for drowning prevention dangerous behaviors in open water obtained by the training method according to the first aspect of the embodiments of this application; When a human target is identified, the precise distance between the location of the human target and the shoreline contour of the water area segmented by the open water drowning prevention dangerous behavior collaborative identification model is calculated. Based on the comparison results between the precise distance and the preset threshold set, the corresponding danger level is determined and a warning of the corresponding level is triggered.

[0014] In some embodiments of this application, calculating the precise distance between the location of the person target and the shoreline contour segmented by the open water drowning prevention hazard behavior collaborative identification model includes: The perpendicular compensation nearest point algorithm is used to calculate the shortest distance from the position of the personnel target to each line segment on the shoreline contour of the water area. The algorithm for closest point with perpendicular foot compensation includes: Obtain the location points of the personnel target, the set of vertices of the shoreline contour of the water area, and a desired distance value; Traverse the set of vertices, calculate the distance between each vertex and the position point, and determine the vertex with the smallest distance as the nearest vertex; Determine whether the difference between the distance between the nearest vertex and the location point and the expected distance value is less than a predetermined threshold; If so, then the nearest vertex is determined as the final nearest point; If not, proceed to the perpendicular foot compensation step: calculate the perpendicular foot of the line segment between the position point and the line segment containing the adjacent line segment before and after the nearest vertex, and select the perpendicular foot that is closer to the position point as the final nearest point; The precise distance is calculated based on the final nearest point.

[0015] In some embodiments of this application, the danger level includes a level one alarm and a level two alarm; The determination of the corresponding hazard level based on the comparison result between the precise distance and the preset threshold set includes: If the precise distance is greater than the first threshold, it is determined that there is no danger; If the precise distance is less than or equal to the first threshold and greater than the second threshold, it is determined to be the first-level alarm; If the precise distance is less than or equal to the second threshold, it is determined to be the level two alarm; Wherein, the first threshold is greater than the second threshold.

[0016] A third aspect of this application provides an intelligent early warning system for preventing drowning in open water, including a camera device, an edge server, and a cloud management platform, wherein: The camera equipment is deployed at monitoring points in the target water area to collect and encode water area monitoring video streams; The edge server is communicatively connected to the camera device and is equipped with a collaborative recognition model for dangerous behaviors in open waters to prevent drowning, obtained by the training method according to the first aspect of the present application. It is used to receive the water monitoring video stream and is configured to execute the water dangerous behavior recognition and early warning method according to the second aspect of the present application. The cloud management platform is connected to the edge server and is used to receive and aggregate early warning information from different edge servers and push it to the designated management terminal.

[0017] In some embodiments of this application, the system further includes a field audible and visual alarm device, which is communicatively connected to the edge server and used to respond to an early warning triggered by the edge server; The cloud management platform is also configured to generate an alarm command containing the event location, video clips, and handling suggestions after receiving an early warning message, and push it to the designated rescue terminal.

[0018] In summary, the training method for the collaborative identification model of dangerous behaviors for drowning prevention in open water, the identification and early warning method for dangerous behaviors for drowning prevention in open water, and the intelligent early warning system for drowning prevention in open water proposed in this application construct a collaborative identification model. During training, the water area segmentation mask output by the semantic segmentation network is used as a spatial attention weight on the feature map of the target detection network, enabling the detection network to actively focus on the risk areas of the water area for feature learning. This achieves guidance of detection by segmentation priors at the model architecture level, significantly reducing the model's false detection rate. By employing the perpendicular compensation nearest point algorithm to accurately calculate the distance between personnel and the shoreline, and implementing hierarchical early warning based on dual thresholds, the coarse judgment of regional overlap is transformed into precise spatial relationship quantification, effectively solving the problem of inaccurate early warning judgment. By constructing an "edge-cloud" collaborative architecture and deploying the collaborative identification model and early warning method on edge servers to perform core inference, high-load computation is placed at the network edge, greatly reducing data transmission latency and significantly reducing system response latency. Furthermore, differentiated data augmentation for dangerous behaviors alleviates class imbalance, and a specific composite loss function further optimizes model performance. The aforementioned technical features—from collaborative model training, accurate distance calculation and hierarchical early warning to "end-edge-cloud" system deployment—work together to achieve an overall improvement in model accuracy, early warning accuracy and system real-time performance, providing an efficient and reliable technical solution for intelligent waterway security. Attached Figure Description

[0019] The features and advantages of this application will become clearer with reference to the accompanying drawings, which are illustrative and should not be construed as limiting the application in any way. In the drawings: Figure 1 This is a schematic diagram of the PP-HGNetV2 network structure shown in some embodiments of this application; Figure 2 This is a diagram illustrating the architecture of an intelligent early warning system for preventing drowning in open water, based on some embodiments of this application. Figure 3 This is a flowchart illustrating a training method for a collaborative identification model of dangerous behaviors for preventing drowning in open water, according to some embodiments of this application. Figure 4 This is the training and evaluation curve of a collaborative identification model for drowning prevention hazards in open water, as shown in one embodiment of this application. Figure 5 This is a flowchart illustrating a method for identifying and issuing early warnings of dangerous behaviors that may lead to drowning in open water, according to some embodiments of this application, and is executed by an edge server. Detailed Implementation

[0020] In the following detailed description, numerous specific details of this application are illustrated by example to provide a thorough understanding of the relevant disclosure. However, it will be apparent to those skilled in the art that this application can be practiced without these details. It should be understood that the terms “system,” “apparatus,” “unit,” and / or “module” used in this application are one way of distinguishing different parts, elements, sections, or components at different levels in a sequential arrangement. However, these terms may be replaced with other expressions if other expressions can achieve the same purpose.

[0021] It should be understood that when a device, unit, or module is referred to as being "on," "connected to," or "coupled to" another device, unit, or module, it may be directly connected to or coupled to, or communicate with, other devices, units, or modules, or there may be intermediate devices, units, or modules present, unless the context explicitly indicates otherwise. For example, the term "and / or" as used herein includes any one and all combinations of one or more of the relevant listed items.

[0022] The terminology used in this application is for the purpose of describing specific embodiments only and is not intended to limit the scope of this application. As shown in the specification and claims of this application, unless the context clearly indicates otherwise, words such as "a," "an," "an," and / or "the" do not specifically refer to the singular and may also include the plural. Generally speaking, the terms "comprising" and "including" only indicate that explicitly identified features, integrals, steps, operations, elements, and / or components are included, and such expressions do not constitute an exclusive list, and other features, integrals, steps, operations, elements, and / or components may also be included.

[0023] Referring to the following description and accompanying drawings, these and other features and characteristics, operating methods, functions of related structural elements, combinations of parts, and economics of manufacture of this application can be better understood, wherein the description and drawings form part of the specification. However, it is clearly understood that the drawings are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. It is understood that the drawings are not drawn to scale.

[0024] Various structural diagrams are used in this application to illustrate various variations of the embodiments according to this application. It should be understood that the preceding or following structures are not intended to limit this application. The scope of protection of this application is determined by the claims.

[0025] Drowning is the most serious safety hazard in open waters, often occurring suddenly and unexpectedly. Traditional prevention and control methods relying on manual patrols not only have low coverage but also struggle to achieve 24 / 7, comprehensive monitoring. Therefore, utilizing computer vision technology to build an intelligent drowning prevention early warning system, enabling automatic and real-time identification of dangerous behaviors such as people falling into the water and unauthorized wading, has become an inevitable trend for improving water safety management.

[0026] In the field of deep learning, the YOLO series of algorithms, with their excellent balance between detection speed and accuracy, have become the mainstream choice for real-time object detection tasks. Meanwhile, to accurately understand aquatic scenes, classic semantic segmentation algorithms such as DeepLab and U-Net are also widely used for pixel-level extraction of shorelines. Combining object detection with semantic segmentation to construct a "segmentation + detection" multi-task framework is considered a potential technical framework for building intelligent drowning prevention systems. However, when directly transferring these general technologies to real-world outdoor drowning prevention scenarios, deep-seated systemic obstacles remain, mainly reflected in the following three dimensions: First, at the algorithm and training levels, existing architectures struggle to balance accuracy and learning efficiency in complex scenarios. On one hand, traditional "segmentation + detection" architectures often involve a simple superposition or shallow sharing of the two tasks. The precise water boundary information extracted by the segmentation network fails to translate into effective spatial constraints to guide the detection network to focus on the water area. This lack of architectural synergy makes the detection model highly susceptible to interference from background elements such as pedestrians and trees on the shore, resulting in a persistently high false positive rate. On the other hand, drowning and dangerous behaviors are extremely rare events in reality, leading to an extreme imbalance between positive and negative samples in the training data. Under this data distribution, the model training process naturally tends to optimize the recognition of the majority class (safe behaviors), neglecting the learning of key dangerous features, resulting in a serious risk of missed detections when faced with real signs of drowning.

[0027] Secondly, at the level of early warning and judgment mechanisms, the crude spatial logic cannot adapt to the complex and ever-changing shoreline environment. Even if the model successfully detects personnel, existing systems typically judge risk based solely on pre-defined fixed rectangular areas or simple overlapping frames. This "one-size-fits-all" logic completely ignores the meandering and irregular geometric features of natural waterfronts and struggles to quantify the precise distance between personnel and the water surface. The result is often either misjudging normal activities on the shore as dangerous or ignoring personnel who have already stepped into shallow water. This inaccuracy in early warning judgments severely undermines the reliability of the system as a basis for rescue decisions.

[0028] Finally, at the system deployment and response level, the imbalance in the allocation of computing resources leads to unacceptable response delays. High-precision deep learning models often require expensive computing power. If all computing is concentrated in the cloud, network transmission delays will cause the golden window for rescue to be missed; if simplified models are forcibly deployed on front-end cameras with limited computing power, detection accuracy will be sacrificed. Existing solutions lack an efficient "edge-cloud" collaborative computing architecture, resulting in excessive end-to-end latency from the occurrence of an emergency to the triggering of an alert, making it difficult to meet the stringent requirements of second-level response in drowning prevention scenarios.

[0029] To systematically address the aforementioned shortcomings, this application constructs an intelligent drowning prevention scheme for open water areas through the collaborative optimization of models, algorithms, and systems. Regarding the model architecture, as follows... Figure 1 (See the schematic diagram of the PP-HGNetV2 network structure). This application designs a collaborative recognition model with PP-HGNetV2 as the backbone network and deeply integrates the DeepLabV3 semantic segmentation module and the improved YOLOv7 object detection network. Its core lies in establishing an active guidance and spatial constraint mechanism for the object detection process based on the semantic segmentation results within the model, rather than simple task parallelism. Simultaneously, this application employs differentiated data augmentation strategies and specific composite loss functions for dangerous behavior samples to optimize the model training process.

[0030] In terms of early warning judgment logic, this application abandons the coarse judgment method based on fixed area overlap and proposes a hierarchical early warning mechanism based on precise spatial distance calculation. This mechanism calculates the precise distance between the personnel target and the shoreline contour segmented by the model, and achieves intelligent classification based on dual thresholds, thereby significantly improving the accuracy of early warning judgment.

[0031] To achieve efficient operation of the above algorithm, this application employs the following... Figure 2 The diagram (of an intelligent early warning system for drowning prevention in open waters) illustrates a three-layer collaborative system architecture: edge-cloud. This architecture deploys a trained collaborative recognition model on an edge server for real-time video analysis; a cloud management platform handles global data aggregation and scheduling; and on-site equipment and management terminals form a closed-loop response link. This design effectively reduces end-to-end response latency while ensuring the operation of a high-precision AI model.

[0032] In summary, this application aims to comprehensively improve the accuracy, real-time performance, and reliability of drowning prevention early warning in open waters by organically combining a collaborative model architecture, precise early warning algorithms, and efficient system deployment.

[0033] Figure 3 This is a flowchart illustrating a training method for a collaborative identification model of dangerous behaviors for preventing drowning in open water, according to some embodiments of this application. Figure 3 As shown, the method specifically includes: S310, acquire a dataset of outdoor open water monitoring images, the images being labeled with human targets and their behavior categories.

[0034] This step is used to deploy a monitoring network at key locations in the target water area, collect continuous monitoring videos covering various weather and lighting conditions, and build an initial mixed training sample library.

[0035] In one embodiment of this application, a monitoring network is deployed in the core landscape area, ecological corridor area, and large water body area of ​​the Zhengzhou East District water area, using 192 high-definition and wide dynamic range cameras, achieving a core water area coverage rate of 89.6%; continuous monitoring video covering multiple scenarios including day and night, sunny and rainy weather, and backlighting is collected, accumulating over 10,000 hours of video. Keyframes are extracted from the original videos and fused with a public dataset to construct an initial hybrid training sample library. S320, for image samples in the dataset labeled as dangerous behaviors, a multi-scale enhancement strategy is used for differentiated augmentation.

[0036] To address the class imbalance issue caused by the low proportion of dangerous behavior samples such as falling into water and near water in the initial mixed training sample database, this application employs multi-scale data augmentation techniques for differential expansion, specifically including: uniform image resizing (ResizeImage), automated augmentation combination (TimmAutoAugment), padding (Padv2), random cropping (RandomCropImage), random flipping (RandFlipImage), normalization (NormalizeImage), and random erasing (RandomErasing). After processing, the dataset size is expanded to 8.3 times the original, effectively improving the representativeness of dangerous behavior samples.

[0037] S330, Construct an initial model, which includes a backbone network, a semantic segmentation network for pixel-level segmentation of water areas and non-water areas, and a target detection network for detecting and identifying human behavior within the water area.

[0038] To further improve the accuracy, scene adaptability, and feature representation capabilities of target detection, especially for scenes with complex backgrounds and blurred boundaries, such as waterways and shorelines, some embodiments of this application construct a fusion target detection model with PP-HGNetV2 as the backbone network and integrating prior knowledge of waterway and shoreline segmentation from DeepLabV3.

[0039] PP-HGNetV2, a lightweight, high-precision image classification backbone network proposed by Baidu PaddlePaddle, boasts core advantages such as fewer parameters, strong feature representation capabilities, and high inference efficiency thanks to its innovative HG-Block structure, efficient feature pyramid design, and lightweight parameter configuration. Compared to traditional backbone networks (such as ResNet and MobileNet), it can extract multi-scale features from images more efficiently, providing a high-quality feature input foundation for subsequent object detection tasks. It also adapts to the deployment requirements of edge devices, addressing the pain points of slow inference speed and high resource consumption in traditional models. The network structure of PP-HGNetV2 is as follows: Figure 1 As shown.

[0040] DeepLabV3, as a classic semantic segmentation algorithm, performs excellently in pixel-level region segmentation tasks. For water and shoreline segmentation scenarios, it has accumulated mature prior knowledge, including shoreline edge feature extraction, pixel-level differentiation strategies between water and shoreline regions, and shoreline contour optimization under complex backgrounds (such as vegetation, buildings, and water ripples).

[0041] The target detection part in the embodiment described is built based on the YOLOv7 algorithm. The YOLOv7 algorithm is an advanced target detection algorithm developed through continuous iteration of the YOLO series of algorithms. Compared with the previous generation of YOLO models, it effectively breaks through the technical bottleneck of 'high accuracy and high speed cannot be achieved at the same time' by optimizing the network structure design (such as introducing the ELAN module to improve feature fusion efficiency and using the MP-Conv module to enhance feature extraction capabilities). While ensuring detection accuracy, it significantly improves inference speed, which can meet the application requirements of real-time detection scenarios and is widely applicable to target localization and recognition tasks in various complex environments.

[0042] S340, during the training of the initial model, the water area segmentation mask output by the semantic segmentation network is used as a spatial attention weight and applied to the feature map of the target detection network.

[0043] This application integrates prior knowledge of segmentation into the target detection model, providing explicit scene constraints for target detection. In the aforementioned embodiment, DeepLabV3 pre-segments the input image to define the water area and shoreline, accurately distinguishing between water and non-water areas. This guides YOLOv7 to focus on detecting targets within the water area, reducing false detections in irrelevant areas outside the shoreline (such as shoreline buildings and vegetation). Simultaneously, by utilizing the shoreline edge features obtained from the segmentation, YOLOv7 is assisted in accurately locating the boundaries of targets within the water area, avoiding target localization errors caused by water surface reflection, shoreline blurring, and other issues.

[0044] The combination of PP-HGNetV2, DeepLabV3, and YOLOv7 forms a collaborative mechanism of "backbone feature extraction - shoreline segmentation constraint - accurate target detection": PP-HGNetV2 is responsible for efficiently extracting multi-scale features of the image, providing a foundation for subsequent segmentation and detection; DeepLabV3 utilizes its own segmentation advantages to output the shoreline segmentation results as prior knowledge, constraining the detection range and optimizing the detection accuracy; YOLOv7, with the assistance of segmentation priors, achieves fast and accurate detection of targets in water scenes, thus improving the overall performance of the model in terms of real-time performance, detection accuracy, and scene adaptability.

[0045] S350, the initial model is jointly trained using a composite loss function consisting of a variable focus loss function, a generalized intersection-union loss function, and a distributed focus loss function to obtain a trained collaborative recognition model for drowning prevention hazards in open water.

[0046] This step uses a specific loss function and training strategy to optimize the initial model built by S330.

[0047] Loss function: This application employs a composite loss function consisting of VFL (VariFocal Loss), GIOU Loss (Generalized Cross-Union Loss), and DFL (Distribution Focal Loss) to synergistically improve classification and localization accuracy. This composite loss function is calculated using the following formula: , in, For variable focus loss, For generalized intersection and comparison of losses, For the distribution focus loss, These are the weighting coefficients. The number of positive samples. For target labels.

[0048] Training hyperparameters: One embodiment of this application uses the AdamW optimizer with an initial learning rate of 0.005, in conjunction with a cosine annealing scheduler; a batch size of 64; a training epoch of 200; and an early stopping strategy based on validation set performance; and introduces L2 weight decay (λ=0.01) and Dropout (p=0.2) to prevent overfitting.

[0049] Training process and results: The training and evaluation curve of the collaborative identification model for drowning prevention hazards in open water is as follows: Figure 4 As shown. Figure 4The figure shows the changes in training cross-entropy loss, training loss, and training set Top-1 accuracy over 200 training epochs. As can be seen from the figure, the training loss decreases rapidly with increasing epochs and then tends to stabilize, while the training set Top-1 accuracy increases rapidly and eventually maintains a high level close to 1.0, indicating that the model training process converges well. Through evaluation on the test set, the model ultimately achieved a high TopkAcc value of 0.91.

[0050] To objectively verify the performance advantages of the collaborative recognition model proposed in this application compared with the prior art, an embodiment of this application conducted a performance comparison test between the "collaborative model of this application" trained by the method described in S310-S350 and a variety of benchmark models in the same test dataset and inference hardware environment. The key indicator comparison is shown in Table 1.

[0051] Table 1: Performance comparison experimental data.

[0052] As shown in Table 1, under fixed test conditions, the collaborative model of this application demonstrates a comprehensive leading advantage in both core indicators: Significantly superior recognition accuracy and outstanding robustness in extreme environments: The collaborative model presented in this application achieves the highest recognition accuracy under both normal lighting and extreme weather conditions. Crucially, under extreme conditions such as heavy rain and dense fog, its accuracy (85.5%) represents an absolute improvement of over 17 percentage points compared to the widely used original YOLOv7 model (68.3%), a significant increase exceeding that under normal lighting conditions. This demonstrates that the collaborative training mechanism of the DeepLabV3 semantic segmentation network and the YOLOv7 detection network employed in this application can effectively enhance the model's feature discrimination and generalization capabilities under complex interference and harsh environments, addressing the key deficiency of poor environmental adaptability in existing solutions.

[0053] Processing efficiency is optimized simultaneously: The single-frame processing latency (2.8 seconds) of the collaborative model in this application is the lowest among the comparative models, indicating that the lightweight backbone network and collaborative structure design of PP-HGNetV2 adopted by the model improves accuracy while ensuring and optimizing the computational efficiency of the model, thus achieving a balance between accuracy and speed.

[0054] The comparative data above clearly demonstrate that the collaborative recognition model of this application is not a simple replacement or stacking of existing components. Through the collaborative architecture and training method given in S310-S350, the obtained model achieves a comprehensive, significant, and unexpected improvement in recognition accuracy (especially robustness in extreme environments) and processing efficiency.

[0055] Figure 5This is a flowchart illustrating a method for identifying and issuing early warnings of dangerous behaviors for preventing drowning in open water, according to some embodiments of this application, executed by an edge server. Figure 5 As shown, the method specifically includes: S510 acquires a water monitoring video stream from a camera device.

[0056] In this step, the edge server receives real-time or near real-time water monitoring video streams from camera equipment deployed in open water areas as data input for the method.

[0057] S520, extract the current video frame from the video stream and input the current video frame into the input according to... Figure 3 The collaborative identification model for dangerous behaviors in open waters to prevent drowning, obtained through the training methods described in S310-S350.

[0058] The edge server extracts the current video frame from the received video stream. This video frame is then fed into a pre-trained and deployed collaborative recognition model for drowning prevention behaviors in open water, enabling simultaneous and accurate shoreline segmentation and target detection and behavior recognition of individuals within the water area.

[0059] The model uses the aforementioned PP-HGNetV2 as its backbone, deeply integrates the DeepLabV3 semantic segmentation network and the improved YOLOv7 object detection network, and is further refined through... Figure 3 The training method described in S310-S350 is used to obtain the results.

[0060] S530, when a person target is identified, calculate the precise distance between the position of the person target and the shoreline contour of the water area segmented by the open water drowning prevention dangerous behavior collaborative identification model.

[0061] When the model identifies a person target, this step calculates the precise distance between the target's location and the waterline contour obtained from the model's segmentation. This is specifically achieved using the perpendicular compensation nearest point algorithm.

[0062] Specifically, the precise distance is calculated using the perpendicular compensation nearest point algorithm. This algorithm is used to calculate the nearest point from a given point to the polygon contour, thereby obtaining the shortest distance, and includes a main function and an auxiliary function.

[0063] Main function: Get the nearest point of the contour Input: Target point P (array), contour vertex set C (polygon vertex list), desired distance d. Output: Coordinates of the nearest point.

[0064] The specific steps are as follows: (1) Traverse all vertices in C, calculate the Euclidean distance between each vertex and P, and record the vertex N with the smallest distance and its index i.

[0065] (2) If |dist(P,N) - d| < 0.01, then return N as the nearest point directly; otherwise, enter the perpendicular compensation mode.

[0066] (3) In the perpendicular foot compensation mode, obtain the previous adjacent vertex prev and the next adjacent vertex next of N. When N is the last vertex of the contour, next is the first vertex of the contour. Calculate the perpendicular foot F1 from P to line segment (prev, N) and the perpendicular foot F2 from P to line segment (N, next), compare the distances of F1, F2 and P, and return the perpendicular foot point with the smaller distance.

[0067] Auxiliary function: Calculates the foot of the perpendicular from a point to a line segment. Given a point P(x0, y0) and line segment endpoints P1(x1, y1) and P2(x2, y2): If x1 ≠ x2 (not a perpendicular line segment), calculate the slope a = (y2 - y1) / (x2 - x1), and express the line as Ax + By + C = 0, where A = a, B = -1, and C = y1 - a·x1. The coordinates of the foot of the perpendicular are calculated using the following formula: x = (B 2 ·x0 - A·B·y0 - A·C) / (A 2 + B 2 ) y = (-A·B·x0 + A 2 ·y0 - B·C) / (A 2 + B 2 ) If x1 = x2 (perpendicular line segment), the coordinates of the foot of the perpendicular are (x1, y0).

[0068] This algorithm employs a perpendicular compensation mechanism. When the distance between the target point and the contour vertex deviates from the expected value, it searches for the perpendicular point on adjacent line segments, thus obtaining a more accurate nearest point than simple vertex comparison. The algorithm complexity is O(n), where n is the number of contour vertices.

[0069] S540, based on the comparison result between the precise distance and the preset threshold set, determine the corresponding danger level and trigger the corresponding level of warning.

[0070] This embodiment presets a first threshold and a second threshold, where the first threshold is greater than the second threshold. The hazard level is determined based on the precise distance within the specified range. If the precise distance is greater than the first threshold, it is determined that there is no danger and no warning is triggered; If the precise distance is less than or equal to the first threshold and greater than the second threshold, it is determined to be close to a dangerous area, and the danger level is level one alarm. If the precise distance is less than or equal to the second threshold, it is determined that the person has entered a dangerous water area or fallen into the water, and the danger level is level two alarm.

[0071] Implement corresponding early warning strategies based on different risk levels. For example: Level 1 alarms can be recorded locally and displayed on-site; Level 2 alarms generate warning information including time, location, and behavior description, which is reported to the management platform and triggers emergency response.

[0072] Figure 2 This is a diagram illustrating the architecture of an intelligent early warning system for preventing drowning in open water, based on some embodiments of this application. Figure 2 As shown, this system adopts a three-layer collaborative architecture of "device-edge-cloud", which specifically includes the following components: Camera equipment: The system includes camera equipment on the "end side" for video capture and primary encoding. The camera equipment is deployed at key locations within the target water area.

[0073] In one embodiment of this application, the camera equipment is deployed in the core landscape area, ecological corridor area, and large water body area of ​​the Zhengzhou East District water area, forming a monitoring network. The monitoring network uses 192 high-definition and wide dynamic range cameras, with a core water area coverage rate of over 89.6%.

[0074] Edge server: The system deploys edge servers at the "edge side". The edge servers carry the trained model and perform real-time inference on the video stream to identify two types of behavior: "approaching a dangerous area" (Level 1 alarm) and "entering / falling into a dangerous area" (Level 2 alarm).

[0075] The model deployed on the edge server is based on the aforementioned PP-HGNetV2 as its backbone, deeply fusing the DeepLabV3 semantic segmentation network and the improved YOLOv7 object detection network, and then... Figure 3 The training method described in S310-S350 is used to obtain the results.

[0076] Cloud management platform: The system includes a smart water management platform in the "cloud" for data aggregation, model iteration and updates, and global scheduling.

[0077] The cloud-based management platform works in conjunction with emergency resources to form a closed-loop management system of "monitoring-early warning-response-feedback". Alarm information is pushed to the cloud-based management platform in real time.

[0078] In some embodiments of this application, the system further includes: On-site audible and visual alarm device: Once an alarm is triggered, a Level 1 alarm will activate an on-site audible and visual alarm.

[0079] Management / Rescue Terminal: Level 2 alarms automatically push alarm information, including time, location, behavior description, and video clips, to the management terminal.

[0080] The "end-edge-cloud" collaborative computing architecture of this application deploys a trained high-precision collaborative recognition model on an edge server, performing core video stream analysis and inference at the network edge close to the data source. This significantly reduces the network transmission time required to upload massive amounts of video data to the cloud, which is key to achieving low latency. Simultaneously, the model employs a lightweight design, including the PP-HGNetV2 backbone network, enabling it to adapt to edge computing resources and ensuring inference efficiency.

[0081] Based on the above architecture, the system achieved excellent real-time performance in long-term operation tests in real-world scenarios. During a continuous trial run and performance statistics period of 21 months (August 2023 to April 2025), the average end-to-end latency from the occurrence of the behavior to the triggering of the warning information was 2.8 seconds, meeting the stringent requirements of real-time warning and verifying the effectiveness of the "end-edge-cloud" collaborative architecture in reducing system response latency.

[0082] To make the inventive objectives, technical solutions, and advantages of this application clearer, a detailed description will be provided below with reference to a specific embodiment. This embodiment is not intended to limit the scope of protection of this application, but rather to demonstrate the complete deployment, operation process, and verification effect of the intelligent waterway drowning prevention early warning method and system in a typical open water scenario (Zhengzhou East District waterway, China).

[0083] 1. System Hardware Deployment A total of 192 high-definition cameras (1920×1080 pixels resolution) have been deployed at key water areas such as Ruyi Lake, Longhu Lake, and scenic waterways in the eastern district of Zhengzhou, forming a monitoring network with a core water area coverage rate of 89.6%. The camera system is connected to lightning protection and voltage stabilization power supplies and is equipped with nighttime supplementary lighting equipment to ensure stable operation around the clock.

[0084] 2. Data Processing and Model Training Based on the aforementioned hardware network, over 10,000 hours of surveillance video covering various scenarios were collected. 9,963 valid keyframes were extracted from the original videos and merged with 7,188 related images from a publicly available dataset to form an initial training sample library. Subsequently, the initial sample library was preprocessed and expanded strictly according to the methods described in S310-S320 (i.e., employing a differentiated multi-scale enhancement strategy for dangerous behavior samples), ultimately obtaining an enhanced training set of approximately 100,000 images. Using this dataset, the collaborative recognition model, based on PP-HGNetV2 as the backbone and deeply fused with DeepLabV3 and an improved YOLOv7, was trained according to the methods described in S330-S350.

[0085] 3. System software deployment and integration After the trained collaborative recognition model is lightweighted, it is developed together with the early warning logic software using a microservice architecture and containerized and deployed on the aforementioned "end-edge-cloud" hardware environment: Edge nodes: The collaborative identification model is deployed on edge servers equipped with Intel Xeon processors to enable real-time analysis and hierarchical early warning judgment of access video streams.

[0086] Cloud platform: Integrated with the regional smart water management platform (i.e., cloud management platform), providing a web management interface for global alarm information aggregation and viewing, operational data statistics and model version management.

[0087] Alarm linkage: When the system triggers a level 2 alarm, in addition to sending warning information to the mobile terminals of on-duty personnel through the cloud platform, it also automatically triggers the on-site broadcast system to issue voice warnings through the API interface, and can display real-time images in a pop-up window on the smart screen of the command center, thus forming a complete management closed loop of "monitoring-early warning-handling-feedback".

[0088] 4. Application Effects During the 21-month continuous trial operation (August 2023 to April 2025), a comprehensive statistical analysis and evaluation of the system performance was conducted, and the results are as follows: Recognition accuracy: After manual verification, the accuracy rate of dangerous behavior recognition is 95.7% under normal lighting conditions, and still maintains an accuracy rate of 85.5% under extreme weather conditions such as heavy rain and dense fog.

[0089] Real-time response: The average end-to-end delay from the occurrence of dangerous behavior to the triggering of the warning information is 2.8 seconds, which fully meets the stringent requirements of real-time warning.

[0090] System operation statistics: A total of 3.4679 million valid alarms were generated. The ratio of Level 1 alarms (approaching dangerous areas) to Level 2 alarms (entering / falling into dangerous areas) was approximately 6.4:1. The spatiotemporal distribution of alarm events is highly consistent with the empirical risk patterns.

[0091] Prevention effectiveness: No drowning accidents occurred in the deployment area during the entire pilot period, and the prevention effect was significant compared with historical data for the same period.

[0092] This specific implementation example demonstrates that by applying the collaborative identification model training method, dangerous behavior identification and early warning method, and intelligent early warning system provided in this application, high-precision identification, low-latency response, and highly reliable closed-loop management can be achieved in complex real open water environments, effectively improving the level of intelligence and proactive prevention capabilities of water safety management.

[0093] In summary, the training method for the collaborative identification model of dangerous behaviors for drowning prevention in open water, the identification and early warning method for dangerous behaviors for drowning prevention in open water, and the intelligent early warning system for drowning prevention in open water proposed in this application construct a collaborative identification model. During training, the water area segmentation mask output by the semantic segmentation network is used as a spatial attention weight on the feature map of the target detection network, enabling the detection network to actively focus on the risk areas of the water area for feature learning. This achieves guidance of detection by segmentation priors at the model architecture level, significantly reducing the model's false detection rate. By employing the perpendicular compensation nearest point algorithm to accurately calculate the distance between personnel and the shoreline, and implementing hierarchical early warning based on dual thresholds, the coarse judgment of regional overlap is transformed into precise spatial relationship quantification, effectively solving the problem of inaccurate early warning judgment. By constructing an "edge-cloud" collaborative architecture and deploying the collaborative identification model and early warning method on edge servers to perform core inference, high-load computation is placed at the network edge, greatly reducing data transmission latency and significantly reducing system response latency. Furthermore, differentiated data augmentation for dangerous behaviors alleviates class imbalance, and a specific composite loss function further optimizes model performance. The aforementioned technical features—from collaborative model training, accurate distance calculation and hierarchical early warning to "end-edge-cloud" system deployment—work together to achieve an overall improvement in model accuracy, early warning accuracy and system real-time performance, providing an efficient and reliable technical solution for intelligent waterway security.

[0094] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the devices and modules described above can be referred to the corresponding descriptions in the foregoing device embodiments, and will not be repeated here.

[0095] Although the subject matter described herein is provided in the general context of execution on a computer system in conjunction with an operating system and applications, those skilled in the art will recognize that other implementations can also be executed in conjunction with other types of program modules. Generally, program modules include routines, programs, components, data structures, and other types of structures that perform specific tasks or implement specific abstract data types. Those skilled in the art will understand that the subject matter described herein can be practiced using other computer system configurations, including handheld devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, minicomputers, mainframes, etc., and can also be used in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In a distributed computing environment, program modules may reside on both local and remote memory storage devices.

[0096] Those skilled in the art will recognize that the units and method steps of the various examples described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0097] It should be understood that the specific embodiments described above are merely illustrative or explanatory of the principles of this application and do not constitute a limitation thereof. Therefore, any modifications, equivalent substitutions, improvements, etc., made without departing from the spirit and scope of this application should be included within the protection scope of this application. Furthermore, the appended claims are intended to cover all variations and modifications falling within the scope and boundaries of the appended claims, or equivalent forms of such scope and boundaries.

Claims

1. A training method for a collaborative identification model of dangerous behaviors for preventing drowning in open water, characterized in that, include: A dataset of outdoor open water surveillance images is obtained, wherein the images are labeled with human targets and their behavior categories; For image samples in the dataset labeled as dangerous behaviors, a multi-scale enhancement strategy is used for differential augmentation; An initial model is constructed, which includes a backbone network, a semantic segmentation network for pixel-level segmentation of water areas and non-water areas, and a target detection network for detecting and identifying human behavior within the water area. During the training of the initial model, the water area segmentation mask output by the semantic segmentation network is used as a spatial attention weight and applied to the feature map of the target detection network. The initial model is jointly trained using a composite loss function consisting of a variable focus loss function, a generalized intersection-union loss function, and a distributed focus loss function to obtain a trained collaborative identification model for drowning prevention hazards in open water.

2. The method according to claim 1, characterized in that, The differential augmentation is used to mitigate the class imbalance of dangerous behavior samples in the dataset.

3. The method according to claim 1, characterized in that, The multi-scale enhancement strategy includes one or more of the following operations: Image resizing, automatic enhancement, fill, random cropping, random flipping, normalization, and random erasure.

4. The method according to claim 1, characterized in that: The backbone network is the PP-HGNetV2 network, the semantic segmentation network is the DeepLabV3 network, and the object detection network is the YOLOv7 network.

5. The method according to claim 1, characterized in that, The composite loss function is calculated using the following formula: ， in, For variable focus loss, For generalized intersection and comparison of losses, For the distribution focus loss, These are the weighting coefficients. The number of positive samples. For target labels.

6. A method for identifying and issuing early warnings of dangerous behaviors that could lead to drowning in open water, executed by an edge server, characterized in that... include: Acquire water monitoring video streams from camera equipment; Extract the current video frame from the video stream and input the current video frame into the collaborative recognition model for open water drowning prevention dangerous behaviors obtained by the training method according to claim 1; When a human target is identified, the precise distance between the location of the human target and the shoreline contour of the water area segmented by the open water drowning prevention dangerous behavior collaborative identification model is calculated. Based on the comparison results between the precise distance and the preset threshold set, the corresponding danger level is determined and a warning of the corresponding level is triggered.

7. The method according to claim 6, characterized in that, The calculation of the precise distance between the location of the personnel target and the shoreline contour segmented by the open water drowning prevention hazard behavior collaborative identification model includes: The perpendicular compensation nearest point algorithm is used to calculate the shortest distance from the position of the personnel target to each line segment on the shoreline contour of the water area. The algorithm for closest point with perpendicular foot compensation includes: Obtain the location points of the personnel target, the set of vertices of the shoreline contour of the water area, and a desired distance value; Traverse the set of vertices, calculate the distance between each vertex and the position point, and determine the vertex with the smallest distance as the nearest vertex; Determine whether the difference between the distance between the nearest vertex and the location point and the expected distance value is less than a predetermined threshold; If so, then the nearest vertex is determined as the final nearest point; If not, proceed to the perpendicular foot compensation step: calculate the perpendicular foot of the line segment between the position point and the line segment containing the adjacent line segment before and after the nearest vertex, and select the perpendicular foot that is closer to the position point as the final nearest point; The precise distance is calculated based on the final nearest point.

8. The method according to claim 6, characterized in that: The danger levels include Level 1 alarms and Level 2 alarms; The determination of the corresponding hazard level based on the comparison result between the precise distance and the preset threshold set includes: If the precise distance is greater than the first threshold, it is determined that there is no danger; If the precise distance is less than or equal to the first threshold and greater than the second threshold, it is determined to be the first-level alarm; If the precise distance is less than or equal to the second threshold, it is determined to be the level two alarm; Wherein, the first threshold is greater than the second threshold.

9. An intelligent early warning system for preventing drowning in open waters, characterized in that, This includes camera equipment, edge servers, and a cloud management platform, among which: The camera equipment is deployed at monitoring points in the target water area to collect and encode water area monitoring video streams; The edge server is communicatively connected to the camera device and is equipped with a collaborative recognition model for dangerous behaviors in open waters to prevent drowning, obtained by the training method according to claim 1. It is used to receive the water monitoring video stream and is configured to execute the water dangerous behavior recognition and early warning method according to claim 6. The cloud management platform is connected to the edge server and is used to receive and aggregate early warning information from different edge servers and push it to the designated management terminal.

10. The system according to claim 9, characterized in that: The system also includes an on-site audible and visual alarm device, which is communicatively connected to the edge server and used to respond to early warnings triggered by the edge server; The cloud management platform is also configured to generate an alarm command containing the event location, video clips, and handling suggestions after receiving an early warning message, and push it to the designated rescue terminal.

Images