Personnel falling-into-water detection method and system and readable medium

Abstract

The invention belongs to the technical field of image processing, and relates to a personnel falling-into-water detection method and system and a readable medium, and the method comprises the following steps: obtaining a video or image of a to-be-detected water surface; segmenting the video or image of the water surface through an improved YOLOv8 model to obtain a water surface mask; performing pedestrian detection in the video or image of the water surface through a YOLOv7 model; and if the YOLOv7 model detects a pedestrian and the pedestrian is in the segmented water surface mask, judging that a person is drowning, otherwise, judging that no person is drowning. According to the scheme, the multi-scale features can be effectively represented and processed, so that the efficiency and accuracy of personnel falling-into-water detection are improved.

Description

Technical Field

[0001] The invention relates to a method, a system and a readable medium for detecting a person falling into water, and belongs to the technical field of image processing. Background Art

[0002] A large number of natural waters, such as rivers, lakes, and oceans, lack real-time and comprehensive supervision, which leads to the possibility of drowning. The golden rescue time for drowning is relatively short. If the drowning person cannot be found in the first time, it is easy to cause casualties. Even if there are monitored waters, supervisors such as lifeguards cannot guard them all day long, so there is an urgent need for methods and devices for automatically detecting people falling into the water.

[0003] The existing methods for automatic detection of people falling into water are usually implemented by extracting video data from the camera and then implementing it through an image recognition algorithm. The most commonly used image recognition algorithm is the YOLO algorithm. For example, the YOLOv8 algorithm is the latest version of Ultralytics' YOLO. As a cutting-edge, state-of-the-art (SOTA) model, YOLOv8 introduces new features and improvements based on the success of previous versions to improve performance, flexibility, and efficiency. YOLOv8 supports a full range of visual AI tasks, including detection, segmentation, pose estimation, tracking, and classification. This versatility enables users to leverage the capabilities of YOLOv8 to meet the needs of a variety of applications and fields.

[0004] One of the problems with the deep learning target detection model based on the YOLOv8 algorithm is that it is too large and faces the problem of insufficient memory. Secondly, these scenarios require low latency, or fast response speed. The traditional YOLOv8 and Backbone use a relatively complex C2F network structure, which greatly increases the model's computational workload, slows down the detection speed, and limits its application. It is difficult to be used in some application scenarios such as mobile terminals or embedded devices. Summary of the invention

[0005] In view of the above problems, the purpose of the present invention is to propose a method, system and readable medium for detecting people falling into water, by improving the YOLOv8 algorithm so that it can effectively represent and process multi-scale features, thereby improving the efficiency and accuracy of people falling into water detection.

[0006] To achieve the above-mentioned purpose, the present invention proposes the following technical scheme: a method for detecting people falling into water, comprising the following steps: obtaining a video or image of a water surface to be detected; segmenting the video or image of the water surface through an improved YOLOv8 model to obtain a water surface mask; performing pedestrian detection in the video or image of the water surface through a YOLOv7 model; if the YOLOv7 model detects a pedestrian and the pedestrian is within the segmented water surface mask, it is judged that someone is drowning, otherwise it is judged that no one is drowning.

[0007] Furthermore, the Neck part of the improved YOLOv8 model is composed of several bidirectional feature pyramid networks.

[0008] Furthermore, the bidirectional feature pyramid network adds an additional weight to each input feature and allows features to be fused in both top-down and bottom-up directions.

[0009] Furthermore, the weight is a scalar, a vector or a multi-dimensional tensor.

[0010] Furthermore, the scalar is a feature of the image, the vector is the data of each channel, and the multidimensional tensor is the data of each pixel.

[0011] Furthermore, the Neck part of the improved YOLOv8 model is composed of a MobileNetV3 network or a RevColV1 reversible network.

[0012] The present invention also discloses a system for detecting people falling into water, comprising: an image acquisition module, used to obtain a video or image of a water surface to be detected; an image segmentation module, used to segment the video or image of the water surface through an improved YOLOv8 model to obtain a water surface mask; a pedestrian detection module, used to detect pedestrians in the video or image of the water surface through a YOLOv7 model; and a drowning detection module, used to determine that someone is drowning if a pedestrian is detected by the YOLOv7 model and the pedestrian is within the segmented water surface mask, otherwise it is determined that no one is drowning.

[0013] Furthermore, the Neck part of the improved YOLOv8 model is composed of several bidirectional feature pyramid networks, which add an additional weight to each input feature and allow features to be fused in both top-down and bottom-up directions.

[0014] The present invention also discloses a computer-readable storage medium, on which a computer program is stored. The computer program is executed by a processor to implement any of the above-mentioned methods for detecting a person falling into water.

[0015] The technical solution of the present invention has at least the following technical effects or advantages: 1. The present invention improves the YOLOv8 algorithm so that it can effectively represent and process multi-scale features, thereby accurately and efficiently detecting people falling into water.

[0016] 2. The present invention improves the trade-off between accuracy and efficiency through efficient bidirectional cross-scale connections and repeated block structures. BiFPN allows feature information to flow bidirectionally between different scales through a bidirectional path. This bidirectional flow can be regarded as an effective information exchange between different scales. This design aims to improve the efficiency and effectiveness of feature fusion by strengthening the bidirectional flow of features, thereby improving the performance of target detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a flow chart of judging whether a person falls into water in one embodiment of the present invention. DETAILED DESCRIPTION

[0018] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is described in detail through specific embodiments. However, it should be understood that the provision of specific embodiments is only for a better understanding of the present invention, and they should not be understood as limitations of the present invention. In the description of the present invention, it should be understood that the terms used are only for the purpose of description and cannot be understood as indicating or implying relative importance.

[0019] YOLOv8 is built as a unified framework for training object detection, instance segmentation, and image classification models. It provides a brand new SOTA model. The original YOLOv8n network is divided into the backbone network Backbone, Neck part and the detection head Head part, including convolution module, C2f module, SPPF module, upsampling module and splicing module. The backbone network adopts the C2f structure and adjusts the number of channels for different scale models. The operations such as Split in this C2f module have disadvantages for specific hardware deployment.

[0020] In order to solve the problems existing in the prior art, such as the substantial increase in model calculation amount, slow detection speed, limited application, and difficulty in application in certain application scenarios such as mobile terminals or embedded devices, the present invention proposes a method, system and readable medium for detecting people falling into water. On the basis of the original YOLOv8, BiFPN bidirectional cross-scale connection and weighted feature fusion have better accuracy and efficiency trade-offs. BiFPN: Introduces learnable weights to learn the importance of different input features, while repeatedly applying top-down and bottom-up multi-scale feature fusion, and improves the trade-off between accuracy and efficiency through efficient bidirectional cross-scale connections and repeated block structures. BiFPN allows feature information to flow bidirectionally between different scales through a bidirectional path. This bidirectional flow can be regarded as effective information exchange between different scales. The scheme of the present invention is described in detail below through examples.

[0021] Embodiment 1 This embodiment discloses a method for detecting a person falling into water. Figure 1 As shown, the following steps are included: S1 obtains a video or image of the water surface to be measured; S2 uses the improved YOLOv8 model to segment the video or image of the water surface and obtain the water surface mask; The Neck part of the improved YOLOv8 model consists of several bidirectional feature pyramid networks. The bidirectional feature pyramid network adds an additional weight to each input feature and allows the features to be fused in both the top-down and bottom-up directions. The weight is a scalar, vector, or multidimensional tensor. The scalar is the feature of the image, the vector is the data for each channel, and the multidimensional tensor is the data for each pixel.

[0022] S3 uses the YOLOv7 model to detect pedestrians in videos or images on the water surface; Figure 1 As shown in the figure, if the YOLOv7 model detects a pedestrian and the pedestrian is within the segmented water surface mask, it is judged that someone is drowning, otherwise it is judged that no one is drowning.

[0023] In another embodiment of the present invention, the Neck part of the improved YOLOv8 model is composed of a MobileNetV3 network or a RevColV1 reversible network.

[0024] Embodiment 2 Based on the same inventive concept, this embodiment discloses a system for detecting a person falling into water, comprising: An image acquisition module, used to obtain a video or image of the water surface to be measured; An image segmentation module is used to segment the video or image of the water surface through an improved YOLOv8 model to obtain a water surface mask; The Neck part of the improved YOLOv8 model is composed of several bidirectional feature pyramid networks, which add an additional weight to each input feature and allow features to be fused in both top-down and bottom-up directions.

[0025] Pedestrian detection module, used to detect pedestrians in videos or images on the water surface using the YOLOv7 model; The drowning detection module is used to detect pedestrians in the YOLOv7 model. If the pedestrians are within the segmented water surface mask, it is judged that someone is drowning, otherwise it is judged that no one is drowning.

[0026] Embodiment 3 Based on the same inventive concept, this embodiment discloses a computer-readable storage medium, on which a computer program is stored. The computer program is executed by a processor to implement any of the above-mentioned methods for detecting a person falling into water.

[0027] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0028] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0029] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0030] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0031] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit them. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation of the present invention can still be modified or replaced by equivalents, and any modification or equivalent replacement that does not deviate from the spirit and scope of the present invention should be included in the protection scope of the claims of the present invention. The above content is only a specific implementation of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or replacements within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for detecting a person falling into water, characterized in that: The following steps are involved: Obtain a video or image of the water surface to be measured; The video or image of the water surface is segmented by using an improved YOLOv8 model to obtain a water surface mask; Perform pedestrian detection in the video or image of the water surface by using a YOLOv7 model; If the YOLOv7 model detects a pedestrian and the pedestrian is within the segmented water surface mask, it is determined that someone is drowning, otherwise it is determined that no one is drowning.

2. The method for detecting a person falling into water according to claim 1, characterized in that: The Neck part of the improved YOLOv8 model consists of several bidirectional feature pyramid networks.

3. The method for detecting a person falling into water according to claim 2, characterized in that: The bidirectional feature pyramid network adds an additional weight to each input feature and allows features to be fused in both top-down and bottom-up directions.

4. The method for detecting a person falling into water according to claim 3, characterized in that: The weight is a scalar, a vector or a multi-dimensional tensor.

5. The method for detecting a person falling into water according to claim 4, characterized in that: The scalar is a feature of the image, the vector is the data of each channel, and the multidimensional tensor is the data of each pixel.

6. The method for detecting a person falling into water according to any one of claims 2 to 5, characterized in that: The Neck part of the improved YOLOv8 model consists of a MobileNetV3 network or a RevColV1 reversible network.

7. A person falling into water detection system, characterized in that: include: An image acquisition module, used to obtain a video or image of the water surface to be measured; An image segmentation module, used to segment the video or image of the water surface through an improved YOLOv8 model to obtain a water surface mask; A pedestrian detection module, used for detecting pedestrians in the video or image of the water surface by using a YOLOv7 model; The drowning detection module is used to determine that a person is drowning if a pedestrian is detected by the YOLOv7 model and the pedestrian is within the segmented water surface mask, otherwise it is determined that no one is drowning.

8. The person falling into water detection system according to claim 7, characterized in that: The Neck part of the improved YOLOv8 model is composed of several bidirectional feature pyramid networks, which add an additional weight to each input feature and allow features to be fused in both top-down and bottom-up directions.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method for detecting a person falling into water as described in any one of claims 1 to 6.

Images