Water falling identification method based on artificial intelligence

Abstract

The invention provides a falling-into-water identification method based on artificial intelligence, and belongs to the technical field of target identification. Preprocessing the data; water area detection; target detection and posture recognition; behavior recognition and early warning; identifying a drowning state; and real-time alarm and cloud processing are carried out. By combining target detection, human body posture recognition, behavior recognition and early warning technologies, the invention provides an efficient, accurate and real-time drowning recognition method, various defects in the prior art are overcome, and the intelligent level of water area safety monitoring is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of target recognition, and in particular to a method for identifying falling into water based on artificial intelligence. Background Art

[0002] With the increase of water recreational activities and the increasing prominence of water safety issues, how to effectively monitor and identify falling into the water has become an urgent problem to be solved. At present, most water safety monitoring relies on manual patrols and traditional surveillance cameras. However, this method has many shortcomings:

[0003] Limitations of manual patrols: Manual patrols require a lot of human resources and are difficult to monitor 24 hours a day. Even if monitoring equipment is installed, there are still problems such as fatigue and distraction of personnel, which can easily lead to failure to detect and handle drowning incidents in a timely manner.

[0004] Disadvantages of traditional surveillance cameras: Although traditional surveillance cameras can provide video images, they lack intelligent analysis functions and cannot automatically identify and alarm water-falling incidents. They still need to rely on manual monitoring, which is inefficient.

[0005] Limitations of existing automatic recognition technologies: Although some automatic recognition technologies based on computer vision have been applied to water fall detection, most of these technologies rely on rules and settings or simple image processing methods, with low recognition accuracy and difficulty in coping with complex water environments and changing lighting conditions.

[0006] Unable to respond in real time: Existing technologies make it difficult to respond in real time. The rescue window after a person falls into the water is very small. If it is not discovered in time, the rescue will be delayed. However, the water fall recognition based on artificial intelligence can immediately respond to the alarm and even make a prediction in advance, significantly improving the efficiency and success rate of rescue.

[0007] Therefore, there is an urgent need in the art for a technical solution that can understand the action or state of a person falling into water in a video image and realize efficient and accurate automatic recognition and alarm of falling-into-water events.

[0008] The information disclosed in this background technology section is only intended to enhance the understanding of the overall background of the invention and should not be regarded as an acknowledgment or any form of suggestion that the information constitutes the prior art already known to a person skilled in the art. Summary of the invention

[0009] The purpose of the present invention is to provide a more intelligent water safety monitoring solution.

[0010] To achieve the above object, the present invention provides the following solutions:

[0011] A method for identifying falling into water based on artificial intelligence, comprising:

[0012] Video data collection;

[0013] Data preprocessing;

[0014] Water area inspection;

[0015] Object detection and gesture recognition;

[0016] Behavior recognition and early warning;

[0017] Identification of the state of falling into water;

[0018] Real-time alarm and cloud processing.

[0019] Optionally, the video data acquisition includes:

[0020] High-resolution cameras are used to monitor the target waters 24 hours a day and collect video data in real time.

[0021] Optionally, the data preprocessing specifically preprocesses the collected video data, including denoising, deblurring, and brightness adjustment to ensure image quality.

[0022] Optionally, the water area detection includes:

[0023] Train a deep learning model to detect and segment water areas in videos.

[0024] Optionally, the target detection and posture recognition include:

[0025] Use deep learning models to detect targets and identify and annotate people in video footage;

[0026] Use human pose estimation models to identify key joints of people in videos;

[0027] Combining the target detection and posture recognition results, it provides the personnel’s location information, motion trajectory and joint point data.

[0028] Optionally, the behavior recognition and warning include:

[0029] Through the pre-trained behavior recognition model, the movement trajectory and joint point data of the personnel are analyzed to identify whether there is a falling action or state;

[0030] By tracking the direction and speed of people's movement and the relationship with the water area, algorithms or models are used to make predictions and early warnings;

[0031] When people are close to water or at risk of falling into the water, the system will issue an on-site alarm and shout to remind them.

[0032] Optionally, the water-falling state identification includes:

[0033] Analyze the person who enters the video screen and determine whether he or she is falling into the water through target detection and human joint recognition, even if the person's falling into the water is not captured;

[0034] Utilize the relationship between water area masks and personnel locations to identify people floating or swimming in the water, and issue alarms and handle situations in a timely manner.

[0035] Optionally, the real-time alarm and cloud processing include:

[0036] Once the falling-into-water action or status is identified, the system immediately sends out an alarm signal to notify relevant personnel for emergency rescue;

[0037] In a multi-camera monitoring terminal, the images with warning or alarm information will be automatically prioritized and displayed in the front;

[0038] Upload video data and recognition results to cloud servers for further processing and analysis;

[0039] Based on big data and machine learning technology, the recognition model is continuously optimized and updated to improve the accuracy and robustness of the system.

[0040] Optionally, the deep learning models include: YOLO, SSD, Faster, R-CNN; the human posture estimation models include: OpenPose, HRNet.

[0041] Optionally, the behavior recognition model includes: LSTM, 3D-CNN.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] 1. Efficient and accurate: The use of deep learning models for target detection and posture recognition significantly improves the accuracy and efficiency of identifying falling into the water, and reduces false alarms and missed alarms.

[0044] 2. Real-time response: The system can monitor and respond to falling into water events in real time, and immediately send out an alarm signal when people fall into water, shortening the rescue time window and improving the rescue success rate.

[0045] 3. Intelligent prediction: By analyzing the movement trajectory and speed of people, as well as their relationship with the water boundary, the system can predict the potential risk of falling into the water in advance and issue early warnings to effectively prevent the occurrence of falling into the water.

[0046] 4. Comprehensive coverage: The system can not only identify the movements during the falling process, but also identify those people who are already in the state of falling into the water in the picture, ensuring comprehensive coverage of the monitoring range.

[0047] 5. High degree of automation: The system realizes fully automated intelligent identification and alarm, without relying on manual monitoring, reducing labor costs and improving monitoring efficiency.

[0048] 6. Strong adaptability: The system can adapt to different water environments and complex lighting conditions, has strong environmental adaptability, and is suitable for a variety of scenarios such as rivers, reservoirs, lakes, beaches, ships, private properties and industrial facilities.

[0049] 7. Cloud processing and model optimization: The system performs further data processing and analysis through cloud servers, continuously optimizes and updates the recognition model, and improves the accuracy and robustness of the system.

[0050] The present invention provides an efficient, accurate and real-time method for identifying falling into water by combining target detection, human posture recognition, behavior recognition and early warning technology, which solves many deficiencies in the prior art and significantly improves the intelligent level of water safety monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0052] Figure 1 A schematic flow chart of a method for identifying a fall into water based on artificial intelligence provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0053] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0054] The purpose of the present invention is to provide a method that can realize automatic recognition and alarm of the action or status of a person falling into water in a video screen, so as to improve the efficiency and accuracy of water area safety monitoring, reduce the risk of people falling into water, and ensure personal safety.

[0055] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0056] Embodiment 1:

[0057] This embodiment provides a method for water fall recognition based on artificial intelligence. The hardware can use mainstream computers or embedded development boards on the market, such as Jetson Nano. The specific steps are as follows:

[0058] Video data collection:

[0059] High-resolution cameras are used to monitor the target waters 24 hours a day and collect video data in real time.

[0060] Data preprocessing:

[0061] The collected video data is preprocessed, including denoising, deblurring, brightness adjustment, etc., to ensure image quality.

[0062] The video data is decomposed into continuous image frames, and inter-frame differences are performed to detect whether there are moving targets. Real-time water fall recognition is performed only when a moving target is found. At the same time, the inter-frame differences are used to determine the specific pixel area that has changed, and a binary mask image of the moving target is generated. 1 represents the area with changes, and 0 represents no changes. The original image is cropped according to the mask range of 1 and scaled to the fixed resolution required by the model (such as 640x640 commonly used in YOLO models). These regions of interest (ROIs) are then input into the target detection or posture recognition model for further processing. This avoids calculating the full frame and only focuses on the area of ​​motion changes, optimizing the efficiency of the model.

[0063] Water area detection:

[0064] Train a deep learning model (such as swintransformer) to detect and segment the water area in the video. In order to better achieve accurate water segmentation, we must first prepare water data sets of various weather, lighting, and angles before training, and then train under the swintransformer framework, which can calculate the self-attention between pixels and better segment the water area. After training the model, we can deploy it to run under the TensorRT framework. Because this example is running on jetson nano hardware, the framework has more efficient reasoning, so the following models will be deployed to run under this framework.

[0065] Determine the boundary of the water area and generate a binary mask of the water area, where 1 represents the water area and 0 represents the non-water area, which is used for subsequent detection of people approaching the water area.

[0066] Object detection and gesture recognition:

[0067] Train a deep learning model (such as YOLO) for target detection, identify and label people in the video. In order to achieve more accurate person recognition, in addition to conventional human data sets, it is also necessary to collect more data sets of people with only their heads or hands exposed in the water. This makes it easier to distinguish between people who have fallen into the water and floating objects on the water.

[0068] Train a human pose estimation model (such as OpenPose, HRNet, etc.) to identify the key joints of people in the video.

[0069] Combined with the target detection and posture recognition results, and using tracking algorithms (such as Deep SORT), it provides single-target or multi-target personnel location information, joint point information, and personnel motion trajectory, direction, and speed data.

[0070] Behavior recognition and early warning:

[0071] Through the pre-trained behavior recognition model (such as LSTM), the motion data and joint point data of the person are analyzed to identify whether there is a falling action or state. LSTM is a neural network suitable for time series data. It takes the motion trajectory and joint point information of the person as input and trains the LSTM model to predict the falling behavior. The state update formula of LSTM is:

[0072] H t =LSTM(X t ,H t-1 )

[0073] Among them, H t is the hidden state at the tth time step, X t is the input of the tth frame (such as the position and posture data of a person).

[0074] By tracking the angle between the direction of movement of the person and the water area (or shoreline), combined with the person's current movement speed, an algorithm or model is used for prediction and warning. For example, a simple uniform linear motion model can predict the future position of the target based on its current position, speed and direction, and can predict whether there is a risk of falling into the water through a preset safety threshold.

[0075] In addition, the risk of falling into the water can also be warned based on the distance of the person from the shoreline. The distance from the shoreline is calculated based on the bottom coordinates of the person's feet or the target detection frame, and the risk of falling into the water is determined by judging whether it exceeds the set safety distance threshold.

[0076] When a person is about to fall into the water, is close to the water, or is at risk of falling into the water, the system will issue an on-site alarm and shout to remind them.

[0077] Identification of the water-falling state:

[0078] Analyze the people who enter the video screen, and use the position information and water mask information obtained through target detection and human joint recognition to judge whether they are in the state of falling into the water. For example, whether the target detection frame of the person is completely within the range of mask 1 can be used to judge whether they are in the state of falling into the water, even if the process of the person falling into the water is not captured.

[0079] Real-time alarm and cloud processing:

[0080] Once the action or state of falling into the water is recognized, the system will immediately send out an alarm signal to notify relevant personnel for emergency rescue. The alarm method can be a live voice broadcast, or an audible and visual alarm in the monitoring room to remind security personnel. In addition, in a monitoring terminal with multiple cameras, the pictures with warning or alarm information will be automatically prioritized and displayed in the front, because in a system with dozens or hundreds of cameras, it is often impossible to display all of them, or personnel do not have enough energy to observe all the pictures.

[0081] Upload the video data and recognition results to the cloud server for further processing and analysis.

[0082] Based on big data and machine learning technology, the recognition model is continuously optimized and updated to improve the accuracy and robustness of the system.

[0083] In view of the various deficiencies in the prior art, this embodiment provides a method for identifying a person falling into water based on artificial intelligence, which specifically solves the following technical problems:

[0084] Low recognition accuracy: Existing water fall detection technology relies on simple image processing methods or rule settings, which are difficult to cope with complex water environments and changing lighting conditions, resulting in low recognition accuracy. The present invention improves the recognition accuracy of water fall events by introducing a deep learning model.

[0085] Lack of real-time performance: Traditional monitoring systems are difficult to achieve real-time response to drowning events. The rescue time window after a person falls into the water is very small. If it is not discovered in time, rescue will be delayed. The present invention achieves real-time response and alarm to drowning events through real-time video monitoring and intelligent identification.

[0086] Low efficiency of manual monitoring: The traditional method of relying on manual monitoring has problems such as fatigue and distraction of personnel, which easily leads to failure to timely discover and handle the falling into the water. The present invention adopts a fully automated intelligent recognition system, which greatly improves the monitoring efficiency.

[0087] No early warning: The existing technology cannot predict possible water-falling events in advance and can only identify and warn after the event occurs. The present invention can identify potential water-falling risks in advance and issue early warnings by analyzing the movement trajectory and posture of the person.

[0088] Lack of recognition of people who have fallen into water: The existing technology is difficult to identify people who have fallen into water in the picture but whose falling process is not captured. The present invention can identify people floating or swimming in the water through water area detection and human posture recognition, and issue an alarm in time.

[0089] Advantages of this embodiment:

[0090] Compared with the prior art, the present invention has the following advantages:

[0091] Efficient and accurate: The use of deep learning models for target detection and posture recognition significantly improves the accuracy and efficiency of identifying falling into the water, reducing false alarms and missed alarms.

[0092] Real-time response: The system can monitor and respond to falling into water events in real time, and immediately send out an alarm signal when people fall into water, shortening the rescue time window and increasing the rescue success rate.

[0093] Intelligent prediction: By analyzing the movement trajectory and speed of people, as well as their relationship with the water boundary, the system can predict the potential risk of falling into the water in advance and issue early warnings to effectively prevent the occurrence of falling into the water.

[0094] Comprehensive coverage: The system can not only identify the movements during the falling-in process, but also identify those who are already in the water in the picture, ensuring comprehensive coverage of the monitoring range.

[0095] High degree of automation: The system realizes fully automated intelligent identification and alarm, without relying on manual monitoring, reducing labor costs and improving monitoring efficiency.

[0096] Strong adaptability: The system can adapt to different water environments and complex lighting conditions, has strong environmental adaptability, and is suitable for a variety of scenarios such as rivers, reservoirs, lakes, beaches, ships, private properties and industrial facilities.

[0097] Cloud processing and model optimization: The system performs further data processing and analysis through cloud servers, continuously optimizes and updates the recognition model, and improves the accuracy and robustness of the system.

[0098] The present invention provides an efficient, accurate and real-time method for identifying falling into water by combining target detection, human posture recognition, behavior recognition and early warning technology, which solves many deficiencies in the prior art and significantly improves the intelligent level of water safety monitoring.

[0099] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0100] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A method for identifying falling into water based on artificial intelligence, characterized in that: include: Video data collection; Data preprocessing; Water area inspection; Object detection and gesture recognition; Behavior recognition and early warning; Identification of the state of falling into water; Real-time alarm and cloud processing.

2. The method for identifying falling into water based on artificial intelligence according to claim 1, characterized in that: The video data collection includes: High-resolution cameras are used to monitor the target waters 24 hours a day and collect video data in real time.

3. The method for identifying falling into water based on artificial intelligence according to claim 1, characterized in that: The data preprocessing specifically preprocesses the collected video data, including denoising, deblurring, and brightness adjustment, to ensure image quality.

4. The method for identifying falling into water based on artificial intelligence according to claim 1, characterized in that: The water area detection includes: Train a deep learning model to detect and segment water areas in videos.

5. The method for identifying falling into water based on artificial intelligence according to claim 1, characterized in that: The target detection and posture recognition include: Use deep learning models to detect targets and identify and annotate people in video footage; Use human pose estimation models to identify key joints of people in videos; Combining the target detection and posture recognition results, it provides the personnel’s location information, motion trajectory and joint point data.

6. The method for identifying falling into water based on artificial intelligence according to claim 1, characterized in that: The behavior recognition and early warning include: Through the pre-trained behavior recognition model, the movement trajectory and joint point data of the personnel are analyzed to identify whether there is a falling action or state; By tracking the direction and speed of people's movement and the relationship with the water area, algorithms or models are used to make predictions and early warnings; When people are close to water or at risk of falling into the water, the system will issue an on-site alarm and shout to remind them.

7. The method for identifying falling into water based on artificial intelligence according to claim 1, characterized in that: The water-falling state recognition includes: Analyze the person who enters the video screen and determine whether he or she is falling into the water through target detection and human joint recognition, even if the person's falling into the water is not captured; Utilize the relationship between water area masks and personnel locations to identify people floating or swimming in the water, and issue alarms and handle situations in a timely manner.

8. The method for identifying falling into water based on artificial intelligence according to claim 1, characterized in that: The real-time alarm and cloud processing include: Once the falling-into-water action or status is identified, the system immediately sends out an alarm signal to notify relevant personnel for emergency rescue; In a multi-camera monitoring terminal, the images with warning or alarm information will be automatically prioritized and displayed in the front; Upload video data and recognition results to cloud servers for further processing and analysis; Based on big data and machine learning technology, the recognition model is continuously optimized and updated to improve the accuracy and robustness of the system.

9. The method for identifying falling into water based on artificial intelligence according to claim 5, characterized in that: The deep learning models include: YOLO, SSD, Faster, R-CNN; the human posture estimation models include: OpenPose, HRNet.

10. The method for identifying falling into water based on artificial intelligence according to claim 6, characterized in that: The behavior recognition model includes: LSTM and 3D-CNN.

Images