Intelligent anti-drowning early warning system based on multi-sensor fusion

Abstract

The invention belongs to the technical field of anti-drowning detection systems, and discloses an intelligent anti-drowning early warning system based on multi-sensor fusion, which comprises an image acquisition device, a display device, a server and an alarm device arranged in a natatorium, and is characterized in that the server receives image data of the image acquisition device and displays the image data on the display device; the state sensing module is worn on a swimmer and used for collecting posture information, water level information and position information of the swimmer in real time, the state sensing module comprises a posture sensor, a liquid level sensor and a UWB positioning system, the posture sensor obtains angle, acceleration and angular velocity data of the swimmer in real time, and the liquid level sensor obtains the angle, acceleration and angular velocity data of the swimmer in real time; the liquid level sensor obtains water level information of a swimmer in real time. Compared with a traditional active rescue method, the system does not need human intervention, can automatically give an alarm under the conditions that a drowning person cannot call for help and is out of consciousness and the like, and accurately provides position information for rescue workers.

Description

Technical Field

[0001] The present invention relates to the technical field of drowning prevention detection systems, and more specifically, to an intelligent drowning prevention warning system based on multi-sensor fusion. Background Art

[0002] Drowning detection systems based on science and technology can be divided into two categories: systems based on image processing technology and systems based on sensors. Drowning detection based on image processing technology uses a camera to capture the movements and postures of people in the water, and then uses computer vision technology to analyze in real time whether the swimmer is drowning. Sensor-based drowning detection technology mainly uses a wearable sensor system to integrate heart rate sensors, water level sensors, and acceleration sensor data to determine whether the swimmer is drowning.

[0003] The defects of image processing technology-based systems and sensor-based systems are as follows:

[0004] (1) The practical application and promotion of the system based on image processing technology have certain limitations. This is because the method usually requires the video to fully cover the swimming scene and is easily affected by the external environment. For example, light, occlusion between people, and the inability to accurately identify and track swimmers in turbid water. Obviously, the use of this technology to achieve drowning warning is not only complex and expensive to deploy, but also requires long-term maintenance. This cannot be popularized in some underdeveloped low-income areas where the drowning mortality rate is high all year round.

[0005] (2) The core of the sensor-based judgment of whether a swimmer is drowning is set according to the physiological changes of the human body after drowning. For example, when the drowning time is less than or equal to 1 minute, the heart rate will increase. However, other physiological values ​​such as heart rate have individual differences, and it is impossible to accurately define the threshold value.

[0006] (3) Currently, the method of judging whether a swimmer is drowning is based on sensors. When a water level sensor is used, the water level at the wrist is measured, which has a low accuracy rate.

[0007] (4) The existing technology is unable to accurately locate the drowning person, and cannot ensure the timeliness of rescue and improve the survival rate of the drowning person.

[0008] Therefore, in order to ensure the accuracy, timeliness, reliability and stability of anti-drowning warning detection, an intelligent anti-drowning warning system based on multi-sensor fusion is proposed. Summary of the invention

[0009] The purpose of the present invention is to solve the shortcomings of the prior art and to propose an intelligent drowning prevention warning system based on multi-sensor fusion.

[0010] To solve the above problems, the present invention adopts the following technical solutions:

[0011] Intelligent drowning prevention warning system based on multi-sensor fusion, including:

[0012] An image acquisition device, a display device, a server and an alarm device are arranged in the swimming pool, wherein the server receives image data from the image acquisition device and displays the image data on the display device;

[0013] A state perception module, which is worn on the swimmer and is used to collect the swimmer's posture information, water level information and position information in real time. The state perception module includes a posture sensor, a liquid level sensor and a UWB positioning system. The posture sensor obtains the swimmer's angle, acceleration and angular velocity data in real time, the liquid level sensor obtains the swimmer's water level information in real time, and the UWB positioning system obtains the swimmer's position information in real time;

[0014] A communication module, the communication module is used to transmit the posture information, water level information and position information data of the swimmer monitored by the state sensing module to the server in real time;

[0015] The server comprises:

[0016] State monitoring module: The state monitoring module acquires and processes the information data collected by the state perception module in real time, monitors the swimmer's abnormal posture, abnormal water level, heart rate, blood oxygen saturation and location information to determine whether the swimmer is drowning and the location information;

[0017] The alarm module includes an active alarm mode and an automatic alarm mode. The active alarm mode is a one-button calling device worn on the swimmer. When the detected person drowns, the drowning person's location information is sent to the server and displayed on the display device and an alarm is issued through the alarm device.

[0018] As a further description of the above technical solution: the posture sensor is a nine-axis posture sensor.

[0019] As a further description of the above technical solution: the liquid level sensor is a non-contact liquid level sensor, and the liquid level sensor is worn on the back of the swimmer's head and at the same level as the nostrils through a wearable device.

[0020] As a further description of the above technical solution: the UWB positioning system includes multiple UWB base stations arranged on the shore of the swimming pool and UWB positioning tags arranged on the swimmer's wearable device. The UWB positioning tags send out UWB positioning data and are received by each UWB base station. The UWB base station sends the received positioning data to the server.

[0021] As a further description of the above technical solution: the server includes a status display module, and the status display module displays swimmer information, and the swimmer information includes user name, emergency contact phone number, current swimming style and swimming time.

[0022] As a further description of the above technical solution: the state perception module includes a vital signs collection module, and the vital signs collection module is used to monitor the swimmer's heart rate, blood oxygen, and body temperature.

[0023] As a further description of the above technical solution: the communication module is a Lora wireless communication module.

[0024] As a further description of the above technical solution: the alarm device includes a swimming place broadcast and a lifeguard watch / mobile phone.

[0025] As a further description of the above technical solution: the abnormal posture of the swimmer includes that the average interval between the swimmer's frequency of entering and exiting the water and the frequency of raising and lowering the head exceeds the set time threshold, the head-down posture that does not appear in normal swimming exceeds the set time threshold, and the nearly vertical posture exceeds the set time threshold.

[0026] As a further description of the above technical solution: the water level abnormality information is when the swimmer's mouth and nose are in the water for more than a set time threshold

[0027] Compared with the prior art, the advantages of the present invention are:

[0028] 1. Compared with the traditional active rescue method, this system does not require human intervention. It can automatically send out an alarm when the drowning person is unable to call for help or loses consciousness, and provide accurate location information to rescue personnel. It can quickly and accurately assess the current state of the swimmer, make decisions in time, and increase the chance of rescue.

[0029] Second, this solution makes a judgment on the swimmer's current situation by calculating the swimmer's posture, position and other information. At the same time, it combines active alarm mode, which is more reliable and intelligent, and improves the survival rate.

[0030] 3. This solution is equipped with additional functions such as swimming style recognition, water entry and exit interval calculation, and swimming path calculation to provide information support for swimmers. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 One of the system diagrams of the present invention;

[0032] Figure 2 This is the second system diagram of the present invention;

[0033] Figure 3 is a flow chart of the present invention;

[0034] Figure 4 This is a schematic diagram of a large-screen display screen in a swimming pool showing the position of a swimmer in an embodiment of the present invention;

[0035] Figure 5 The swimmer's state is determined and the corresponding waveform diagram in Embodiment 2 of the present invention;

[0036] Figure 6 Schematic diagram of a relatively ideal swimming posture waveform of a swimmer in Embodiment 2 of the present invention;

[0037] Figure 7 A schematic diagram of a waveform smoothed by the present invention;

[0038] Figure 8 This is a peak / trough schematic diagram of the present invention. DETAILED DESCRIPTION

[0039] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention; it is obvious that the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments, and all other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making creative work are within the scope of protection of the present invention.

[0040] Example 1

[0041] See also Figure 1-3 The intelligent drowning prevention warning system based on multi-sensor fusion includes: image acquisition equipment, display device, server, alarm device, state perception module and communication module set in the swimming pool. The server receives the image data of the image acquisition equipment and displays it on the display device. The image acquisition equipment is a high-definition camera that collects the image data of the swimming pool in real time. The display device is a large-screen display of the swimming pool. The alarm device includes a swimming place broadcast and a lifeguard watch / mobile phone.

[0042] Among them, the state perception module is worn on the swimmer and is used to collect the swimmer's posture information, water level information and location information in real time. The state perception module includes a posture sensor, a liquid level sensor, a UWB positioning system and a vital signs collection module. The vital signs collection module is used to monitor the swimmer's heart rate, blood oxygen and body temperature.

[0043] The attitude sensor is a JY901S nine-axis attitude sensor, which acquires the swimmer's angle, acceleration, angular velocity and other data in real time. Most traditional swimming styles are performed in a rhythm of breathing, with the head raised out of the water and the head lowered into the water. If drowning occurs. For example, if you have cramps, the swimmer's head raises and lowers, and the breathing rhythm is easily disrupted, especially in the case of sudden physical discomfort such as choking, the frequency of raising and lowering the head will change greatly. This solution can determine whether the swimmer is in a normal swimming posture by detecting the swimmer's average head-raising and head-lowering intervals + water-entry and exit intervals.

[0044] The liquid level sensor obtains the swimmer's water level information in real time. The liquid level sensor is an XKC-Y21-NPN non-contact liquid level sensor that can quickly and accurately measure the liquid level, provide precise liquid level information, and has a small size. The liquid level sensor is worn on the back of the swimmer's head through a wearable device and is level with the nostrils. Compared with other locations such as the wrist, this position has higher reliability and accuracy.

[0045] When someone is drowning, it is very important to know the location of the drowning person accurately at the first time. Taking into account the problem of positioning accuracy, the present invention chooses to use ultra-wideband (UWB, Ultra Wi de Band) positioning technology. Since UWB technology is a new type of wireless communication technology, it has the advantages of high precision (centimeter level), low power consumption, high safety and low cost. The system is implemented based on the DWM1000 module. The UWB positioning system includes a plurality of UWB base stations arranged on the shore of the swimming pool and a UWB positioning tag arranged on the swimmer's wearable device. The UWB positioning tag sends UWB positioning data and is received by each UWB base station. The UWB base station sends the received positioning data to the server.

[0046] Communication module: The communication module is used to transmit the swimmer's posture information, water level information and location information data monitored by the state sensing module to the server in real time. The communication module is a Lora wireless communication module. LoRa is a technology suitable for long-distance, low-power and anti-interference wireless communication, which can ensure the effectiveness of communication in crowded places.

[0047] The server includes a status monitoring module, a status display module and an alarm module.

[0048] The status display module displays the swimmer's information, including the user name, emergency contact number, current swimming style and swimming time.

[0049] The state monitoring module acquires and processes the information data collected by the state perception module in real time, monitors the swimmer's abnormal posture, abnormal water level, heart rate, blood oxygen saturation and location information to determine whether the swimmer is drowning and the location information.

[0050] This module is equivalent to the brain of the system, processing the information collected by the state perception module in real time. If any of the information is abnormal, it will be judged that the swimmer is drowning (such as Figure 3 shown).

[0051] Among them, abnormal postures of swimmers include the average interval between the swimmer's frequency of entering and exiting the water and the frequency of raising and lowering the head exceeding the set time threshold (500ms), postures that do not appear in normal swimming, such as head down and backward, exceeding the set time threshold (10s), and postures close to vertical (setting range is greater than or equal to 75° and less than or equal to 110°) exceeding the set time threshold (10s).

[0052] The water level abnormality information is set as drowning when the swimmer's mouth and nose are in the water for more than the set time threshold. For heart rate abnormality, referring to the existing technology, the heart rate high and low thresholds are set to 150bpm and 45bpm respectively. Blood oxygen saturation abnormality, when the blood oxygen saturation is less than or equal to 90%, it is drowning. Position information abnormality is an alarm when the swimmer is in a static posture (unconscious) in the water for more than the set time threshold (10s).

[0053] The alarm module includes active alarm mode and automatic alarm mode. The active alarm mode is a one-button call device worn by the swimmer. When the detected person drowns, the drowning person's location information is immediately sent to the swimming pool's large screen display and the lifeguard's watch / mobile phone, and the swimming venue broadcast is used to alarm.

[0054] The large pool screen can show the swimmer's position in real time, such as Figure 4 As shown in the figure, this function is implemented based on HTML5+CSS3. The display screen presents a swimming pool model and displays the positions of swimmers 1 and 2 in the swimming pool in real time. When swimmer 2 is drowning, it is highlighted on the screen and an alarm message is displayed on the large screen.

[0055] Example 2

[0056] Based on the above embodiment, this solution proposes to use the interval between the swimmer's head raising and head lowering as a criterion for determining whether the swimmer is drowning. Figure 5 As shown in the waveform below, there are three waveforms in this part, where green represents whether the person is in the water (this test chart does not enter the water so the green waveform is a straight line), yellow represents the average head-down interval, and blue represents the average head-up interval. When the swimmer is in a normal state, the head-up and head-down waveforms are in a close fit. If the head-up and head-down waveforms are offset, it represents an abnormal state. The data is obtained from the nine-axis attitude sensor, and an algorithm is designed to extract the head-up and head-down information from the acquired data. The specific algorithm is implemented as follows:

[0057] The data from the nine-axis attitude sensor in this solution will be input into the server for further calculation. The information such as the swimmer's attitude, head-raising and head-lowering frequency will be collected, and the average interval between the swimmer's head-raising and head-lowering will be calculated, so as to further analyze whether the swimmer is drowning. Figure 6 The waveform in the figure is the angle change waveform of the sensor around the X-axis when the swimmer is doing breaststroke in an ideal swimming posture (about 5s, sampling frequency 20Hz, a total of 100). Note: One line is the angle record within 5s. The microcontroller calculates the data of the last 5s every 1s and appends new data.

[0058] To identify the specific posture of the swimmer, it is necessary to identify the peaks (raising head) and troughs (lowering head) in the waveform. This solution uses a relatively simple algorithm - the second-order difference method. First, the input data is smoothed to eliminate the burrs in the waveform to facilitate subsequent identification ( Figure 7 ). In this way, the sampling point sequence is obtained: F = [f1, f2, f3, ..., f n ], where n∈[1,2,3,…,N]. Then perform the first-order difference: Diff=f i+1 -f i , where i∈1,2,3,…N-1. Since the specific differential amplitude is not important, the differential data Diff is normalized to -1,0,1 using the sign function. The function is as follows:

[0059]

[0060] The point with a differential value of 0 is the same point. If the comparison rule is used, peak detection may fail and more conditions are required. Here, we directly set the same point 0 to the previous non-0 (leading slope processing).

[0061]

[0062] Finally, the second-order difference of Diff is -2 / +2, which is the peak / trough ( Figure 8 Example). The above is the peak / trough recognition on the X-axis direction. The Y-axis algorithm is similar. Then, by comparing the fluctuation amplitudes of the X-axis and the Y-axis, it can be determined whether it is freestyle or breaststroke.

[0063] The above is only a preferred specific implementation of the present invention; however, the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical solution and its improved conception within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. Intelligent drowning prevention warning system based on multi-sensor fusion, characterized by: include: An image acquisition device, a display device, a server and an alarm device are arranged in the swimming pool, wherein the server receives image data from the image acquisition device and displays the image data on the display device; A state perception module, which is worn on the swimmer and is used to collect the swimmer's posture information, water level information and position information in real time. The state perception module includes a posture sensor, a liquid level sensor and a UWB positioning system. The posture sensor obtains the swimmer's angle, acceleration and angular velocity data in real time, the liquid level sensor obtains the swimmer's water level information in real time, and the UWB positioning system obtains the swimmer's position information in real time; A communication module, the communication module is used to transmit the posture information, water level information and position information data of the swimmer monitored by the state sensing module to the server in real time; The server comprises: State monitoring module: The state monitoring module acquires and processes the information data collected by the state perception module in real time, monitors the swimmer's abnormal posture, abnormal water level, heart rate, blood oxygen saturation and location information to determine whether the swimmer is drowning and the location information; The alarm module includes an active alarm mode and an automatic alarm mode. The active alarm mode is a one-button calling device worn on the swimmer. When the detected person drowns, the drowning person's location information is sent to the server and displayed on the display device and an alarm is issued through the alarm device.

2. The intelligent drowning prevention warning system based on multi-sensor fusion according to claim 1 is characterized by: The attitude sensor is a nine-axis attitude sensor.

3. The intelligent drowning prevention warning system based on multi-sensor fusion according to claim 1 is characterized in that: The liquid level sensor is a non-contact liquid level sensor, which is worn on the back of the swimmer's head and at the same level as the nostrils through a wearable device.

4. The intelligent drowning prevention warning system based on multi-sensor fusion according to claim 1 is characterized in that: The UWB positioning system includes multiple UWB base stations arranged at the bank of the swimming pool and UWB positioning tags arranged on the swimmer's wearable device. The UWB positioning tags send out UWB positioning data and are received by each UWB base station. The UWB base station sends the received positioning data to a server.

5. The intelligent drowning prevention warning system based on multi-sensor fusion according to claim 1 is characterized in that: The server includes a status display module, which displays swimmer information, including user name, emergency contact phone number, current swimming style and swimming time.

6. The intelligent drowning prevention warning system based on multi-sensor fusion according to claim 1 is characterized in that: The state sensing module includes a vital sign collection module, and the vital sign collection module is used to monitor the swimmer's heart rate, blood oxygen, and body temperature.

7. The intelligent drowning prevention warning system based on multi-sensor fusion according to claim 1 is characterized by: The communication module is a Lora wireless communication module.

8. The intelligent drowning prevention warning system based on multi-sensor fusion according to claim 1 is characterized by: The alarm device includes a swimming place broadcast and a lifeguard watch / mobile phone.

9. The intelligent drowning prevention warning system based on multi-sensor fusion according to claim 1 is characterized in that: The abnormal posture of the swimmer includes that the average interval between the swimmer's frequency of entering and exiting the water and the frequency of raising and lowering the head exceeds a set time threshold, the head-down posture that does not occur in normal swimming exceeds a set time threshold, and the nearly vertical posture exceeds a set time threshold.

10. The intelligent drowning prevention warning system based on multi-sensor fusion according to claim 1 is characterized by: The water level abnormality information is when the mouth and nose of the swimmer are in the water for more than a set time threshold.

Images