A rescue underwater robot and rescue method based on multimodal drowning recognition

Abstract

The invention discloses a rescue underwater robot and a rescue method based on multimodal drowning recognition. The rescue underwater robot is provided with a multimodal recognition module, a rescue launch device module and a diving goggles auxiliary rescue module; the multimodal recognition module obtains images through a high-definition camera and sonar, and improves the recognition accuracy by using the joint calibration of the azimuth and feature fusion technology, so as to realize the accurate detection of the drowning target; the rescue launch device module adopts a spring launch device to quickly release the rescue device, and the rescue device is equipped with a high-elasticity silicone strip and a built-in inflatable bag. The silicone strip has an anti-slip texture structure, which can quickly surround and fix the waist of the drowning person, and the inflatable bag automatically starts to inflate after confirming that it is in firm contact with the drowning person through a magnetic delay mechanism; the diving goggles auxiliary rescue module is provided with a heart rate sensor, a motion sensor, an emergency button and a black box module to monitor the state of the diver in real time.

Description

Technical Field

[0001] The present invention belongs to the technical field of underwater robots, and in particular relates to a rescue underwater robot and a rescue method based on multi-modal drowning recognition. Background Art

[0002] In recent years, the demand for underwater rescue has increased rapidly, covering multiple fields such as marine resource development, recreational diving, scientific research and exploration. With the increasing vitality of underwater activities around the world, the application of underwater rescue systems has become more and more extensive, and its importance is self-evident. However, the underwater environment is complex and changeable, and factors such as insufficient light, unstable flow rate, and signal attenuation pose severe challenges to drowning identification and emergency response. Existing rescue systems mostly rely on a single sensor with limited recognition accuracy, and often use rigid structures with poor fixation effects, making it difficult to cope with the dynamic position of drowning people in complex water flows. The lack of information transmission technology also makes it difficult for rescuers to obtain underwater conditions in a timely manner, thus affecting the effectiveness and safety of rescue. How to improve the intelligence level and response speed of underwater rescue systems has become an urgent problem to be solved in the field of underwater rescue.

[0003] Deficiencies and reasons of existing technologies: 1. Existing underwater rescue systems usually analyze optical images and sonar images independently, and then superimpose the results. This method is prone to error accumulation and false targets, and fails to make full use of the information of the two sensors, thereby reducing the recognition accuracy, increasing the risk of misjudgment, and seriously affecting the accurate recognition of drowning situations and the rescue effect. 2. Traditional rescue devices mostly use rigid structures or fixing methods with weak adsorption force, which cannot effectively fix drowning people in strong water flow or unstable environment, increasing the risk of the rescue process. 3. The inflatable bags of traditional rescue devices need to rely on manual triggering, which increases the complexity and uncertainty of operations in emergency situations. In addition, the existing inflatable bags lack an intelligent delayed start mechanism, fail to play a role in time at critical moments, and affect the rescue efficiency. 4. Existing auxiliary rescue devices (such as diving goggles) lack intelligent monitoring and automatic distress functions, and fail to effectively coordinate with rescue underwater robots. It is difficult to achieve timely and effective signal transmission in emergency situations, and cannot provide sufficient information support for underwater rescue. Summary of the invention

[0004] The present invention provides a rescue underwater robot and rescue method based on multi-modal drowning recognition, which solves the deficiencies of existing underwater rescue technology in recognition accuracy, fixation stability, operation automation and information transmission, etc. It aims to improve the intelligence level and reaction speed of underwater rescue and provide safer and more efficient rescue guarantee for drowning people.

[0005] To solve the above problems, the technical solution provided by the present invention is as follows:

[0006] An embodiment of the present invention provides a rescue underwater robot based on multi-modal drowning recognition, wherein the rescue underwater robot is provided with a multi-modal recognition module and a rescue launch device module;

[0007] The multimodal recognition module includes a high-definition camera, a sonar device and an LED light source. The high-definition camera is used to capture clear optical images, and the sonar device is used to obtain underwater sonar images and target positions. The high-definition camera and the sonar device provide a comprehensive data basis for subsequent drowning target recognition.

[0008] The rescue launching device module comprises a launching tube, a spring system, an electric control system and a rescue device. The rescue device is arranged on the launching tube. The spring system is compressed and locked between the launching tube and the rescue underwater robot. The electric control system is used to unlock the locking mechanism of the spring system so that the launching tube moves toward the drowning person. The rescue device is used to directly contact the drowning person and play a role of fixing and rescuing. A pressure sensor is arranged on the end face of the launching tube.

[0009] An optional embodiment of the present invention, the rescue device includes an inflatable bag module, an elastic strip and a bionic suction cup structure, the bionic suction cup structure is fixed on the end face of the launching tube, the inflatable bag module is arranged inside the launching tube, the elastic strip is arranged on the opening side of the launching tube, and the opening side of the launching tube is connected to the inflatable bag module through a slit with the elastic strip; when the bionic suction cup structure comes into contact with the arm of the drowning person, the inflatable bag module automatically inflates with a delay of 1 second and forms an annular structure on both sides of the opening of the launching tube, and the annular structure is fixed on the waist of the drowning person.

[0010] In an optional embodiment of the present invention, the inflatable bag module is connected to a gas cylinder via a valve, and the inflatable bag module is used to provide buoyancy protection for the drowning person after the rescue device is fixed.

[0011] In an optional embodiment of the present invention, the spring system is a coil spring, which is clamped on the fixed axis of the rescue underwater robot, and the spring system is the source of the launching power of the launching tube.

[0012] In an optional embodiment of the present invention, the rescue underwater robot also includes a goggles-assisted rescue module, and the rescue device and the goggles-assisted rescue module work in cooperation with each other. The goggles-assisted rescue module is provided with a heart rate sensor, a motion sensor, an emergency button and a black box module. The double-layer pressure-proof lens of the goggles-assisted rescue module provides visual protection for the diver, the heart rate sensor and the motion sensor are used to monitor the diver's physical condition in real time, the emergency button is a key component for triggering a rescue mechanism, and the black box module is responsible for quickly floating to the surface and transmitting rescue information after being triggered, thereby realizing monitoring of the diver's condition and effective rescue and information transmission in emergency situations.

[0013] In an optional embodiment of the present invention, the diving goggle auxiliary rescue module is further provided with an anti-impact foam protective gear, and the anti-impact foam protective gear is transparent glass.

[0014] An embodiment of the present invention further provides a rescue method based on multimodal drowning recognition, which is implemented by a rescue underwater robot based on multimodal drowning recognition of the above embodiment, and is characterized in that it includes the following steps:

[0015] Step 1, multimodal recognition module workflow: When the rescue underwater robot is operating underwater, the high-definition camera and sonar equipment on the underwater robot are started synchronously to obtain optical images and sonar images at specific frame rates and frequencies respectively; then, a joint calibration method based on azimuth is used to accurately correspond the two image fields of view and coordinate systems through the processing of the angle sensor and microcontroller installed on the rescue underwater robot; then, the ResNet model is used to extract features for different image characteristics, and then the optical and sonar features are effectively combined through the multi-scale feature fusion layer, and finally, the drowning target is identified through the fusion features of the classifier, and the relevant information is transmitted to other functional modules;

[0016] Step 2, workflow of the rescue launch device module: After receiving the information such as the location of the drowning person from the multimodal recognition module, the rescue launch device module enters the launch preparation state. At this time, the electronic control system checks the spring system and various components of the rescue device to ensure that the spring parameters are normal and the pressure sensor, elastic strip, and inflatable bag module of the rescue device can work normally; during launch, the electronic control system is triggered, and the electronic control system unlocks the spring system. The spring releases potential energy to push the rescue device out of the launch tube; during the ejection process, the pressure sensor detects the contact pressure with the drowning person in real time. Once the pressure threshold is exceeded, the elastic strip quickly pops up and bends to form a ring structure to firmly adsorb the waist of the drowning person. The bionic suction cup enhances the adsorption force. After confirming the stable contact, the gas cylinder valve opens to drive the inflatable bag module to inflate, providing buoyancy for the drowning person;

[0017] Step 3, working process of the rescue device: After the rescue device contacts the drowning person, the bionic suction cup structure and elastic strip inside it will quickly unfold, and the anti-slip texture of the elastic strip will increase the adhesion with the drowning person and firmly fix it on the drowning person; the pressure sensor senses the pressure change, triggers the inflatable bag module, and quickly fills the inflatable bag with gas to provide additional buoyancy to ensure that the drowning person can float to the surface smoothly;

[0018] Step 4, working process of the goggles-assisted rescue module: after the diver goes into the water, the heart rate sensor and motion sensor monitor the diver's physical condition in real time at a frequency of 1 time per second; the heart rate sensor detects heart rate changes at a certain frequency, and the motion sensor updates the diver's motion status related data; when the heart rate or motion status is abnormal, or the diver triggers it manually, the black box on the side of the goggles quickly pops out with the help of a spring structure, floats to the surface due to its own buoyancy, and at the same time sends a rescue signal containing the diver's heart rate, motion status, water depth, positioning and other information to the shore safety officer and rescue underwater robot through wireless communication technology. The built-in data storage module in the black box also records relevant data to help determine the diver's location; each functional module works closely together to achieve efficient underwater rescue functions.

[0019] Compared with the prior art, the embodiment of the present invention provides a rescue underwater robot and a rescue method based on multi-modal drowning recognition, which has the following beneficial effects:

[0020] (1) The present invention has accurate drowning warning and fast response. Compared with the traditional single sensor system, the present invention integrates sensor data to greatly improve the detection accuracy of drowning risk. Through comprehensive analysis of sensor data, the smart diving goggles can issue a warning in time when the diver is in an abnormal state (such as being still for a long time or having abnormal heart rate changes), which significantly reduces the reaction delay caused by drowning.

[0021] (2) The present invention enhances underwater fixing ability and impact resistance. The rescue device of the present invention has a bionic suction cup structure and elastic strip design, which can significantly improve the adhesion ability of the device in a complex water flow environment. Compared with the traditional rigid structure, the rescue device of the present invention can quickly increase the friction when it contacts the drowning person, ensuring that it can remain stable and fixed even in high-speed water flow.

[0022] (3) The present invention has the characteristics of efficient rescue and safe buoyancy. The rescue device of the present invention adopts intelligent delayed start to automatically trigger the inflatable bag. After confirming that the contact with the drowning person is stable, it will automatically inflate after 1 second, avoiding manual delay and improving the rescue response speed. The inflatable bag is designed to be compact to ensure rapid deployment in a short time, reducing the deployment delay problem caused by excessive volume in traditional designs.

[0023] (3) The present invention improves the overall rescue efficiency, and the black box auxiliary rescue technology of the diving goggles has obvious advantages. The double-layer anti-pressure lens has a clear field of view and strong pressure resistance. The sensor detects abnormalities and triggers the emergency button. The black box pops up and floats up to transmit rescue information. It can also record data to help determine the diver's location, comprehensively improving the overall efficiency of underwater rescue, making rescue more timely, accurate and efficient. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments or the prior art, the drawings used in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0025] Figure 1 A schematic structural diagram of a rescue underwater robot based on multimodal drowning recognition provided in an embodiment of the present application.

[0026] Figure 2 A schematic diagram of a rescue launch device module of a rescue underwater robot based on multimodal drowning recognition provided in an embodiment of the present application.

[0027] Figure 3 A schematic diagram of an inflatable bag module of a rescue device provided in an embodiment of the present application.

[0028] Figure 4 A schematic diagram of a rescue process of a rescue launching device module provided in an embodiment of the present application.

[0029] Figure 5 A schematic diagram of a diving goggle-assisted rescue module provided in an embodiment of the present application.

[0030] Figure 6 A flowchart of a rescue method based on multimodal drowning recognition is provided in an embodiment of the present application.

[0031] Figure 7 A schematic diagram of an operation flow chart of a rescue launch device module provided in an embodiment of the present application. DETAILED DESCRIPTION

[0032] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0033] like Figure 1 , Figure 2, Figure 3 and Figure 4 As shown, an embodiment of the present invention provides a rescue underwater robot based on multimodal drowning recognition, and the rescue underwater robot is provided with a multimodal recognition module 1 and a rescue launch device module 2. The multimodal recognition module 1 and the rescue launch device module 2 work closely together to realize an efficient underwater rescue function.

[0034] The multimodal recognition module 1 includes a high-definition camera 1-1, a sonar device 1-2 and an LED light source 1-3. The high-definition camera 1-1 is responsible for capturing clear optical images, and the sonar device 1-2 is used to obtain underwater sonar images and target positions. The high-definition camera 1-1 and the sonar device 1-2 provide a comprehensive data basis for subsequent drowning target identification.

[0035] The rescue launch device module 2 includes a launch tube 2-1, a spring system, an electric control system and a rescue device. The rescue device is arranged on the launch tube 2-1. The spring system is compressed and locked between the launch tube and the rescue underwater robot. The electric control system is used to unlock the locking mechanism of the spring system so that the launch tube moves toward the drowning person. The rescue device is used to directly contact the drowning person and play a role in fixing and rescuing. A pressure sensor 2-2 is also arranged on the end surface of the launch tube 2-1.

[0036] The rescue device includes an inflatable bag module 2-7, an elastic strip 2-3 and a bionic suction cup structure 2-4, wherein the bionic suction cup structure 2-4 is fixed on the open end face of the launch tube 2-1, the inflatable bag module 2-7 is arranged inside the launch tube 2-1, the elastic strip 2-3 is arranged on the open side of the launch tube 2-1, and the open side of the launch tube 2-1 and the elastic strip 2-3 are connected to the inflatable bag module 2-7 through a slit; when the bionic suction cup structure 2-4 contacts the waist of the drowning person, the inflatable bag module 2-7 automatically inflates after a delay of 1 second and forms an annular structure on both sides of the opening of the launch tube 2-1, and the annular structure is fixed on the waist of the drowning person. That is, the elastic strip 2-3 drives the inflatable bag module 2-7 to form an annular structure on both sides of the opening of the launch tube 2-1, and the annular structure is fixed on the waist of the drowning person. The elastic strip 2-3 is provided with an anti-slip texture 2-5. A balancing tail structure 2-6 is provided at the opening of the launch tube 2-1.

[0037] The inflatable bag module 2-7 is connected to the gas cylinder 2-9 through the valve 2-8, and the inflatable bag module 2-7 is used to provide buoyancy protection for the drowning person after the rescue device is fixed. The spring system is a coil spring, which is clamped on the fixed axis of the rescue underwater robot, and the spring system is the source of the launching power of the launch tube.

[0038] like Figure 5As shown, the rescue underwater robot also includes a diving goggles auxiliary rescue module 3, and the rescue device and the diving goggles auxiliary rescue module 3 work in cooperation with each other. The diving goggles auxiliary rescue module 3 is provided with a heart rate sensor 3-2, a motion sensor 3-3, an emergency rescue button 3-4, a black box module 3-5 and a wireless transmission system 3-6. The double-layer anti-pressure lens 3-1 of the diving goggles auxiliary rescue module provides visual protection for the diver, the heart rate sensor 3-2 and the motion sensor 3-3 are used to monitor the diver's physical state in real time, the emergency rescue button 3-4 is a key component for triggering the rescue mechanism, and the black box module 3-5 is responsible for quickly floating to the surface and transmitting rescue information after being triggered, so as to realize the monitoring of the diver's state and effective rescue and information transmission in emergency situations. The diving goggles auxiliary rescue module is also provided with an anti-impact foam protective gear 3-7, and the anti-impact foam protective gear 3-7 is transparent glass.

[0039] refer to Figure 6 and Figure 7 The embodiment of the present invention further provides a rescue method based on multi-modal drowning recognition, which is implemented by a rescue underwater robot based on multi-modal drowning recognition of the above embodiment, and includes the following steps:

[0040] Step 1, multimodal recognition module workflow: When the rescue underwater robot is operating underwater, the high-definition camera and sonar equipment on the underwater robot are started synchronously to obtain optical images and sonar images at specific frame rates and frequencies respectively; then, a joint calibration method based on azimuth is used to accurately correspond the two image fields of view and coordinate systems through the processing of the angle sensor and microcontroller installed on the rescue underwater robot; then, the ResNet model is used to extract features for different image characteristics, and then the optical and sonar features are effectively combined through the multi-scale feature fusion layer, and finally, the drowning target is identified through the fusion features of the classifier, and the relevant information is transmitted to other functional modules;

[0041] Step 2, workflow of the rescue launch device module: After receiving the information such as the location of the drowning person from the multimodal recognition module, the rescue launch device module enters the launch preparation state. At this time, the electronic control system checks the spring system and various components of the rescue device to ensure that the spring parameters are normal and the pressure sensor, elastic strip, and inflatable bag module of the rescue device can work normally; during launch, the electronic control system is triggered, and the electronic control system unlocks the spring system. The spring releases potential energy to push the rescue device out of the launch tube; during the ejection process, the head pressure sensor detects the contact pressure with the drowning person in real time. Once the pressure threshold is exceeded, the elastic strip quickly pops up and bends to form a ring structure to firmly adsorb the waist of the drowning person. The bionic suction cup enhances the adsorption force. After confirming the stable contact, the gas cylinder valve opens to drive the inflatable bag module to inflate, providing buoyancy for the drowning person;

[0042] Step 3, working process of the rescue device: After the rescue device contacts the drowning person, the bionic suction cup structure and elastic strip inside it will quickly unfold, and the anti-slip texture of the elastic strip will increase the adhesion with the drowning person and firmly fix it on the drowning person; the pressure sensor senses the pressure change, triggers the inflatable bag module, and quickly fills the inflatable bag with gas to provide additional buoyancy to ensure that the drowning person can float to the surface smoothly;

[0043] Step 4, working process of the goggles-assisted rescue module: after the diver goes into the water, the heart rate sensor and motion sensor monitor the diver's physical condition in real time at a frequency of 1 time per second; the heart rate sensor detects heart rate changes at a certain frequency, and the motion sensor updates the diver's motion status related data; when the heart rate or motion status is abnormal, or the diver triggers it manually, the black box on the side of the goggles quickly pops out with the help of a spring structure, floats to the surface due to its own buoyancy, and at the same time sends a rescue signal containing the diver's heart rate, motion status, water depth, positioning and other information to the shore safety officer and rescue underwater robot through wireless communication technology. The built-in data storage module in the black box also records relevant data to help determine the diver's location; each functional module works closely together to achieve efficient underwater rescue functions.

[0044] The above is a rescue underwater robot and rescue method based on multimodal drowning recognition. The rescue underwater robot is provided with a multimodal recognition module, a rescue launcher module and a diving goggles assisted rescue module. The multimodal recognition module obtains images through high-definition cameras and sonars, and uses the joint calibration of azimuth and feature fusion technology to improve the recognition accuracy, so as to achieve accurate detection of drowning targets. The rescue launcher module uses a spring launcher to quickly release the rescue device. The rescue device is equipped with a highly elastic silicone strip and a built-in inflatable bag. The silicone strip has an anti-slip texture structure, which can quickly surround and fix the waist of the drowning person. The inflatable bag automatically starts to inflate after confirming a firm contact with the drowning person through a magnetic delay mechanism. The diving goggles assisted rescue module is equipped with a double-layer anti-pressure lens and a heart rate and motion sensor to monitor the diver's status in real time. When an abnormality is detected, the emergency call button triggers the built-in black box, which will quickly float to the surface and send rescue information to the shore safety personnel and the underwater robot through wireless communication, while recording the water depth and positioning data to help quickly determine the diver's location.

[0045] In order to overcome the limitations of a single sensor in terms of information integrity and accuracy, the present invention aims to enhance the intelligent perception capability of underwater robots by combining the feature fusion technology of optical images and sonar images. First, a high-resolution optical camera and sonar sensor are configured on the underwater robot, and a joint calibration method based on the azimuth angle is adopted to ensure that the field of view and coordinate system of the two are consistent. In the feature extraction process, ResNet (residual network) is used as a deep learning model. The model effectively solves the gradient vanishing and overfitting problems in deep network training by introducing the residual learning principle, and improves the accuracy and efficiency of feature extraction. In the specific implementation, in view of the low resolution, fuzziness and strong noise characteristics of sonar images, 2, 3, 4, and 2 modules are set in the conv2_x to conv5_x stages of ResNet (residual network), respectively. These modules are designed with residual connections to make the network more robust when extracting features, thereby effectively reducing the impact of noise and improving model performance. At the same time, considering the high resolution and rich details of visible light images, 3, 4, 5, and 3 modules are set in sequence at the corresponding stages to make full use of the information of visible light images and prevent overfitting, thereby enhancing feature extraction and generalization capabilities. Subsequently, the optical and sonar features are effectively combined through a multi-scale feature fusion layer to further improve the accuracy and stability of recognition. The system trains the deep learning model by constructing annotated data sets, processes optical and sonar data in real time, and realizes rapid identification of drowning people in complex underwater environments.

[0046] The rescue launcher module is mainly composed of a launch tube, a spring system, an electronic control system and a rescue device. In an emergency, the module can launch the rescue device quickly and efficiently. The launch tube is made of high-strength, corrosion-resistant materials and has good pressure resistance. Its interior is a cylindrical cavity structure specially used to accommodate the rescue device. The spring system is made of high-strength spring steel and can generate sufficient thrust during launch to ensure that it has penetration power underwater and does not cause harm to the drowning person. When triggered, the spring system can quickly release the elastic potential energy and push the rescue device out of the launch tube. The parameters of the spring are designed to ensure that its launch force is sufficient to penetrate the underwater resistance without causing additional harm to the drowning person. The electronic control system ensures that the spring system is unlocked quickly and safely when needed. The main body of the rescue device is designed to be compact and cylindrical to ensure efficient transmission during the launch process. Its head adopts a semicircular design and is equipped with a pressure sensor and a sponge layer to ensure that it can quickly sense pressure changes when in contact with the drowning person to avoid any harm to the drowning person; elastic strip structures are designed on both sides of the main body, and the outer side of the elastic strip adopts soft high-friction material and bionic suction cup structure. When the pressure sensor senses the target, the elastic strip can quickly bounce up and form an angle greater than 90 degrees with the side of the main body. When the elastic strip contacts the waist of the drowning person, the elastic strip will quickly bend to form a stable ring structure, firmly surrounding and adsorbing the waist of the drowning person; the inflatable bag module is embedded in the tail end of the deformable buoyancy rescue device, which consists of a waterproof shell, an inflatable device, a trigger device, and a pressure release valve. The waterproof shell is made of lightweight and durable materials. The inflatable device includes a small gas cylinder, a valve, and an inflatable bag. The trigger setting has a built-in electromagnetic delay mechanism, which cooperates with the continuous monitoring of the multimodal recognition module to ensure that the device is stably attached to the waist of the drowning person during inflation.

[0047] In the diving goggles auxiliary rescue module, first, the physical properties of different materials of the diving goggles are studied, such as compressive strength, optical properties and corrosion resistance, to determine the most suitable material for the double-layer anti-pressure lens. Considering the complexity of the underwater environment, the lens material needs to have high strength to resist water pressure. The outer lens can be made of high-hardness polycarbonate material, which has excellent impact resistance and can effectively resist the collision that may be encountered underwater. The inner lens can be made of resin material with excellent optical properties to ensure that the diver can obtain a clear field of vision. At the same time, the working principles of the heart rate sensor and motion sensor are analyzed. The heart rate sensor usually adopts the photoelectric detection principle. When the heart beats, the blood flow will change, causing the intensity of the reflected light to change. The sensor converts these changes into electrical signals, and after processing, the heart rate value is obtained. The motion sensor generally uses a combination of accelerometers and gyroscopes. The accelerometer can measure the acceleration changes of the diver in three directions, and the gyroscope can measure the rotation angle and angular velocity. By analyzing these data, the diver's motion state can be judged, such as whether he is swimming or doing strenuous exercise. In addition, the black box's floating process is simulated, taking into account factors such as water resistance, buoyancy and water flow, to determine the optimal black box structure and shape to ensure that it can quickly float to the surface. The communication signal strength between the black box and the shore safety officer and the underwater robot is also simulated, analyzing the impact of obstacles and distance in the communication environment on the signal, and optimizing the parameter settings of the wireless communication module.

[0048] A rescue underwater robot and rescue method based on multimodal drowning recognition has the following innovations:

[0049] Innovation 1: Multimodal sensing drowning identification and rescue system. To overcome the limitations of single sensor images in terms of information integrity and accuracy, the present invention combines the feature fusion technology of optical images and sonar images, and utilizes joint calibration based on azimuth, feature extraction based on visible light and sonar image differences, and feature fusion methods based on multiple scales to enhance the intelligent perception capability of underwater unmanned vehicles. At the same time, PID motion control and visual servo control are used to realize the fully automated rescue process of drones. The drones can autonomously identify drowning targets, accurately adjust their postures and enter the effective rescue range. The microcontroller automatically launches the rescue device, which greatly enhances the intelligence and accuracy of underwater rescue.

[0050] Innovation 2: Bionic adsorption quick binding technology for connecting drowning people and inflatable bags - guarantee of stable rescue in rescue devices. In order to solve the problem that existing rescue devices mostly use rigid structures or weak adsorption fixing mechanisms, which are difficult to adapt to complex water flows, the present invention adopts a flexible and stable bionic adsorption technology to ensure that drowning people can be firmly fixed under the impact of water flow. The rescue device is equipped with a silicone elastic strip, and the surface is provided with a tiny cross-grid anti-slip texture. When the multimodal recognition module determines the position of the drowning person and contacts the target, the elastic strip is quickly tightened under the feedback of the pressure sensor, and surrounds the waist of the drowning person to form a stable enclosure. At the same time, a tiny cross-grid anti-slip texture is designed on the surface of the elastic strip to increase friction and avoid sliding when in contact underwater. In addition, the elastic strip is also equipped with a small bionic suction cup. The shape of the bionic suction cup refers to the octopus suction cup. The concave structure in the suction cup generates negative pressure when in contact with the skin of the drowning person, making the adsorption force more solid. This technology can adjust the adsorption strength according to the water flow environment, ensuring that the drowning person will not break away from the rescue device even in a strong water flow, and improving the reliability and adaptability of underwater fixation.

[0051] Innovation point 3: Delayed start of the automatic trigger inflatable bag - a guarantee of safe floating in the rescue device. Traditional inflatable rescue devices usually need to be manually triggered, and the inflation speed is slow, and the buoyancy adjustment range is limited. The present invention designs an automatic inflatable bag system with intelligent delayed start, which automatically starts the inflation process after full contact with the drowning person and confirms the encirclement state, without manual operation. The present invention has a built-in electromagnetic delay mechanism, combined with the continuous monitoring of the multimodal recognition module, to ensure that inflation is delayed by 1 second after triggering, to ensure that the device is stably attached to the waist of the drowning person during inflation, and to prevent inflation from affecting the fixation effect. At the same time, the inflatable bag is designed as a compact Z-folding structure with a compression ratio of more than 90%. It can expand rapidly within 1 second when released. A small replaceable carbon dioxide cylinder is used as the inflation source to ensure that the equipment is lightweight and can be quickly inflated after the delay ends.

[0052] Innovation 4: Black box assisted rescue technology with intelligent triggering and information transmission for diving goggles: In order to improve the overall efficiency of underwater robot rescue, the present invention adopts a black box assisted rescue technology on the diving goggles, which can quickly trigger the rescue mechanism and provide key rescue information in an emergency. In terms of lens design, a double-layer anti-pressure lens design is adopted, which can provide a clearer underwater field of vision and stronger pressure resistance compared with a single-layer lens. The diving goggles are equipped with a heart rate sensor and a motion sensor to monitor the diver's heart rate and movement status. When there are abnormal heart rate changes (such as too fast, too slow, irregular beating) or abnormal movement status (such as difficulty breathing, dizziness, etc.), the emergency rescue button will be passively triggered. In addition, even if the sensor does not detect obvious abnormalities, if the diver is unwell, the button can be manually triggered. Once the emergency rescue button is triggered, the black box on the side of the diving goggles will pop out quickly with the help of a spring structure. The black box adopts a lightweight and buoyant structural design. After popping out, it quickly floats to the surface of the water by its own buoyancy, and sends a signal containing rescue information in time through wireless communication technology. The signal of this rescue information will not only be sent to the safety officer on the shore, but also to the rescue underwater robot, which will trigger the underwater robot to launch the rescue device. At the same time, its built-in data storage module can record and save water depth data, positioning information and other content, helping rescuers to quickly determine the location of the diver, thereby greatly reducing the scope of rescue.

[0053] In summary, although the present invention has been disclosed as above in terms of preferred embodiments, the above preferred embodiments are not intended to limit the present invention. A person skilled in the art may make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be based on the scope defined in the claims.

Claims

1. A rescue underwater robot based on multimodal drowning recognition, characterized in that: The rescue underwater robot is provided with a multi-modal recognition module and a rescue launching device module; The multimodal recognition module includes a high-definition camera, a sonar device and an LED light source. The high-definition camera is used to capture clear optical images, and the sonar device is used to obtain underwater sonar images and target positions. The high-definition camera and the sonar device provide a comprehensive data basis for subsequent drowning target recognition. The rescue launch device module includes a launch tube, a spring system, an electric control system and a rescue device. The rescue device is arranged on the launch tube. The spring system is compressed and locked between the launch tube and the rescue underwater robot. The electric control system is used to unlock the locking mechanism of the spring system so that the launch tube moves toward the drowning person. The rescue device is used to directly contact the drowning person and play a role in fixing and rescuing. A pressure sensor is arranged on the end surface of the launch tube. The rescue device includes an inflatable bag module, an elastic strip and a bionic suction cup structure, wherein the bionic suction cup structure is fixed on the end surface of the launching tube, the inflatable bag module is arranged inside the launching tube, the elastic strip is arranged on the opening side of the launching tube, and the elastic strip is connected to the inflatable bag module through a slit; when the bionic suction cup structure contacts the waist of the drowning person, the inflatable bag module automatically inflates with a delay of 1 second and forms an annular structure on both sides of the opening of the launching tube, and the annular structure is fixed on the waist of the drowning person.

2. A rescue underwater robot based on multimodal drowning recognition according to claim 1, characterized in that: The inflatable bag module is connected to a gas cylinder via a valve, and the inflatable bag module provides buoyancy for the drowning person after the rescue device is fixed.

3. A rescue underwater robot based on multimodal drowning recognition according to claim 2, characterized in that: The spring system is a coil spring, which is clamped on the fixed axis of the rescue underwater robot. The spring system is the source of the launching power of the launching tube.

4. The underwater rescue robot based on multimodal drowning recognition according to claim 3 is characterized in that: The rescue underwater robot also includes a goggles-assisted rescue module. The rescue device and the goggles-assisted rescue module work in cooperation with each other. The goggles-assisted rescue module is provided with a heart rate sensor, a motion sensor, an emergency button and a black box module. The double-layer pressure-proof lens of the goggles-assisted rescue module provides visual protection for the diver. The heart rate sensor and the motion sensor are used to monitor the diver's physical condition in real time. The emergency button is a key component for triggering a rescue mechanism. The black box module is responsible for quickly floating to the surface and transmitting rescue information after being triggered, so as to realize monitoring of the diver's condition and effective rescue and information transmission in emergency situations.

5. The underwater rescue robot based on multi-modal drowning recognition according to claim 4, characterized in that: The diving goggle auxiliary rescue module is also provided with an anti-impact foam protective gear, and the anti-impact foam protective gear is transparent glass.

6. A rescue method based on multi-modal drowning recognition, implemented by a rescue underwater robot based on multi-modal drowning recognition as claimed in claim 5, characterized in that: The following steps are involved: Step 1, multimodal recognition module workflow: When the rescue underwater robot is operating underwater, the high-definition camera and sonar equipment on the underwater robot are started synchronously to obtain optical images and sonar images at specific frame rates and frequencies respectively; then, a joint calibration method based on azimuth is used to accurately correspond the two image fields of view and coordinate systems through the processing of the angle sensor and microcontroller installed on the rescue underwater robot; then, the ResNet model is used to extract features for different image characteristics, and then the optical and sonar features are effectively combined through the multi-scale feature fusion layer, and finally, the drowning target is identified through the fusion features of the classifier, and the relevant information is transmitted to other functional modules; Step 2, workflow of the rescue launch device module: After receiving the drowning person's location information from the multimodal recognition module, the rescue launch device module enters the launch preparation state. At this time, the electronic control system checks the spring system and various components of the rescue device to ensure that the spring parameters are normal and the pressure sensor, elastic strip, and inflatable bag module of the rescue device can work normally; during launch, the electronic control system is triggered, and the electronic control system unlocks the spring system. The spring releases potential energy to push the rescue device out of the launch tube; during the ejection process, the pressure sensor detects the contact pressure with the drowning person in real time. Once the pressure threshold is exceeded, the elastic strip quickly pops up and bends to form a ring structure to firmly adsorb the drowning person's waist. The bionic suction cup enhances the adsorption force. After confirming the stable contact, the gas cylinder valve opens to drive the inflatable bag module to inflate, providing buoyancy for the drowning person; Step 3, working process of the rescue device: After the rescue device contacts the drowning person, the bionic suction cup structure and elastic strip inside it will quickly unfold, and the anti-slip texture of the elastic strip will increase the adhesion with the drowning person and firmly fix it on the drowning person; the pressure sensor senses the pressure change, triggers the inflatable bag module, and quickly fills the inflatable bag with gas to provide additional buoyancy to ensure that the drowning person can float to the surface smoothly; Step 4, working process of the goggles-assisted rescue module: after the diver goes into the water, the heart rate sensor and motion sensor monitor the diver's physical condition in real time at a frequency of 1 time per second; the heart rate sensor detects heart rate changes at a certain frequency, and the motion sensor updates the diver's motion status related data; when the heart rate or motion status is abnormal, or the diver triggers it manually, the black box on the side of the goggles quickly pops out with the help of a spring structure, floats to the surface due to its own buoyancy, and at the same time sends a rescue signal containing the diver's heart rate, motion status, water depth, and positioning information to the shore safety officer and rescue underwater robot through wireless communication technology. The built-in data storage module in the black box also records relevant data to help determine the diver's location; each functional module works closely together to achieve efficient underwater rescue functions.

Images