Multi-mode water rescue robot cooperative control method and related device
By combining multimodal data fusion and swarm intelligence collaborative control technology with a biomimetic octopus tentacle rescue mechanism, the problem of target detection and collaborative efficiency of water rescue robots in complex water environments has been solved, enabling efficient and accurate execution of rescue missions.
Patent Information
- Application Number
- CN202510706103.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-29
- Publication Date
- 2025-11-11
AI Technical Summary
Existing water rescue robots lack accuracy in target detection in complex water environments, have low coordination efficiency, and lack an efficient dynamic task allocation mechanism, making it difficult to improve rescue efficiency.
Multimodal data fusion technology is used to acquire target water information by combining visual, sonar and radar sensors. The accuracy of target detection is improved by multimodal information fusion algorithm, and task allocation and path planning are optimized by swarm intelligence collaborative control algorithm. A biomimetic octopus tentacle rescue mechanism is used to achieve flexible rescue.
The robot system improves the accuracy of target detection and rescue efficiency, enabling it to quickly and accurately locate and execute collaborative rescue actions in complex environments, significantly increasing the success rate and efficiency of rescue operations.
Smart Images
Figure CN120922313A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of robot collaborative control technology, and in particular to a collaborative control method and related device for a multimodal water rescue robot. Background Technology
[0002] Against the backdrop of intensifying global warming and frequent extreme weather events, water rescue robots, with their advantages of rapid deployment and operation in high-risk environments, have become a key technology in modern emergency rescue systems. They are widely used in maritime patrol, flood control, and maritime accident handling. Existing water rescue robots have demonstrated functions such as remote control, autonomous navigation, obstacle avoidance, search and towing of distressed individuals, and some high-end models are equipped with advanced sensors such as infrared thermal imagers and underwater cameras, enabling them to perform rescue missions in adverse weather and at night. However, existing technologies still have some problems, such as limited perception: current robots mostly rely on single sensors, which are easily interfered with in complex aquatic environments, leading to false positives and false negatives in target detection. Low collaborative efficiency: existing water rescue robots mostly operate alone, or even when there are a few attempts at multi-robot collaboration, the lack of an efficient dynamic task allocation mechanism often hinders the improvement of rescue efficiency. Summary of the Invention
[0003] The purpose of this application is to provide a multimodal water rescue robot cooperative control method and related device, which can improve the accuracy of target detection of water robots and realize cooperative control among various water rescue robots.
[0004] To achieve the above objectives, this application provides the following solution:
[0005] Firstly, this application provides a cooperative control method for a multimodal water rescue robot, including:
[0006] Acquire multimodal data of the target water area; the multimodal data includes water environment images acquired based on visual sensors, distance and depth information of underwater targets determined based on sonar sensors, and position and velocity information of underwater targets determined based on radar sensors;
[0007] Determine rescue targets within the target waters based on multimodal data;
[0008] The rescue mission of each rescue robot is determined based on the location information of the rescue target, the current location and status of each rescue robot, and the current water environment information.
[0009] The optimal path for each rescue robot to reach the matching rescue target is determined based on the rescue target corresponding to the rescue mission of each rescue robot, the current location of the rescue robot, and the current water environment information.
[0010] Each rescue robot is driven to perform coordinated rescue actions based on its optimal path.
[0011] Secondly, this application provides a multimodal water rescue robot collaborative control system, including:
[0012] The data acquisition module is used to acquire multimodal data of the target water area; the multimodal data includes water environment images collected based on visual sensors, distance and depth information of underwater targets determined based on sonar sensors, and position and velocity information of underwater targets determined based on radar sensors.
[0013] The target determination module is used to determine rescue targets within the target water area based on multimodal data;
[0014] The task allocation module is used to determine the rescue task of each rescue robot based on the location information of the rescue target, the current location and status of each rescue robot, and the current water environment information.
[0015] The path planning module is used to determine the optimal path for each rescue robot to reach the matching rescue target location based on the rescue target corresponding to the rescue mission of each rescue robot, the current location of the rescue robot, and the current water environment information.
[0016] The execution module is used to drive each rescue robot to perform coordinated rescue actions based on the optimal path of each robot.
[0017] Thirdly, this application provides a multimodal water rescue robot collaborative control device, including: a host computer, a water rescue robot body, a rescue mechanism and a controller disposed on the water rescue robot body;
[0018] The water rescue robot body and the rescue mechanism are connected to the controller; the rescue mechanism adopts a biomimetic octopus tentacle rescue mechanism.
[0019] The controller is also connected to a host computer for communication.
[0020] The controller is used to receive instructions from the host computer, control the sensors on the water rescue robot body to collect data and transmit it to the host computer according to the instructions, and drive the drive components on the water rescue robot body to drive the rescue mechanism to perform rescue actions according to the instructions.
[0021] The host computer is used to execute the above-mentioned multimodal water rescue robot collaborative control method.
[0022] Fourthly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described multimodal water rescue robot collaborative control method.
[0023] Fifthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described multimodal water rescue robot cooperative control method.
[0024] Sixthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the above-described multimodal water rescue robot collaborative control method.
[0025] According to the specific embodiments provided in this application, the following technical effects are disclosed:
[0026] This application provides a multimodal collaborative control method and related apparatus for water rescue robots. The method acquires water environment images collected by visual sensors, distance and depth information of underwater targets determined by sonar sensors, and position and velocity information of underwater targets determined by radar sensors within the target water area. Based on the multimodal data, it determines the rescue targets within the target water area. Based on the position information of the rescue targets, the current position and status of each rescue robot, and the current water environment information, it determines the rescue task of each rescue robot. Finally, it determines the optimal path for each rescue robot to reach the matched rescue target location to execute collaborative rescue actions. This application improves the accuracy of target detection by using multi-source perception data, and achieves collaborative control of multiple water rescue robots through task allocation and path planning. Attached Figure Description
[0027] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0028] Figure 1 This is an application environment diagram of a collaborative control method for a multimodal water rescue robot according to an embodiment of this application;
[0029] Figure 2 A flowchart illustrating a collaborative control method for a multimodal water rescue robot provided in an embodiment of this application;
[0030] Figure 3A schematic diagram of the functional modules of a multimodal water rescue robot collaborative control system provided in an embodiment of this application;
[0031] Figure 4 A functional module schematic diagram of a multimodal water rescue robot collaborative control device provided in another embodiment of this application;
[0032] Figure 5 This is a schematic diagram of the structure of a control mechanism provided in one embodiment of this application;
[0033] Figure 6 This is a schematic diagram of the first cross-sectional structure of a control mechanism provided in an embodiment of this application;
[0034] Figure 7 This is a schematic diagram of the second cross-sectional structure of a control mechanism provided in an embodiment of this application;
[0035] Figure 8 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation
[0036] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0037] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0038] The multimodal water rescue robot collaborative control method provided in this application embodiment can be applied to, for example... Figure 1The application environment shown is as follows. The terminal communicates with the server via a network. The data storage system stores the data that the server needs to process. The data storage system can be set up independently, integrated into the server, or placed in the cloud or on another server. The terminal can send multimodal data (water environment images collected by visual sensors, distance and depth information of underwater targets determined by sonar sensors, and position and velocity information of underwater targets determined by radar sensors) to the server. After receiving the multimodal data, the server determines the rescue target within the target water area based on the multimodal data; determines the rescue task of each rescue robot based on the location information of the rescue target, the current position and status of each rescue robot, and the current water environment information; determines the optimal path for each rescue robot to reach the matched rescue target location based on the rescue target corresponding to each rescue robot's rescue task, the current position of the rescue robot, and the current water environment information; and drives each rescue robot to perform coordinated rescue actions based on the optimal paths of each rescue robot. The server can feed back the obtained optimal paths of each rescue robot to the terminal. In addition, in some embodiments, the multimodal water rescue robot collaborative control method can also be implemented by the server or the terminal alone. For example, the terminal can directly perform multimodal water rescue robot collaborative control based on multimodal data, or the server can obtain multimodal data from the data storage system and perform multimodal water rescue robot collaborative control.
[0039] The terminal can be, but is not limited to, various desktop computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. The server can be a standalone server or a server cluster composed of multiple servers, or it can be a cloud server.
[0040] In one exemplary embodiment, such as Figure 2 As shown, a collaborative control method for a multimodal water rescue robot is provided. This method is executed by computer equipment, specifically by a terminal or server alone, or by both a terminal and a server. In this embodiment, the method is applied to... Figure 1 The following steps, 101 to 105, are used as an example of the server in the example.
[0041] Step 101: Acquire multimodal data of the target water area; the multimodal data includes water environment images collected based on visual sensors, distance and depth information of underwater targets determined based on sonar sensors, and position and velocity information of underwater targets determined based on radar sensors.
[0042] Multi-dimensional perception of the aquatic environment provides foundational data for subsequent target detection and environmental understanding.
[0043] Step 102: Determine the rescue target within the target water area based on multimodal data.
[0044] Step 103: Determine the rescue mission of each rescue robot based on the location information of the rescue target, the current location and status of each rescue robot, and the current water environment information.
[0045] Step 104: Determine the optimal path for each rescue robot to reach the matching rescue target location based on the rescue target corresponding to the rescue mission of each rescue robot, the current location of the rescue robot, and the current water environment information.
[0046] Step 105: Drive each rescue robot to perform coordinated rescue actions based on the optimal path of each rescue robot.
[0047] In another exemplary embodiment of this application, step 102, determining the rescue target within the target water area based on multimodal data, specifically includes:
[0048] (1) Input the water environment image into the YOLOv8 model to obtain the target detection results.
[0049] Noise is removed from the water environment image using algorithms such as Gaussian filtering to obtain a smooth image, which is then input into the model.
[0050] (2) Calculate the distance and depth information of underwater targets based on the reflected signals received by the sonar sensor.
[0051] (3) Determine the position and velocity information of underwater targets based on the reflected signals received by radar sensors.
[0052] (4) Normalize the distance and depth information of underwater targets determined by sonar sensors and the position and velocity information of underwater targets determined by radar sensors to obtain normalized sonar sensing data and normalized radar sensing data.
[0053] Using algorithms such as Gaussian filtering to remove noise from image data and normalizing sonar and radar data to adjust their numerical range to a uniform interval can improve data quality, eliminate the differences in dimensions and values between data from different sensors, and facilitate subsequent fusion processing.
[0054] (5) Based on the target detection results, the normalized sonar sensing data and normalized radar sensing data are time-synchronized and aligned to obtain the aligned sonar sensing data and aligned radar sensing data.
[0055] The system primarily uses visual sensor data, while sonar and radar sensor data are aligned via a time synchronization module.
[0056] (6) Filter the target detection results, aligned sonar sensing data and aligned radar sensing data to obtain filtered target detection data, filtered sonar sensing data and filtered radar sensing data.
[0057] Algorithms such as Kalman filtering are used to filter the aligned data and remove noise interference.
[0058] (7) The filtered target detection data, filtered sonar sensing data and filtered radar sensing data are fused to obtain multi-source fused data.
[0059] By using feature association algorithms to establish correlations between different modal data and fusing multi-source data based on the correlation results, the limitations of a single visual sensor can be overcome. By utilizing the complementarity of multi-source information, the accuracy and reliability of target detection can be improved, and the system's adaptability to complex aquatic environments can be enhanced.
[0060] (8) Determine the rescue targets in the target water area based on multi-source fusion data.
[0061] By combining visual sensors with other sensors such as sonar and radar, a multimodal information fusion technology has been developed. Time stamp alignment of multi-source data (radar, vision, sonar, etc.) is achieved through hardware synchronization signals. This algorithm can perform denoising, filtering, normalization, and feature extraction on data from different modalities, fully leveraging the advantages of each sensor and overcoming the limitations of a single sensor, thereby enabling more reliable rescue missions in harsh environments.
[0062] In another exemplary embodiment of this application, in step (1), a large amount of labeled water rescue-related image data is collected for model training. A YOLOv8 model is built using the PyTorch deep learning framework, and parameters such as learning rate, batch size, and number of training rounds are set. Optimization algorithms such as stochastic gradient descent are used for model training, and data augmentation techniques are used to expand the dataset. This enables the training of a target detection model suitable for complex aquatic environments, which can quickly and accurately identify targets such as people who have fallen into the water.
[0063] This application integrates the C2f-ContextGuided module into the feature extraction part of the basic YOLOv8 model and introduces the AFPN-P345 module into the feature fusion part. The model is retrained and optimized, and the model parameters are adjusted so that the improved model can better capture target features in complex backgrounds. The model improvement enhances the model's detection capabilities for small targets and complex scenes, improving target recognition performance in harsh environments. Therefore, the YOLOv8 model can be an improved YOLOv8 model; in this improved YOLOv8 model, the C2f module is replaced by the C2f-ContextGuided module, and the PANet module is replaced by the AFPN-P345 module. The introduction of the C2f-ContextGuided and AFPN-P345 modules improves the basic YOLOv8 model. The YOLOv8-C2f-ContextGuided model optimizes small target detection through cross-stage feature sharing, and the introduction of the ContextGuided module enhances context awareness, enabling the network to better utilize environmental information and improve target detection capabilities in complex backgrounds. The YOLOv8-AFPN-P345 model uses an Advanced Feature Pyramid Network (AFPN) as its core module. Through deeper feature fusion, it enhances the diversity and richness of feature extraction, thereby improving the detection capability of small targets and complex scenes.
[0064] In another exemplary embodiment of this application, step 104, determining the rescue task of each rescue robot based on the location information of the rescue target, the current location and current status of each rescue robot, and the current water environment information, specifically includes:
[0065] Based on the location information of the rescue target, the current location and status of each rescue robot, and the current water environment information, an optimization algorithm is applied to determine the rescue task of each rescue robot, thereby achieving reasonable allocation of rescue tasks and improving rescue efficiency; the optimization algorithm includes ant colony algorithm and particle swarm algorithm.
[0066] A high-precision GPS module is installed on the top of the robot, which receives satellite signals through an antenna and calculates the robot's latitude and longitude coordinates in real time.
[0067] The Kalman filter algorithm is used to filter the positioning data, remove noise interference, and improve positioning accuracy.
[0068] The positioning data is transmitted to the ESP8266 module via serial port, and the ESP8266 module then sends the data to the host computer (remote control terminal). This provides the robot with real-time and accurate positioning information, supports multi-robot collaborative operation and dynamic path planning, and ensures the orderly conduct of rescue operations.
[0069] Each rescue robot periodically transmits its location, target information, and environmental data to other robots and remote control terminals via the LoRa wireless communication network, according to a preset communication protocol. Simultaneously, it receives information from other robots and remote control terminals, achieving information synchronization among multiple robots. This allows each robot to understand the overall environment and task status, providing a basis for collaborative decision-making.
[0070] In another exemplary embodiment of this application, step 105, determining the optimal path for each rescue robot to reach the matched rescue target location based on the rescue target corresponding to the rescue task of each rescue robot, the current location of the rescue robot, and the current water environment information, specifically includes:
[0071] Based on the rescue target corresponding to each rescue robot's rescue mission, the robot's current location, and the current aquatic environment, an optimization algorithm is applied to determine the optimal path for each rescue robot to reach the matched rescue target location. This path considers factors such as path length, safety, and energy consumption, and is dynamically adjusted according to environmental changes to provide the robot with a safe and efficient movement path, ensuring that the robot can quickly reach the target location and improve the rescue success rate. The optimization algorithm includes ant colony optimization and particle swarm optimization.
[0072] Develop a multi-robot collaborative control algorithm based on swarm intelligence algorithms such as ant colony optimization and particle swarm optimization. This enables multiple rescue robots to share location, target, and environmental information in real time, dynamically adjusting task allocation and path planning. Through simulated learning and optimized decision-making processes, the system can predict the rescue success rate under different task allocation schemes, automatically select the optimal scheme, and improve rescue efficiency and success rate. Establish an intelligent decision-making system based on machine learning, enabling robots to autonomously adjust rescue strategies based on real-time environmental data. Through continuous learning and optimization, robots can better adapt to complex and ever-changing rescue scenarios, further improving rescue efficiency.
[0073] This application has the following effects:
[0074] (I) High-efficiency rescue capabilities
[0075] Advantages: The water rescue robot system can respond quickly to rescue commands and reach the location of the person who has fallen into the water in a short time, which significantly improves rescue efficiency.
[0076] Corresponding technical points:
[0077] Multimodal information fusion algorithm: By fusing data from multiple sensors such as vision, sonar, and radar, the robot can locate the person who has fallen into the water more quickly and accurately, reducing search time.
[0078] Swarm intelligence collaborative control algorithm: Multiple robots work together, dynamically adjusting path planning based on real-time environmental data to avoid obstacles and achieve rapid and efficient rescue.
[0079] (II) Precise Positioning and Environmental Perception
[0080] Advantages: The system utilizes a high-precision GPS module and multiple sensors to accurately obtain its own location and perceive the surrounding environment in complex environments. Even in dark or murky waters, it can find and rescue people who have fallen into the water through advanced technology.
[0081] Corresponding technical points:
[0082] Multimodal information fusion algorithm: integrates data from multiple sources such as vision, sonar, and radar, gives full play to the advantages of each sensor, overcomes the limitations of a single sensor, and improves the accuracy of target detection and the comprehensiveness of environmental perception.
[0083] Improved YOLOv8 algorithm: Enhances the accuracy of target detection and adaptability to complex environments, and improves target recognition performance in harsh environments.
[0084] High-precision GPS module: Provides robots with real-time and accurate positioning information, supports multi-robot collaborative operation and dynamic path planning, and ensures the orderly conduct of rescue operations.
[0085] (III) Cooperative Operations Capability
[0086] Advantages: The system supports efficient collaborative work among multiple rescue robots. By sharing environmental information and task status, the robots can synchronize data in real time and achieve joint operations. This collaborative mechanism can optimize task allocation, avoid resource conflicts, and through dynamic path planning, multiple robots can approach the target from different directions, effectively improving rescue efficiency and enhancing the safety of rescue operations.
[0087] Corresponding technical points:
[0088] Swarm intelligence collaborative control algorithm: Enables information sharing and collaborative decision-making among multiple robots, and optimizes task allocation and path planning.
[0089] Communication and positioning modules: ensure stable communication and positioning synchronization between the rescue robot and the control center (host computer) and other robots, enabling efficient collaborative operations.
[0090] (iv) High level of intelligence and automation
[0091] Advantages: The system features autonomous navigation, automatic return-to-home, and return-to-home upon loss of contact, reducing manual intervention, simplifying operations, and improving rescue efficiency and accuracy. Furthermore, intelligent technology makes rescue operations more reliable.
[0092] Corresponding technical points:
[0093] Swarm intelligence collaborative control algorithm: By analyzing and learning from feedback data during the rescue process through machine learning algorithms, the algorithm continuously optimizes task allocation and path planning strategies, thereby improving the level of intelligence.
[0094] Data processing and system integration: Integrating hardware, software and communication systems to achieve joint control and information sharing of multiple modules, thereby improving the system's intelligence and automation level.
[0095] This application also provides an application scenario in which the above-described multimodal water rescue robot collaborative control method is applied. Specifically, the multimodal water rescue robot collaborative control method provided in this embodiment can be applied in water rescue scenarios. This scenario includes a data acquisition stage, a data processing stage, and a rescue stage; the data acquisition stage is used to collect multimodal data of the target water area; the data processing stage generates rescue instructions based on the collected multimodal data of the target water area; and the rescue stage executes rescue actions according to the rescue instructions. The multimodal water rescue robot collaborative control method provided in this embodiment belongs to the data processing stage.
[0096] Based on the same inventive concept, this application also provides a multimodal water rescue robot cooperative control system for implementing the multimodal water rescue robot cooperative control method described above. The solution provided by this device is similar to the implementation scheme described in the above method. Therefore, the specific limitations in one or more embodiments of the multimodal water rescue robot cooperative control system provided below can be found in the limitations of the multimodal water rescue robot cooperative control method described above, and will not be repeated here.
[0097] In one exemplary embodiment, such as Figure 3 As shown, a multimodal water rescue robot collaborative control system is provided, including:
[0098] The data acquisition module M1 is used to acquire multimodal data of the target water area; the multimodal data includes water environment images collected based on visual sensors, distance and depth information of underwater targets determined based on sonar sensors, and position and velocity information of underwater targets determined based on radar sensors.
[0099] The target determination module M2 is used to determine rescue targets within the target water area based on multimodal data.
[0100] The task allocation module M3 is used to determine the rescue task of each rescue robot based on the location information of the rescue target, the current location and status of each rescue robot, and the current water environment information.
[0101] The path planning module M4 is used to determine the optimal path for each rescue robot to reach the matching rescue target location based on the rescue target corresponding to the rescue mission of each rescue robot, the current location of the rescue robot, and the current water environment information.
[0102] The execution module M5 is used to drive each rescue robot to perform coordinated rescue actions based on the optimal path of each rescue robot.
[0103] In another exemplary embodiment of this application, a multimodal water rescue robot collaborative control device is provided, such as... Figures 4 to 7 As shown, it includes: a host computer, a water rescue robot body, a rescue mechanism 10 mounted on the water rescue robot body, and a controller.
[0104] The water rescue robot body and the rescue mechanism are connected to the controller.
[0105] Traditional mechanical grasping mechanisms suffer from insufficient stability and response speed under complex water flow and heavy loads. While robotic arms offer high flexibility and grasping accuracy, their stability and response speed remain inadequate under complex water flow and heavy loads. Hoisting mechanisms perform well in traction and linear rescue, but are inefficient and prone to entanglement when facing uncertain locations of drowning victims or complex obstacles. Therefore, the rescue mechanism 10 described in this application employs a biomimetic octopus tentacle rescue mechanism. Upon receiving a control command from the control mechanism, the miniature stepper motor at the end of the tentacle is activated. The motor drive circuit generates a corresponding drive current based on the control signal, causing the motor to rotate in a predetermined direction and speed. The movement of the tentacle joints drives the entire biomimetic octopus tentacle rescue mechanism to perform corresponding actions, such as extending and grasping. By precisely controlling the motor's movement, the flexible movement and grasping actions of the biomimetic octopus tentacle rescue mechanism are achieved, ensuring the accuracy and reliability of rescue operations and solving the problem of poor adaptability of existing robotic arms in mechanical grasping.
[0106] The biomimetic octopus tentacle rescue mechanism features a multi-segment flexible joint design. Each joint is connected by high-strength, lightweight engineering plastic, ensuring both the flexibility and structural strength of the tentacle. Steel wire ropes are embedded inside the tentacle to connect to motors for control; miniature stepper motors are connected to the ends of the octopus tentacles, which are then connected to the steel wire ropes via a rope-driven mechanism to achieve complex mechanical movements.
[0107] The controller is also connected to a host computer for communication.
[0108] The controller is used to receive instructions from the host computer, control the sensors on the water rescue robot to collect data and transmit it to the host computer according to the instructions, and drive the drive components on the water rescue robot to drive the rescue mechanism to perform rescue actions according to the instructions.
[0109] The host computer is used to execute the above-mentioned multimodal water rescue robot collaborative control method.
[0110] The water rescue robot body includes an outer hull 2, a bow 6 installed on the front side inside the outer hull, and a deck 3 installed in the middle and aft; a control mechanism installed above the deck 3; a rescue mechanism 10 installed at the rear of the deck 3; and a drive mechanism installed at the bottom of the outer hull 2 for driving the system to move.
[0111] The control mechanism includes a bottom housing 11, fixedly mounted on the deck 3; a middle housing 12, detachably mounted on the bottom housing 11; and a top housing 13, detachably mounted on the middle housing 12. An antenna 4 is fixedly mounted on the bottom housing 11; a battery 8 is installed inside the bottom housing 11; a microcontroller 15 is installed inside the bottom housing 11; a voltage regulator circuit 14 is installed inside the middle housing 12; and a first stepper motor 5 and a second stepper motor 7 are installed inside the bottom housing 13. The battery 8 is electrically connected to the voltage regulator circuit 14, the microcontroller 15 is electrically connected to the voltage regulator circuit 14, and the microcontroller 15 is electrically connected to the first stepper motor 5 and the second stepper motor 7. The drive mechanism includes two thrusters, a first thruster 1 and a second thruster 9, mounted on the rear side of the bottom of the outer hull 2. The rescue mechanism 10 includes a bionic octopus tentacle, mounted on the rear of the deck 3, which is linearly driven by the first stepper motor 5 and the second stepper motor 7.
[0112] The microcontroller is the controller of the underwater robot. The control circuit of the controller is focused on receiving instructions and data from the control center or remote terminal, exchanging information with the control center or remote terminal through the ESP8266 wireless communication module, decoding and converting the received instructions, and then transmitting them to the receiving circuit of the controller for execution through the LoRa communication module.
[0113] The receiver circuit of this controller is mainly responsible for the core control functions of the robot, including the acquisition and processing of sensor data, the driving and control of the drive mechanism, and communication and coordination with other modules. The controller enables precise management of the overall operation of the robot, ensuring that the robot can make accurate action responses based on preset algorithms and real-time decision information.
[0114] This application develops a concept for collaborative control of a water rescue robot. Through multimodal information fusion algorithms, swarm intelligence collaborative control algorithms, and a biomimetic octopus tentacle rescue mechanism, it aims to overcome the limitations of existing technologies. The multimodal information fusion algorithm integrates data from multiple sensors, including vision, sonar, and radar, fully leveraging the advantages of each sensor and overcoming the limitations of a single sensor, thereby achieving more reliable rescue missions in harsh environments. The swarm intelligence collaborative control algorithm enables multiple rescue robots to share location, target, and environmental information in real time, dynamically adjusting task allocation and path planning, optimizing rescue paths, rationally allocating tasks, and avoiding resource waste. The biomimetic octopus tentacle rescue mechanism, inspired by the flexible movement and powerful grasping ability of octopus tentacles, can adapt to people of different sizes who have fallen into the water, automatically adjusting the grasping force and angle to ensure stable and safe rescue of people in complex water currents and obstacle environments. Through the application of these innovative technologies, we are committed to creating a water rescue robot system capable of efficiently and accurately performing rescue missions in complex aquatic environments, significantly improving rescue success rates and survival rates.
[0115] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 8 As shown, the computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores collaborative control data for multimodal water rescue robots. The I / O interfaces are used for information exchange between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a collaborative control method for multimodal water rescue robots.
[0116] Those skilled in the art will understand that Figure 8The structures shown are merely block diagrams of some structures related to the present application and do not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than shown in the figures, or combine certain components, or have different component arrangements. In an exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.
[0117] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.
[0118] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.
[0119] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.
[0120] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).
[0121] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.
[0122] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0123] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.
Claims
1. A collaborative control method for a multimodal water rescue robot, characterized in that, include: Acquire multimodal data of the target water area; the multimodal data includes water environment images acquired based on visual sensors, distance and depth information of underwater targets determined based on sonar sensors, and position and velocity information of underwater targets determined based on radar sensors; Determine rescue targets within the target waters based on multimodal data; The rescue mission of each rescue robot is determined based on the location information of the rescue target, the current location and status of each rescue robot, and the current water environment information. The optimal path for each rescue robot to reach the matching rescue target is determined based on the rescue target corresponding to the rescue mission of each rescue robot, the current location of the rescue robot, and the current water environment information. Each rescue robot is driven to perform coordinated rescue actions based on its optimal path.
2. The multimodal water rescue robot cooperative control method according to claim 1, characterized in that, Based on multimodal data, rescue targets within the target waters are identified, specifically including: The aquatic environment image is input into the YOLOv8 model to obtain the target detection results; Based on the reflected signals received by the sonar sensor, the distance and depth information of the underwater target are calculated; The position and velocity information of underwater targets are determined based on the reflected signals received by radar sensors. The distance and depth information of underwater targets determined by sonar sensors and the position and velocity information of underwater targets determined by radar sensors are normalized to obtain normalized sonar sensing data and normalized radar sensing data. Based on the target detection results, the normalized sonar sensing data and normalized radar sensing data are synchronized and aligned in time to obtain the aligned sonar sensing data and aligned radar sensing data. The target detection results, aligned sonar sensing data, and aligned radar sensing data are filtered to obtain filtered target detection data, filtered sonar sensing data, and filtered radar sensing data. The filtered target detection data, filtered sonar sensing data, and filtered radar sensing data are fused to obtain multi-source fused data. Rescue targets within the target waters are determined based on multi-source fusion data.
3. The multimodal water rescue robot cooperative control method according to claim 2, characterized in that, The YOLOv8 model adopts an improved YOLOv8 model; in the improved YOLOv8 model, the C2f module is replaced by the C2f-ContextGuided module, and the PANet module is replaced by the AFPN-P345 module.
4. The multimodal water rescue robot cooperative control method according to claim 1, characterized in that, The rescue mission for each rescue robot is determined based on the location information of the rescue target, the current location and status of each rescue robot, and the current water environment information. Specifically, this includes: Based on the location information of the rescue target, the current location and status of each rescue robot, and the current water environment information, an optimization algorithm is applied to determine the rescue task of each rescue robot; the optimization algorithm includes ant colony optimization and particle swarm optimization.
5. The multimodal water rescue robot cooperative control method according to claim 1, characterized in that, Based on the rescue target corresponding to the rescue mission of each rescue robot, the current location of the rescue robot, and the current water environment information, the optimal path for each rescue robot to reach the matched rescue target location is determined, specifically including: Based on the rescue target corresponding to the rescue mission of each rescue robot, the current location of the rescue robot, and the current water environment information, an optimization algorithm is applied to determine the optimal path for each rescue robot to reach the matching rescue target location; the optimization algorithm includes ant colony algorithm and particle swarm algorithm.
6. A multimodal water rescue robot collaborative control system, characterized in that, include: The data acquisition module is used to acquire multimodal data of the target water area; the multimodal data includes water environment images collected based on visual sensors, distance and depth information of underwater targets determined based on sonar sensors, and position and velocity information of underwater targets determined based on radar sensors. The target determination module is used to determine rescue targets within the target water area based on multimodal data; The task allocation module is used to determine the rescue task of each rescue robot based on the location information of the rescue target, the current location and status of each rescue robot, and the current water environment information. The path planning module is used to determine the optimal path for each rescue robot to reach the matching rescue target location based on the rescue target corresponding to the rescue mission of each rescue robot, the current location of the rescue robot, and the current water environment information. The execution module is used to drive each rescue robot to perform coordinated rescue actions based on the optimal path of each robot.
7. A multimodal water rescue robot collaborative control device, characterized in that, include: The host computer, the water rescue robot body, the rescue mechanism and controller installed on the water rescue robot body; The water rescue robot body and the rescue mechanism are connected to the controller; the rescue mechanism adopts a biomimetic octopus tentacle rescue mechanism. The controller is also connected to a host computer for communication. The controller is used to receive instructions from the host computer, control the sensors on the water rescue robot body to collect data and transmit it to the host computer according to the instructions, and drive the drive components on the water rescue robot body to drive the rescue mechanism to perform rescue actions according to the instructions. The host computer is used to execute the multimodal water rescue robot collaborative control method according to any one of claims 1 to 5.
8. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the multimodal water rescue robot collaborative control method according to any one of claims 1-5.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the multimodal water rescue robot cooperative control method according to any one of claims 1-5.
10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the multimodal water rescue robot cooperative control method according to any one of claims 1-5.