A method and system for salvaging floating objects on the water surface based on the cooperation of multiple ships

Abstract

The present invention relates to the technical field of surface garbage salvage, and proposes a method and system for salvaging floating objects on the water surface based on multi-vessel cooperation. The method and system are implemented based on a mother ship and two daughter ships, with a flexible barrier net belt provided between the mother ship and the daughter ships. The barrier net belt is used to form a salvage area in the direction of the mother ship's travel, so that the mother ship can salvage floating objects in the salvage area while traveling. The method for salvaging floating objects on the water surface includes the following steps: S1: a drone collects water surface images and constructs an initial model. The method and system for salvaging floating objects on the water surface based on multi-vessel cooperation forms a salvage area by providing a mother ship and two daughter ships, and collects images by drone to establish an environmental map. Based on the characteristics of the salvage area, salvage nodes are generated in the environmental image, and a mother ship movement path is generated. The daughter ships cooperate with the mother ship according to the mother ship's movement path to quickly and efficiently salvage floating garbage in the water area.

Description

Technical Field

[0001] The present invention relates to the technical field of surface garbage salvage, and in particular to a method and system for salvaging surface floating objects based on multi-vessel cooperation. Background Art

[0002] The salvage of floating garbage on the water surface is an important environmental protection task, aimed at cleaning up floating garbage on the surface of rivers, lakes, oceans and other water bodies. These garbage not only affects the landscape, but also causes serious harm to aquatic ecosystems, threatens the survival of aquatic organisms, and may affect human health and economic activities. The main methods of salvaging floating garbage include manual salvage and mechanized salvage. Manual salvage is usually suitable for small areas of water and can flexibly deal with different types of garbage. Mechanized salvage uses specially designed salvage ships and equipment, which is more efficient and suitable for cleaning large areas of water. In recent years, with the development of science and technology, intelligent and automated salvage equipment has gradually been applied to actual operations, such as unmanned salvage ships and surface garbage collection robots, which can monitor the situation of surface garbage in real time and perform automatic cleaning, improving work efficiency and garbage cleaning effects.

[0003] However, unmanned salvage ships and surface garbage collection robots are usually single-device operations with limited coverage, making it difficult to cope with the needs of cleaning floating objects in large areas of water or complex waters, which directly leads to their low efficiency in cleaning floating objects in large areas of water or complex waters. At the same time, traditional unmanned salvage ships and garbage collection robots mainly rely on simple sensors or basic image processing technology to detect surface garbage, and are unable to accurately assess the global environment and garbage distribution of the water area, which directly leads to the inability to optimize the path during garbage salvage operations, which will directly lead to repeated salvage or missed areas, thereby reducing salvage efficiency. Summary of the Invention

[0004] In view of this, the present invention proposes a method and system for salvaging floating objects on the water surface based on multi-ship cooperation. By setting up a mother ship and two daughter ships, a salvage area is formed, and the drone collects images to establish an environmental map. According to the characteristics of the salvage area, salvage nodes are generated in the environmental image, and the mother ship's movement path is generated. The daughter ships cooperate with the mother ship according to the mother ship's movement path to carry out floating garbage salvage operations in the water area quickly and efficiently.

[0005] The technical solution of the present invention is achieved as follows:

[0006] On the one hand, the present invention provides a method for salvaging floating objects on the water surface based on multi-ship cooperation, which is realized based on a mother ship and two sub-ships. A flexible blocking net belt is arranged between the mother ship and the sub-ships. The blocking net belt is used to form a salvage area in the advancing direction of the mother ship, so as to salvage the floating objects in the salvage area during the advancing process of the mother ship. The method for salvaging floating objects on the water surface includes the following steps:

[0007] S1 Collect water surface images by an unmanned aerial vehicle (UAV) and construct an initial model;

[0008] S2 Train the initial model through the water surface images to obtain a target detection model, which is used to mark the category and position information of the floating objects on the water surface images;

[0009] S3 Collect real-time water surface images, input the real-time water surface images into the target detection model, and obtain a marked image of the floating objects;

[0010] S4 Form an initial map through the marked image of the floating objects, and integrate the marked information of the target floating objects into the initial map to form an environmental map;

[0011] S5 Generate a moving path of the mother ship on the environmental map based on the salvage area;

[0012] S6 The mother ship and the sub-ships move on the water surface according to the moving path of the mother ship to carry out the operation of salvaging floating objects.

[0013] Based on the above technical solutions, preferably, the step S4 includes the following sub-steps:

[0014] S41 Obtain the pose of the UAV collecting the real-time water surface images, and extract feature points on the marked image of the floating objects;

[0015] S42 Match the feature points of the marked images of the floating objects in consecutive frames;

[0016] S43 Based on the UAV pose information, obtain the three-dimensional space coordinates of the matched feature points and generate an initial map;

[0017] S44 Corresponding the marked information of the target floating objects in the marked image of the floating objects to the initial map to form an environmental map.

[0018] Based on the above technical solutions, preferably, the step S5 includes the following steps:

[0019] S51 Generate several salvage nodes in the environmental map according to the characteristics of the salvage area;

[0020] S52 Generate a moving path of the mother ship according to the salvage nodes on the environmental map.

[0021] Further preferably, the category of floating objects includes garbage, and step S51 includes the following steps:

[0022] S511 Determine the length of the entrance of the salvage area, that is, the spacing between the two sub-ships during salvage operations;

[0023] S512 According to the target floating objects of the garbage category, divide several circular areas on the environmental map so that all target floating objects of the garbage category are at least located in one of the circular areas;

[0024] S513 Use the center of the circular area as the salvage node.

[0025] Further preferably, step S52 includes using the Dijkstra algorithm to find the shortest path by traversing all salvage nodes in the environmental map and optimizing it to obtain the moving path of the mother ship.

[0026] Based on the above technical solutions, preferably, in step S6, the mother ship moves on the water surface, including the following steps:

[0027] S61 Extract path points from the moving path of the mother ship;

[0028] S62 Convert the path points into a two-dimensional array, and each element in the two-dimensional array represents the coordinates of the corresponding path point;

[0029] S63 According to the coordinates of the three consecutive path points that the mother ship is about to reach in the two-dimensional array, calculate the curvature and length of the arc segment formed by the current position of the mother ship and these three path points;

[0030] S64 Generate a control command according to the curvature and length of the arc segment. The control command includes heading, speed, and curvature;

[0031] S65 The mother ship moves through the control command in step S64, and when moving to the first of the three consecutive path points in step S63, returns to step S63 until there are less than three subsequent path points that the mother ship is about to reach, and moves along the arc segment formed by the last three path points.

[0032] Based on the above technical solutions, preferably, in step S6, the sub-ships move on the water surface according to the moving path of the mother ship, including the following steps:

[0033] S66 Set the relative positions of the two sub-ships and the mother ship;

[0034] S67 Offset the moving path of the mother ship to both sides to obtain the synchronous paths of the two sub-ships respectively;

[0035] The two sub-ships move respectively according to two synchronous paths, and calibrate the relative positions with the mother ship in real time.

[0036] Further preferably, the step S6 further includes that if an obstacle is encountered during the movement of the sub-ship, the local path of the sub-ship near the obstacle is re-planned, and this plan will not allow the obstacle to enter the salvage area during the movement of the mother ship and the sub-ship.

[0037] Further preferably, the step S6 further includes that if an obstacle is encountered during the movement of the mother ship, the local path of the mother ship near the obstacle is re-planned, and at the same time, the synchronous paths of the two sub-ships are adjusted.

[0038] On the other hand, the present invention provides a water surface floating object salvage system based on multi-ship cooperation, including an unmanned aerial vehicle, a ground control station, a mother ship and two sub-ships, wherein,

[0039] The unmanned aerial vehicle is used to collect real-time images of the water surface;

[0040] The ground control station is used to receive the real-time images of the water surface collected by the unmanned aerial vehicle and the attitude position information of the unmanned aerial vehicle, and implement the above-mentioned water surface floating object salvage method;

[0041] The mother ship and the two sub-ships are used to receive the control instructions issued by the ground control station and perform garbage salvage operations according to the control instructions.

[0042] The water surface floating object salvage method and system based on multi-ship cooperation of the present invention have the following beneficial effects compared with the prior art:

[0043] (1) By setting one mother ship and two sub-ships to form a salvage area, and the unmanned aerial vehicle collects images to establish an environmental map, according to the characteristics of the salvage area itself, salvage nodes are generated in the environmental image, and the moving path of the mother ship is generated. The sub-ships cooperate with the mother ship to quickly and efficiently salvage the floating garbage in the water area according to the moving path of the mother ship;

[0044] (2) By generating salvage nodes based on the characteristics of the salvage area and planning the moving path of the mother ship, the efficiency, coverage rate and safety of the salvage operation are significantly improved, the resource utilization is optimized, the operation cost is reduced, and it has strong environmental adaptability and automation level, so that efficient and precise salvage operations can still be achieved in complex water area environments. Description of the Drawings

[0045] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0046] Figure 1 Schematic diagram of the steps of the method for salvaging floating objects on the water surface of the present invention;

[0047] Figure 2 Schematic diagram of the salvage area formed by the mother ship and the sub-ships in the method for salvaging floating objects on the water surface of the present invention. Detailed implementation manners

[0048] The following will combine the implementation manners of the present invention to clearly and completely describe the technical solutions in the implementation manners of the present invention. Obviously, the described implementation manners are only a part of the implementation manners of the present invention, rather than all of the implementation manners. Based on the implementation manners in the present invention, all other implementation manners obtained by those of ordinary skill in the art without creative efforts belong to the scope protected by the present invention.

[0049] As Figure 1 shown, the method for salvaging floating objects on the water surface based on the cooperation of multiple ships of the present invention is specifically realized based on one mother ship and two sub-ships. A flexible blocking net belt is arranged between the mother ship and the sub-ships. The blocking net belt is used to form a salvage area in the advancing direction of the mother ship for salvaging floating objects in the salvage area during the advancement of the mother ship.

[0050] In a specific embodiment, as Figure 2 shown, a garbage inlet, a collection channel and a garbage storage position are arranged on the mother ship. The garbage inlet is located in the advancing direction of the mother ship. The collection channel is connected to the garbage inlet. The garbage storage position is used to store the garbage entering the mother ship. The mother ship is preferably a catamaran, that is, composed of two separated underwater hulls and an upper strengthening structure, and the two hulls are connected by the upper strengthening structure to form an integral ship. Its bottom has an arched structure. One end of the arched channel is the garbage inlet, and the other end can be used as the garbage storage position. The middle part of the arched channel can be used as the collection channel connecting the garbage inlet and the garbage storage position. After selecting a catamaran as the mother ship, there is no need to independently design the garbage inlet, the collection channel and the garbage storage position, which is more conducive to the realization of this embodiment. At the same time, its good stability and flexibility can also provide favorable conditions for salvaging floating garbage on the water surface.

[0051] Both daughter ships are arranged outside the mother ship and can move relative to the mother ship. The two daughter ships are independently arranged outside the mother ship and have two separate propellers to drive the movement of the daughter ships. The daughter ships are also preferably catamarans, and in order to improve the stability of the daughter ships, and considering that the arresting net belt needs to be pulled during the movement of the daughter ships, the propellers on the daughter ships are centered on their underwater hulls to further improve the carrying capacity and stability of the daughter ships.

[0052] One end of the two blocking net belts is set on the mother ship and is located on both sides of the garbage inlet in the horizontal direction. The other ends of the two blocking net belts are respectively connected to the tail ends of the two daughter ships. The blocking net belts include an underwater part and an above-water part, and floating objects on the water surface are blocked and collected through the underwater part and the above-water part.

[0053] During the salvage operation, the two barrier nets form a salvage area in the direction of the mother ship's movement, and allow floating objects in the salvage area to enter the mother ship through the garbage inlet. Specifically, the salvage area can be funnel-shaped, so that when the mother ship and the daughter ship are moving, the floating objects in the salvage area are blocked by the barrier nets, gradually move to the center of the salvage area, and enter the garbage inlet. After entering the garbage inlet, the floating garbage will enter the garbage storage position through the collection channel, thereby realizing the rapid salvage operation of floating garbage in a certain area.

[0054] The method for salvaging floating objects on the water surface in this embodiment preferably uses the daughter ship and mother ship in the above embodiment to perform salvage operations, and other types of ships with garbage storage and collection functions can also be used as mother ships, specifically including steps S1-S6.

[0055] Step S1: The UAV collects water surface images and constructs an initial model.

[0056] During the acquisition process, the drone is equipped with a high-resolution camera to shoot and form continuous frame images, namely water surface images. The collected water surface images are the training data for the initial model. Before training, the frame images need to be preprocessed, including using the OpenCV library for image enhancement, and using histogram equalization to improve the contrast of the image to make the dark and bright details clearer; and using filtering (such as Gaussian filtering and median filtering) to reduce noise in the image and improve image quality.

[0057] After preprocessing the water surface image, annotation software is used to annotate the floating objects in the water surface image. The annotation information includes the type of floating objects and their location in the image. The types include obstacles, garbage, etc.

[0058] Step S2: An initial model is trained using water surface images to obtain a target detection model, which is used to mark the category and location information of floating objects on the water surface images.

[0059] The pre - processed water surface image is used as the training set to train the initial model, and then the target detection model is obtained. In this embodiment, the YOLOv7 is selected as the target detection model.

[0060] First, download the YOLOv7 code library from GitHub and obtain the pre - trained weight files, such as yolov7.pt, etc. In step S1, the pre - processed water surface images are labeled. The labeling is selected to be in the YOLO format. The YOLO - format labeling includes a text file for each image, which contains the target category and location information. For the data set formed by the pre - processed water surface images, it is divided into a training set, a validation set, and a test set, and the ratio can be selected as 7:2:1. The training set is used to train the initial model, and during the training process, the validation set is used to evaluate the model performance (not participating in the model training process) to adjust the model parameters and monitor the over - fitting situation of the model.

[0061] After the training is completed, the test set is used to evaluate the model to understand the model's performance on the test set, and the trained model is used as the target detection model.

[0062] During the training process, by observing whether the loss value is stable or drops to a certain range, whether the evaluation metrics (such as mAP) of the validation set are stable around a certain value, and by checking the training log to ensure that the loss and evaluation metrics no longer fluctuate significantly, it is determined whether the model training is completed.

[0063] In addition, in order to improve the training speed of YOLOv7, the cosine annealing algorithm can be used to improve the convergence speed and performance of the model.

[0064] Step S3: Collect real - time water surface images, input the real - time water surface images into the target detection model, and obtain the floating object labeled images.

[0065] The real - time water surface images are collected by an unmanned aerial vehicle (UAV). On the UAV, a high - resolution camera, GPS, and IMU need to be equipped. The collected real - time water surface images are synchronized with the position and attitude information of the UAV sensors.

[0066] The images collected by the UAV will be transmitted through wireless communication to ensure the real - time and stability of the data. After transmission, the video decoder will decode the received compressed video stream, restore it to frame images, and process them. Specifically, the OpenCV library is used for image enhancement processing. The contrast of the image is enhanced through histogram equalization to make the details of the dark and bright parts clearer; the noise in the image is reduced through filtering (such as Gaussian filtering, median filtering) to improve the image quality.

[0067] In this step, continuous floating object labeled images will be obtained.

[0068] Step S4: Annotate the image with floating objects to form an initial map, and integrate the annotation information of the target floating object into the initial map to form an environmental map.

[0069] After obtaining the floating object annotated image, it needs to be converted into a map to provide environmental information data for subsequent salvage path planning.

[0070] Specifically, step S4 includes steps S41 - S44.

[0071] Step S41: Obtain the pose of the UAV collecting the real - time water surface image, and extract feature points on the floating object annotated image.

[0072] While the UAV is collecting the real - time water surface image, obtain its pose information (including position and attitude). The position information is usually obtained through GPS, and the attitude information includes heading angle, pitch angle, and roll angle, which can be obtained through sensors such as inertial measurement unit (IMU). Process the collected real - time water surface image to identify and extract feature points in the image. Feature points can be corner points, edges, or specific texture regions in the image. Commonly used feature extraction algorithms include SIFT, SURF, ORB, etc.

[0073] Step S42: Match the feature points of consecutive frames of the floating object annotated image.

[0074] In consecutive frames of images, match the already extracted feature points. The purpose of matching is to find the same floating object feature points in images collected at different time points to determine their corresponding relationships in different images. Methods for feature point matching include distance - based nearest neighbor matching, FLANN matcher, etc. Through these algorithms, the same floating object feature points can be found in consecutive frames.

[0075] In addition, it is necessary to evaluate and optimize the matching results, eliminate the wrongly matched point pairs, and further improve the matching accuracy using methods such as the RANSAC algorithm. It should be noted that if the position and shape of the floating object change significantly in consecutive frames, a more complex matching strategy, such as optical flow method, needs to be adopted to capture the dynamic changes.

[0076] Step S43: Based on the UAV pose information, obtain the three - dimensional spatial coordinates of the matched feature points and generate an initial map.

[0077] Using the drone's pose information and the matched feature points, the 3D coordinates of these feature points are calculated. In this implementation, triangulation is used to determine the 3D coordinates of the feature points. This involves combining the relative positions and angles of multiple image frames to infer the locations of the feature points in 3D space. Based on the calculated 3D coordinates of the feature points, a preliminary 3D map is generated. This initial map primarily includes the spatial distribution of floating objects on the water surface.

[0078] The accuracy of the initial map depends on the accuracy of feature point matching and the effect of the 3D reconstruction algorithm. The map can be further optimized using SLAM (Simultaneous Localization and Mapping) technology.

[0079] Step S44: mapping the annotation information of the target floating object in the floating object annotation image to the initial map to form an environment map.

[0080] Map the annotation information of floating objects in the image (such as location, category, shape, etc.) to the initial three-dimensional map. This process requires converting the two-dimensional image coordinates into coordinates in the three-dimensional map. Specifically, the camera must be calibrated before the salvage operation to obtain the camera's internal parameters (internal parameters) and external parameters (external parameters). Internal parameters include focal length, principal point coordinates, distortion coefficients, etc., and external parameters include the rotation matrix and translation vector of the camera relative to the world coordinate system. The two-dimensional image is then mapped to the three-dimensional map based on the depth information. The depth information here requires a lidar to be set up on the drone and obtained through the lidar.

[0081] By superimposing the annotation information of all frames onto the initial map, a final environment map is formed. The environment map contains not only the locations of floating objects, but also their categories and other attribute information.

[0082] Step S5: Based on the salvage area, generate the mother ship's moving path on the environment map.

[0083] The salvage area is mainly funnel-shaped, with an opening formed on the water surface. The width of the opening is determined according to the performance of the mother ship and the daughter ship, the length of the barrier net, etc. When generating the mother ship's movement path on the environmental map, it is necessary to fully consider the characteristics of the salvage area, which specifically includes steps S51-S52.

[0084] Step S51: generating a number of salvage nodes in the environment map according to the characteristics of the salvage area.

[0085] The salvage area features can be directly set according to actual conditions or adjusted in real time. In order to match the planned path with the salvage area and minimize the required walking distance, the salvage node can be generated based on the maximum coverage of the salvage area. Specifically, it can be subdivided into steps S511-S513.

[0086] Step S511: Determine the length of the entrance to the salvage area, i.e., the spacing between the two sub-ships during salvage operations.

[0087] Step S512: Based on the target floating objects classified as garbage, divide several circular areas on the environmental map so that all target floating objects classified as garbage are at least located in one of the circular areas.

[0088] When generating salvage nodes, according to the distribution of target floating objects, especially the target floating objects classified as garbage, several circular areas are planned. The center of each circular area can be regarded as the reference point of the salvage node. In addition, it is necessary to ensure that each target floating object classified as garbage can be covered by at least one circular area. This can ensure that the salvage operation can completely cover all garbage and avoid omission.

[0089] The length of the entrance to the salvage area is directly positively correlated with the diameter of the circular area. The larger the diameter of the circular area, the shorter the salvage path. However, correspondingly, the smaller the radius, the higher the salvage accuracy, but it may increase the path length.

[0090] Specifically, coverage optimization algorithms (such as Voronoi diagrams, K-means clustering, etc.) can be used to automatically divide and optimize these circular areas to make their coverage efficiency the highest. At the same time, during real-time salvage operations, drones need to be combined with the cameras on the mother ship to monitor and adjust these circular areas in real time. Especially when the distribution of garbage floating objects changes over time, the position and size of the circular areas need to be dynamically adjusted.

[0091] Step S513: Use the center of the circular area as the salvage node.

[0092] The salvage nodes determined in this step are the nodes that the mother ship needs to pass through during salvage operations.

[0093] Step S52: Generate the movement path of the mother ship according to the salvage nodes on the environmental map.

[0094] Adopt the Dijkstra algorithm to find the shortest path by traversing all salvage nodes in the environmental map and optimize it to obtain the movement path of the mother ship.

[0095] In step S51, several salvage nodes have been generated based on the characteristics of the salvage area. These salvage nodes are the key positions that the mother ship needs to pass through during the salvage mission. The task of this step is to generate an optimal movement path according to these salvage nodes to ensure that the mother ship can cover all salvage nodes efficiently with the shortest distance. The core of path planning is to find a path that can minimize the total movement distance and avoid repeated coverage and unnecessary paths. The Dijkstra algorithm is a commonly used graph search algorithm that can effectively find the shortest path from the starting point to the ending point.

[0096] Specifically, a graph model is constructed, where the salvage nodes are regarded as vertices in the graph, and the edges connecting the salvage nodes represent the possible paths for the mother ship to move between two nodes. The weight of an edge can be the physical distance or the time consumed between nodes. By applying Dijkstra's algorithm, starting from the initial position of the mother ship, all salvage nodes are traversed to find the shortest path covering all nodes. Based on the initially generated shortest path, path optimization is carried out using smoothing filters (such as Bezier curves and spline curves to smooth the turning angles and curves on the path) or the gradient descent method (optimizing the positions of each point on the path to make the overall path shorter or smoother), making it smoother and more suitable for actual execution.

[0097] The basic idea of path smoothing optimization is to adjust the positions of path points to make the path smoother, avoid sharp turns, and improve the stability and efficiency of navigation. There are various commonly used path smoothing algorithms, including average smoothing, Gaussian smoothing, and cost function-based methods. In this embodiment, it is preferably to use cost function-based path smoothing optimization. Its basic principle is that the path smoothing algorithm achieves path smoothing by minimizing a certain cost function. Commonly used cost functions include a data retention term and a smoothing term. The data retention term is used to ensure that the smoothed path is as close as possible to the original path, and the smoothing term is used to ensure the smoothness of the path and avoid sharp turns. Its specific cost function form is:

[0098]

[0099] where, P i is the optimized path point, is the original path point, α and β are weight coefficients used to balance the importance of the data retention term and the smoothing term, and N is the number of path points.

[0100] Step S6: The mother ship and the sub-ship move on the water surface according to the moving path of the mother ship to carry out the operation of salvaging floating objects.

[0101] The task of this step is to make the mother ship and the sub-ship work together, move on the water surface along the predetermined path, and perform the actual salvaging operation. The sub-ship usually works in coordination with the mother ship, used to cover areas that are difficult for the mother ship to reach or to carry out more refined salvaging operations. The sub-ship usually moves closely behind the mother ship or on both sides of the mother ship to expand the salvaging coverage.

[0102] Specifically, the mother ship in step S6 moves on the water surface according to the moving path of the mother ship, including steps S61 - S65.

[0103] Step S61: Extract path points from the moving path of the mother ship.

[0104] The mother ship's movement path is composed of a series of path points, and each path point corresponds to the position that the mother ship needs to pass through during the salvage operation. The task of this step is to extract these key path points from the entire path as the basis for subsequent path planning and control instruction generation.

[0105] Step S62: Convert the path points into a two-dimensional array, and each element in the two-dimensional array represents the coordinates of the corresponding path point.

[0106] After the path points are extracted, the coordinates of these path points need to be converted into a two-dimensional array, where each element represents the coordinates of a path point. This array will be used for subsequent path calculation and control instruction generation. Each row of the two-dimensional array corresponds to a path point, and the columns represent the coordinates of the path point (such as x, y).

[0107] Step S63: According to the coordinates of three consecutive path points that the mother ship is about to reach in the two-dimensional array, calculate the curvature and length of the arc segment formed by the mother ship's current position and these three path points.

[0108] By selecting three consecutive path points that the mother ship is about to reach, calculate the geometric properties of the arc segment formed by them, including curvature (the degree of bending of the curve) and the length of the arc. These parameters will be used to generate instructions for controlling the movement of the mother ship. For some path points that form a straight line, they will be regarded as special arc segments.

[0109] Step S64: Generate control instructions according to the curvature and length of the arc segment. The control instructions include heading, speed, and curvature.

[0110] Utilize the curvature and length of the arc segment calculated in the previous step to generate instructions for controlling the movement of the mother ship. The instruction content usually includes adjusting the heading, speed, and curvature of the mother ship to ensure that the mother ship moves smoothly along the predetermined path. The heading instruction will determine the direction of the mother ship, the speed instruction controls the forward speed, and the curvature instruction adjusts the turning angle and speed. These instructions can be directly transmitted to the mother ship's autopilot system to ensure that it can turn and accelerate / decelerate smoothly when performing tasks.

[0111] Step S65: The mother ship moves according to the control instructions in Step S64, and when it moves to the first of the three consecutive path points in Step S63, it returns to Step S63 until there are less than three subsequent path points that the mother ship is about to reach, and then it moves along the arc segment formed by the last three path points.

[0112] This loop mechanism ensures that the movement of the mother ship on the entire path is continuous and dynamically adjusted, enabling it to adapt to complex path conditions. When there are less than three path points, the system will automatically identify and process the remaining path points to smoothly complete the final salvage task.

[0113] The settings in this step can help the mother ship maintain precise path following in a complex water surface environment, while optimizing turning and speed control to ensure the safety and effectiveness of the operation.

[0114] In addition, in step S6, the sub-ships move on the water surface according to the moving path of the mother ship, including steps S66 - S68.

[0115] Step S66: Set the relative positions of the two sub-ships and the mother ship.

[0116] The sub-ships are usually located on both sides of the mother ship, forming a trapezoidal or side-by-side layout to maximize the salvage coverage. The task of this step is to determine the relative positions between the two sub-ships and the mother ship during the salvage operation. This relative position is preferably based on the center line of the mother ship.

[0117] Step S67: Offset the moving path of the mother ship to both sides to obtain the synchronous paths of the two sub-ships respectively.

[0118] According to the moving path of the mother ship, offset it by a certain distance to both sides of the path to generate the synchronous moving paths of the two sub-ships. These two paths will guide how the sub-ships move parallel to both sides of the mother ship to ensure full coverage of the salvage area.

[0119] The generated synchronous paths are parallel to the path of the mother ship, and the sub-ships will move along these paths, maintaining synchronization with the mother ship to ensure the coordination of the entire salvage operation.

[0120] Step S68: The two sub-ships move respectively according to the two synchronous paths and calibrate the relative positions with the mother ship in real time.

[0121] The two sub-ships move according to the synchronous paths generated in step S67. During the movement, the sub-ships need to continuously calibrate the relative positions with the mother ship to cope with the influence of environmental factors such as water flow and changes in the density of floating objects, ensuring that they perform the salvage operation at the correct positions.

[0122] Real-time calibration can be achieved through sensors, GPS positioning or other navigation systems, ensuring that the sub-ships always maintain the correct relative distance and angle with the mother ship during the movement. At the same time, during the calibration process, the sub-ships can make synchronous adjustments according to the actual movement of the mother ship (such as turning or speed changes) to prevent the offset of the relative position and ensure the salvage efficiency and operation safety.

[0123] In addition, if a sub-ship encounters an obstacle during the movement, the local path of the sub-ship near the obstacle is re-planned, and this planning will not allow the obstacle to enter the salvage area during the movement of the mother ship and the sub-ships. If the mother ship encounters an obstacle during the movement, the local path of the mother ship near the obstacle is re-planned, and at the same time, the synchronous paths of the two sub-ships are adjusted.

[0124] The surface floating object salvage system based on multi-ship combination of the present invention includes an unmanned aerial vehicle, a ground control station, a mother ship and two sub-ships. The unmanned aerial vehicle is used to collect real-time images of the water surface. The ground control station is used to receive the real-time images of the water surface collected by the unmanned aerial vehicle and the attitude position information of the unmanned aerial vehicle, and implement the above-mentioned method for salvaging surface floating objects. The mother ship and the two sub-ships are used to receive the control instructions issued by the ground control station and perform garbage salvage operations according to the control instructions.

[0125] Specifically, the sub-ship obtains its initial position through differential GPS (DGPS) and inertial navigation system (INS), and calibrates it with the real-time position of the mother ship to ensure that the three ships are in the correct relative positions at startup.

[0126] If the sub-ship encounters obstacles (such as floating branches, other ships, etc.) on its path, the system will detect the obstacles through lidar (LiDAR) and ultrasonic sensors, and use the Dijkstra algorithm for local path replanning to ensure that the sub-ship can avoid the obstacles without affecting the normal operation of the mesh belt. The path of the sub-ship will be optimized according to the environmental conditions and task requirements. Specifically, the rapidly-exploring random tree (RRT) and dynamic programming algorithm (DP) are combined to generate the optimal path, ensuring that while avoiding obstacles, the sub-ship can maintain high-efficiency garbage collection efficiency. (RRT can be used to quickly generate a preliminary path, and DP is used for subsequent path optimization and smoothing. This combination can balance real-time performance and path quality and adapt to more diverse task requirements.)

[0127] As a preferred implementation, during the driving process, images and environmental data are collected in real time through the vision system on the unmanned ship. The YOLOv7-based data processing system is used to detect, identify and classify the target objects in the images, and judge whether the target needs to be avoided or collected. The embedded intelligent control system of the unmanned ship (Dijkstra algorithm plans a new path) modifies the route and controls the unmanned ship to sail according to the new route to perform obstacle avoidance / collection tasks.

[0128] The unmanned ship is the mother ship and the sub-ships. The vision system includes a high-resolution camera (capturing images and videos of the surrounding environment), a lidar (for precise ranging and constructing a three-dimensional environmental model) sensor, and an ultrasonic sensor (fusing data with other sensor data such as LiDAR, radar, visible light camera) to improve the accuracy and robustness of obstacle detection).

[0129] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for salvaging floating objects on the water surface based on multi-vessel cooperation, characterized in that: The method is based on a mother ship and two daughter ships. A flexible barrier net is provided between the mother ship and the daughter ships. The barrier net is used to form a salvage area in the direction of the mother ship's movement, so that the mother ship can salvage floating objects in the salvage area during its movement. The method includes the following steps: S1 collects water surface images using a drone and builds an initial model; S2 trains an initial model using water surface images to obtain a target detection model, which is used to mark the category and location information of floating objects on the water surface images; S3 collects real-time images of the water surface, inputs the real-time images of the water surface into the target detection model, and obtains the floating object annotated images; S4 forms an initial map by annotating the floating object image, and integrates the annotation information of the target floating object into the initial map to form an environmental map; S5 generates the mother ship's movement path on the environmental map based on the salvage area; The S6 mother ship and daughter ship move on the water surface according to the mother ship's movement path to carry out floating object salvage operations; In step S6, the mother ship moves on the water surface according to the mother ship movement path, including the following steps: S61 extracts path points from the mother ship's moving path; S62 converts the path point into a two-dimensional array, and makes each element in the two-dimensional array represent the coordinates of the corresponding path point; S63 calculates the curvature and length of the arc segment formed by the mother ship's current position and the three consecutive path points that the mother ship is about to reach based on the coordinates of the three path points in the two-dimensional array; S64 generates a control instruction according to the curvature and length of the arc segment, wherein the control instruction includes heading, speed, and curvature; S65 The mother ship moves according to the control instructions in step S64, and when it moves to the first of the three consecutive path points in step S63, it returns to step S63 until the mother ship is about to reach less than three subsequent path points, and then moves along the arc segment formed by the last three path points.

2. The method for salvaging floating objects on the water surface based on multiple ships as claimed in claim 1, characterized in that: The step S4 includes the following sub-steps: S41 obtains the position and posture of the drone collecting real-time images of the water surface and extracts feature points on the floating object annotation images; S42 performs feature point matching on the floating object annotated images of consecutive frames; S43 obtains the three-dimensional spatial coordinates of the matching feature points based on the UAV posture information and generates an initial map; S44 maps the annotation information of the target floating object in the floating object annotation image to the initial map to form an environment map.

3. The method for salvaging floating objects on the water surface based on multi-vessel cooperation as claimed in claim 1, characterized in that: The step S5 comprises the following steps: S51 generates a number of salvage nodes in the environment map according to the characteristics of the salvage area; S52 generates a moving path of the mother ship according to the salvage nodes on the environment map.

4. The method for salvaging floating objects on the water surface based on multi-vessel cooperation as claimed in claim 3, characterized in that: The floating objects include garbage, and step S51 includes the following steps: S511 determines the length of the entrance to the salvage area, i.e., the distance between the two daughter vessels during the salvage operation; S512 divides the environmental map into a plurality of circular areas according to the target floating objects classified as garbage, so that all target floating objects classified as garbage are located in at least one of the circular areas; S513 uses the center of the circular area as the salvage node.

5. The method for salvaging floating objects on the water surface based on multi-vessel cooperation as claimed in claim 3, characterized in that: The step S52 includes using the Dijkstra algorithm to traverse all salvage nodes in the environment map, find the shortest path, and optimize it to obtain the moving path of the mother ship.

6. The method for salvaging floating objects on the water surface based on multi-vessel cooperation as claimed in claim 1, characterized in that: In step S6, the daughter ship moves on the water surface according to the moving path of the mother ship, including the following steps: S66 sets the relative positions of the two daughter ships and the mother ship; S67 shifts the moving path of the mother ship to both sides to obtain the synchronous paths of the two daughter ships; The two daughter ships of S68 move according to two synchronous paths respectively and calibrate their relative positions with the mother ship in real time.

7. The method for salvaging floating objects on the water surface based on multi-vessel cooperation as claimed in claim 6, characterized in that: The step S6 also includes replanning the local path of the daughter ship near the obstacle if the daughter ship encounters an obstacle during the movement, and the planning will not allow the obstacle to enter the salvage area during the movement of the mother ship and the daughter ship.

8. The method for salvaging floating objects on the water surface based on multi-vessel cooperation as claimed in claim 6, characterized in that: The step S6 also includes, if the mother ship encounters an obstacle during the movement, replanning the local path of the mother ship near the obstacle, and adjusting the synchronization paths of the two daughter ships.

9. A surface floating object salvage system based on multiple ships, characterized in that: It includes a UAV, a ground control station, a mother ship and two daughter ships, among which, Drones are used to collect real-time images of the water surface; The ground control station is used to receive real-time images of the water surface collected by the UAV and the attitude and position information of the UAV, and implement the method for salvaging floating objects on the water surface according to any one of claims 1 to 8; The mother ship and two daughter ships are used to receive control instructions from the ground control station and perform garbage salvage operations according to the control instructions.

Images