A Method and System for Unmanned Surface Vessel Waste Recycling Based on Multi-Source Fusion and Causal Reasoning

Abstract

This invention discloses an unmanned surface vessel (USV) waste collection method and system based on multi-source fusion and causal reasoning. It achieves physical consistency fusion of multi-source observation data and uses a multimodal deep learning network to detect, classify, and estimate the material parameters of floating waste. A two-dimensional local flow field is constructed around the USV, and short-term drift trajectory prediction and drift risk indicators are provided at the target scale. A parameterized causal prediction model is constructed and trained to evaluate the expected reduction in the overall water area's total waste load after cleaning each sub-region, resulting in a task priority sequence. A pre-contact behavior prediction network provides the probability of the target entering the effective collection area and the risk of being pushed away by the flow field under different approach directions and velocity strategies. Finally, a fuzzy control input is constructed to adaptively adjust the approach and collection behavior for different waste materials and flow field disturbances, maintaining high collection stability and robustness in complex hydrodynamic environments.

Description

Technical Field

[0001] This invention belongs to the technical field of waste recycling in complex waters, specifically relating to an unmanned vessel waste recycling method and system based on multi-source fusion and causal reasoning. Background Technology

[0002] In existing complex aquatic waste collection scenarios, unmanned surface vessel (USV) systems typically consist of a hull platform, a sensing unit, a mission planning unit, and an execution control unit. The sensing unit is often equipped with sensors such as RGB cameras, LiDAR, GNSS, IMU, and current sensors. It identifies floating debris on the water surface through image processing or target detection algorithms based on convolutional neural networks, obtaining the two-dimensional or three-dimensional location and category label of the debris. Some improved solutions add infrared or multispectral cameras in addition to RGB images, as well as simple water quality parameter acquisition, to enhance detection capabilities under low-light or turbid water conditions.

[0003] In terms of decision-making and task planning, existing technologies generally adopt task allocation strategies based on grids or region division. The operational water area is divided into several sub-regions, and the current amount of trash, target detection confidence, or target density within the grid are used as the basis for task priority, prioritizing sub-regions with higher trash quantities or higher confidence levels for cleanup. Some solutions may consider simple energy consumption and range constraints, but overall, the amount of trash in each sub-region remains the primary criterion for prioritization.

[0004] In terms of execution control, existing unmanned surface vessels (USVs) mostly employ PID control based on position and heading errors or trajectory tracking control with fixed parameters. The process of approaching the debris is usually achieved according to a preset route or a simple tracking strategy. The control law mainly relies on the geometric relationship between the target position and the hull position, lacking explicit modeling of water flow disturbances, surge changes, and differences in debris material. For the collection mechanism, most solutions use fixed opening degree, fixed drum speed, or simple on / off control.

[0005] In general, the closest existing technical approach can be summarized as follows: based on RGB or simple multimodal sensing, using statistically driven regional task ranking, combined with PID-type or fixed-parameter control, to achieve unmanned surface vessel detection and collection of debris. While existing solutions have demonstrated some practicality in engineering, they still exhibit significant shortcomings in recognition stability under complex flow fields and strong interference conditions, the foresight of task decision-making, and the adaptive capability of approach and collection actions.

[0006] (1) Multi-source perception lacks physical consistency modeling, making it difficult to stably support recognition and material discrimination in complex water surface scenarios.

[0007] While existing solutions incorporate RGB, LiDAR, water quality sensors, and some multispectral / infrared sensors, they mostly involve direct feature stitching or simple weighting at the feature level, failing to perform consistent modeling of different modalities under a unified physical model. Hyperspectral or multispectral data, if available, are often merely added to deep networks as additional image channels, lacking a physical inversion process based on water radiative transfer and medium attenuation. Under conditions of strong reflection, surges, and turbidity, surface reflection, spray, and foam significantly alter the appearance of images and point cloud features. Simple feature stitching easily leads to: ① surface reflections being misidentified as debris; ② missed detection of low-contrast or partially submerged debris; ③ debris only having category labels, lacking descriptions of physical properties such as buoyancy, wettability, and absorbency. These problems propagate to subsequent flow field estimation, task decision-making, and control modules, amplifying errors.

[0008] (2) Hydrodynamic effects are treated as background disturbances only and are not explicitly linked to the movement behavior of waste. There is a lack of modeling and prediction of waste drift behavior.

[0009] In existing technologies, flow velocity sensor data is mostly used to assist in ship path planning or trajectory correction, but the relative motion relationship between debris and water flow is not systematically modeled. Debris location is usually treated as a stationary target or detected independently at each moment, lacking prediction of how debris will drift under the influence of the flow field. This leads to two types of problems: ① At the task planning level, it is impossible to accurately assess whether debris in a certain sub-area will be carried away by the water flow or continuously replenished from upstream in a short period of time, thus making short-sighted decisions based solely on the current quantity; ② At the execution control level, the approach path and timing do not fully consider the drift trajectory of debris, often resulting in the target deviating from the collection window during the approach process, which can only be compensated for by temporary corrections.

[0010] (3) Task planning relies on the current quantity or confidence level of correlation ranking, lacking foresight in terms of intervention effect.

[0011] Existing unmanned surface vessel (USV) waste collection systems prioritize regional tasks based on the current amount of waste, detection confidence level, or local density. This approach has two limitations: ① It fails to incorporate factors such as upstream emissions, rainfall, wind fields, and local flow fields into a unified model, making it impossible to determine whether a particular sub-region, despite currently having a large amount of waste, may naturally disperse in the future; ② It focuses on the cleanup tasks of a specific sub-region while ignoring the impact of cleanup activities on the overall future waste load.

[0012] Therefore, task planning tends to focus on the current sub-areas where garbage accumulates, while neglecting the impact on the future evolution of garbage in the overall water area, resulting in the rapid re-accumulation of garbage after cleanup or the long-term neglect of important upstream areas.

[0013] (4) The approach and collection strategies lack pre-contact prediction and adaptive adjustment for materials and water flow, resulting in limited execution stability.

[0014] Existing control schemes are mostly based on geometric error-driven PID or fixed parameter control, which adjusts propulsion and rudder angles according to the position and heading errors between the target and the ship. They do not adequately consider the following factors: ① the dynamic drift trend of the target during the approach process; ② the influence of the material properties of the waste on the force and motion; ③ the influence of the approach speed and the opening time window of the collection port on the capture probability.

[0015] When there are significant surges or large changes in flow velocity, relying solely on position error correction can easily lead to the following problems: the target may be pushed away from the collection window by the water flow during the approach process, resulting in a missed target; excessive speed may cause excessive relative motion, resulting in decreased control accuracy; and the timing of the collection port opening may not match the actual time when the target passes through the collection area. Summary of the Invention

[0016] The main objective of this invention is to overcome the shortcomings and deficiencies of the prior art and provide an unmanned vessel waste recycling method and system based on multi-source fusion and causal reasoning.

[0017] To achieve the above objectives, the present invention adopts the following technical solution:

[0018] One aspect of the present invention provides an unmanned vessel waste recycling method based on multi-source fusion and causal reasoning, comprising the following steps:

[0019] The water area operation area is divided into several non-overlapping sub-regions; observation data from hyperspectral imagers, lidar, water quality sensors, IMUs, and flow velocity sensors are collected and time-aligned; and environmental state vectors for hydrological and meteorological monitoring are obtained.

[0020] Feature extraction is performed on the observation data to construct a joint feature tensor, which is then input into a multimodal deep learning network to obtain the garbage target attributes, including: the location and contour parameter vectors, category probability vectors, and material parameter vectors of the garbage target; the joint feature tensor includes spatial geometric features, intrinsic reflectance estimation, water quality parameter vectors, and perturbation feature vectors; the multimodal deep learning network includes a spatial branch for convolutional encoding of spatial geometric features, a spectral branch for spectral convolutional encoding of the intrinsic reflectance estimation of floating objects, and a water quality and perturbation branch for fully connected encoding of the water quality parameter vectors and perturbation feature vectors;

[0021] The amount of garbage in each sub-region is calculated based on the garbage target attributes.

[0022] Based on the location and contour parameter vectors and material parameter vectors of the debris target, the drift velocity estimation vector of the debris target is calculated, and a two-dimensional local flow field function around the unmanned vessel is constructed to predict the drift trajectory of the debris target.

[0023] The flow field in each sub-region is spatially averaged based on the two-dimensional local flow field function to obtain the regional hydrodynamic characteristics;

[0024] A comprehensive state vector is constructed based on the regional state vector, regional hydrodynamic characteristics, and environmental state vector. The regional state vector includes: the quantity of different types of waste obtained by statistically analyzing the waste categories in each sub-region based on the category probability vector, and the regional-level material statistics obtained by selecting decision components based on the material parameter vector and averaging them in each sub-region.

[0025] By introducing action vectors and combining historical observation data, cleanup records, comprehensive state vectors, and corresponding waste load indicators, a causal graph model of waste load evolution is constructed, and a parameterized causal prediction model is trained. The expected reduction in the total waste load of the overall water area after cleanup of each sub-area is evaluated to obtain a task priority sequence.

[0026] Based on the relative position of the unmanned vessel and the debris target, the flow velocity at the location of the debris target, the material parameter vector of the debris target, and the comprehensive state vector, a pre-contact state feature vector is constructed and input into the pre-contact behavior prediction network; the mapping relationship of the pre-contact behavior prediction network is as follows: , For having a set of parameters Pre-contact behavior prediction network, For the first Each garbage target at any time The feature vector of the pre-contact state, The output of the contact pre-action prediction network includes the recommended approach direction angle range, the recommended approach velocity range, the collection port opening time window, the capture probability index, and the drift risk index.

[0027] Based on the task priority sequence, capture probability index, and drift risk index, candidate garbage targets are weighted and scored within the same sub-region to obtain the priority of the approaching object;

[0028] Using the material parameter vector of the highest priority approaching object, the flow velocity at its location, the wave disturbance intensity index calculated by the IMU, and the output of the pre-contact behavior prediction network, a set of fuzzy control rules is constructed based on fuzzy mathematics, and the control variables for unmanned surface vessel propulsion and collection are output.

[0029] As a preferred technical solution, feature extraction is performed on the observation data to construct a joint feature tensor, which is then input into a multimodal deep learning network to obtain the garbage target attributes, specifically:

[0030] The observation data are denoted as: ;

[0031] In the formula, Discrete time The set of observations; For hyperspectral imagers at position ,wavelength Spectral response value at; For lidar at any time The first collection Each point cloud point includes its three-dimensional coordinates and echo intensity; This is a vector of water quality parameters; The IMU attitude and acceleration vectors; The surface water flow velocity and direction components measured by the flow velocity sensor;

[0032] The spectral response values ​​at different wavelengths are combined into spectral vectors. Based on the shape characteristics of the spectral vectors, a water surface reflection mask is constructed and reflection suppression is applied to the hyperspectral image.

[0033] For each spatial unit Construct joint feature vectors ;

[0034] In the formula, spatial unit The three-dimensional coordinates of a representative point in a unified coordinate system, i.e., the spatial geometric features, are determined by... The matrix is ​​obtained by aggregating coordinate compensation data using IMU attitude information; the superscript T indicates matrix transpose. spatial unit At wavelength The intrinsic reflectance at a given location is estimated by calibration and statistical analysis based on the spectral response values. The perturbation feature vector;

[0035] Arrange the joint eigenvectors of all spatial units according to their spatial indices to obtain the joint feature tensor. ; and These represent the number of spatial grid divisions in the two planar directions, respectively. The number of feature channels corresponds to the joint feature vector. The dimension;

[0036] The multimodal deep learning network concatenates and nonlinearly maps features extracted from the spatial branch, spectral branch, and water quality and perturbation branch at the feature fusion layer to form a joint representation. It then generates target detection outputs describing the location and contour parameter vectors of the waste target through the detection head, classification head, and regression head, respectively. Category probability output matrix and material parameter output matrix The mapping relationship is expressed as: In the formula, For having a set of parameters Multimodal deep learning networks;

[0037] Timekeeping The detected first Garbage target attributes for: In the formula, For the first The location and contour parameter vector of each garbage target, including center coordinates and scale parameters; For the first The category probability vector of each garbage target; For the first The material parameter vector of each garbage target.

[0038] As a preferred technical solution, the step of calculating the drift velocity estimation vector of the waste target based on its position and contour parameter vectors and material parameter vectors, constructing a two-dimensional local flow field function around the unmanned vessel, and predicting the drift trajectory of the waste target specifically involves:

[0039] Location in a given plane At this location, a two-dimensional local flow field function is constructed by kernel-weighted interpolation based on the drift velocity of surrounding debris targets. In the formula, For a moment In position The local velocity estimation vector at the location; For position Compared to the first The weighting coefficients of each garbage target, Let T be the Euclidean norm, and let T denote the matrix transpose. To control the scale parameters of the spatial influence range, In the first Each garbage target at any time The two-dimensional planar position vector, by the first The location and contour parameter vectors of each garbage target were extracted; For the first Each garbage target at any time Estimated drift velocity vector relative to the water body;

[0040] For the A garbage target, in time step The following uses the first-order Euler method to predict the drift position at the next time step:

[0041] ;

[0042] In the formula, For the first The garbage target in time The predicted location; To obtain the second-order flow from the two-dimensional local flow field function The velocity estimation vector at the current location of each garbage target.

[0043] As a preferred technical solution, the construction of the comprehensive state vector based on the regional state vector, regional hydrodynamic characteristics, and environmental state vector specifically involves:

[0044] In the formula, This is the region state vector, with the superscript T indicating matrix transpose; To characterize the regional hydrodynamic features, the two-dimensional local flow field function at several sampling points within the sub-region grid is analyzed. The arithmetic mean is obtained; This represents the current global environment state vector. To describe sub-regions The overall state vector of the current state;

[0045] For each sub-region The region state vector is represented as: In the formula, sub-region The amount of garbage, The time obtained by performing waste category statistics for each sub-area subregion The first to The quantity of each type of waste, indicated by superscript 1 to... Indicates a category index; , and Sub-regions At any moment The average buoyancy deviation, average wettability, and average water absorption characteristics are known as material statistics.

[0046] As a preferred technical solution, the construction of a causal graph model of waste load evolution and the training of a parameterized causal prediction model specifically involves:

[0047] ;

[0048] In the formula, For the entire water area in time Total waste load, For time Time region Waste load index, k=1,…,j,…J; For parametric causal prediction models, This is the combined state vector. Let T be the action vector, and the superscript T denotes matrix transpose. This indicates the sub-region within the current scheduling period. Perform the cleanup task. This indicates that the cleanup task will not be performed; For the parameter set of the parameterized causal prediction model;

[0049] The process of separately assessing the expected reduction in the overall water area's garbage load after cleaning each sub-area yields a task priority sequence, specifically:

[0050] Fix the action vectors of other sub-regions to 0 to obtain the sub-regions Forecast of future total waste load without cleanup tasks And the projected total future waste load for the cleanup mission. ;

[0051] Define subregions The intervention effect is: ; For intervention utility, it represents the effect on the sub-region in the current state. After scheduling a cleanup task, within the time window The expected reduction in the total garbage load of the entire inland waterway;

[0052] Combining the intervention utility of all sub-regions into a vector The superscript T indicates matrix transpose, and is followed by... Sort the sub-regions by index from largest to smallest to obtain the task priority sequence. .

[0053] As a preferred technical solution, the construction of the pre-contact state feature vector specifically includes:

[0054] ;

[0055] In the formula, For time t, the first The feature vector of a garbage target before contact; Relative position For the first The planar position of a garbage target at time t. These are the planar coordinates of the ship's hull in the water surface coordinate system; For the first The flow velocity near the target garbage item; For the first The material parameter vector of each garbage target; For the first The sub-area where the garbage target is located The comprehensive state vector; the superscript T denotes matrix transpose;

[0056] The pre-contact behavior prediction network adopts a structure combining a feature encoder and a multi-task output head, as follows: ,in, For feature encoder, Represented as low-dimensional latent variables, For multi-tasking output;

[0057] The output of the pre-contact behavior prediction network is represented as follows: In the formula, For capture probability indicators; As an indicator of drift risk; To recommend approximating the velocity scalar; Recommended approach angle; and The start and end times of the time window for the collection port, relative to the current time. definition;

[0058] Construct a recommended approach speed range The lower bound of velocity upper speed limit ; and A coefficient set according to the drift risk level;

[0059] Construct recommended approach direction angle interval The lower bound of the direction angle upper bound of direction angle ; Allowable deviation in direction, This is the angle normalization function.

[0060] As a preferred technical solution, the step of obtaining the priority of approaching objects by weighted scoring of candidate garbage targets within the same sub-region based on task priority sequence, capture probability index, and drift risk index is as follows:

[0061] From the task priority sequence Select the sub-region with the highest priority for performing the cleanup task;

[0062] Weighted scoring ;

[0063] In the formula, For the first Each garbage target at any time The higher the overall score, the higher the priority. and These are the weighting coefficients; For capture probability indicators; This is an indicator of drift risk.

[0064] As a preferred technical solution, the method involves constructing a set of fuzzy control rules based on fuzzy mathematics, using the material parameter vector of the highest priority approaching object, the flow velocity at its location, the wave disturbance intensity index calculated by the IMU, and the output of the pre-contact behavior prediction network. The output of these rules determines the control variables for the unmanned surface vessel's propulsion and collection operations. Specifically:

[0065] Construct a fuzzy controller, with the current control cycle targeting the highest priority approaching object. Fuzzy control input vector for:

[0066] ;

[0067] In the formula, For a specific moment; , and The highest priority targets are respectively The buoyancy deviation index, surface wettability index, and water absorption characteristic index; As the highest priority to approach The modulus of the flow velocity at the location; The wave disturbance intensity index is calculated based on the IMU output; As the highest priority to approach Drift risk indicators; The current convergence speed is relative to the highest priority convergence target. Speed ​​reference error; As the highest priority to approach Lateral position relative to the longitudinal centerline of the hull, i.e., lateral deviation; To represent the direction error of convergence; the superscript T indicates matrix transpose;

[0068] The output variable is:

[0069] ;

[0070] In the formula, To target the highest priority groups The output variables that are controlled; This is the speed correction amount along the approach direction; This is for lateral propulsion control. This is a command to correct the yaw angle. To collect opening and closing angle control commands; To collect power or intensity control commands from the mechanism.

[0071] As a preferred technical solution, the construction of the fuzzy control rule set based on fuzzy mathematics specifically involves:

[0072] Based on the fuzzy control input vector and output variables, the empirical control strategy is explicitly represented as a set of rules in the form of "if-then". The fuzzy control input vector and output variables are linguistically divided using triangular membership functions, including: dividing the drift risk index into "low / medium / high", dividing the lateral deviation into "left large / left small / center / right small / right large", and dividing the speed reference error into "negative large / negative small / zero / positive small / positive large".

[0073] Another aspect of the present invention provides an unmanned vessel waste collection system based on multi-source fusion and causal reasoning, applied to the aforementioned unmanned vessel waste collection method based on multi-source fusion and causal reasoning, including a data acquisition module, a hyperspectral-dominated multi-source physical consistency fusion and waste identification module, a local flow field estimation and drift modeling module, a causal reasoning-based waste collection task priority dynamic programming module, a pre-contact approach behavior prediction module, and a fuzzy adaptive approach and collection control module.

[0074] The data acquisition module is used to divide the water area operation area into several non-overlapping sub-regions; collect and time-align observation data from hyperspectral imagers, lidar, water quality sensors, IMUs, and flow velocity sensors; and obtain environmental state vectors for hydrological and meteorological monitoring.

[0075] The hyperspectral-dominated multi-source physical consistency fusion and waste identification module is used to extract features from observation data, construct a joint feature tensor, and input it into a multimodal deep learning network to obtain waste target attributes, including: the location and contour parameter vector, category probability vector, and material parameter vector of the waste target; the joint feature tensor includes spatial geometric features, intrinsic reflectance estimation, water quality parameter vector, and perturbation feature vector; the multimodal deep learning network includes a spatial branch for convolutional encoding of spatial geometric features, a spectral branch for spectral convolutional encoding of the intrinsic reflectance estimation of floating objects, and a water quality and perturbation branch for fully connected encoding of the water quality parameter vector and perturbation feature vector;

[0076] The dynamic programming module for prioritizing waste recycling tasks based on causal reasoning is used to: count the amount of waste in each sub-region according to the waste target attributes; calculate the drift velocity estimation vector of the waste target based on the position and contour parameter vector and material parameter vector of the waste target; construct a two-dimensional local flow field function around the unmanned vessel to predict the drift trajectory of the waste target; spatially average the flow field in each sub-region based on the two-dimensional local flow field function to obtain the regional hydrodynamic characteristics; construct a comprehensive state vector based on the regional state vector, regional hydrodynamic characteristics, and environmental state vector; the regional state vector includes: the quantity of different types of waste obtained by counting waste categories in each sub-region based on the category probability vector, and the regional-level material statistics obtained by selecting decision components based on the material parameter vector and averaging them in each sub-region; introduce action vectors, combine historical observation data, executed cleanup records, comprehensive state vectors, and corresponding waste load indicators to construct a causal graph model of waste load evolution, and train a parameterized causal prediction model; evaluate the expected reduction in the total waste load of the overall water area after cleaning each sub-region to obtain a task priority sequence;

[0077] The pre-contact approach behavior prediction module is used to construct a pre-contact state feature vector based on the relative position of the unmanned vessel and the debris target, the flow velocity at the location of the debris target, the material parameter vector of the debris target, and the comprehensive state vector, and input it into the pre-contact behavior prediction network; the mapping relationship of the pre-contact behavior prediction network is as follows: , For having a set of parameters Pre-contact behavior prediction network, For the first Each garbage target at any time The feature vector of the pre-contact state, The output of the contact pre-action prediction network includes the recommended approach direction angle range, the recommended approach speed range, the collection port opening time window, the capture probability index, and the drift risk index. Based on the task priority sequence, the capture probability index, and the drift risk index, the candidate garbage targets in the same sub-region are weighted and scored to obtain the priority of the approaching object.

[0078] The fuzzy adaptive approach and collection control module is used to construct a set of fuzzy control rules based on fuzzy mathematics, according to the material parameter vector of the highest priority approach object, the flow velocity at its location, the wave disturbance intensity index calculated by the IMU, and the output of the pre-contact behavior prediction network, and output the control variables for the unmanned vessel to perform propulsion and collection.

[0079] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0080] (1) This invention uses hyperspectral-dominated multi-source physical consistency fusion, utilizes hyperspectral reflectance, water quality parameters and water body radiative transfer model to perform intrinsic inversion of the spectrum, and combines attitude compensation and multi-frame point cloud stability analysis to suppress water surface reflection and wave interference, constructs spatial-spectral-physicochemical joint features, and then outputs target-level position, category and material parameters by multimodal deep network, fundamentally improving the stability of recognition and material discrimination in complex scenes, and providing reliable input for subsequent modules.

[0081] (2) This invention uses the local flow field estimation and drift modeling module to separate the drift velocity relative to the water body by using the time series of the garbage target location, the motion state of the ship and the flow velocity observation. It also constructs a two-dimensional flow field around the unmanned ship by kernel weighted interpolation. At the same time, it predicts the short-term drift trajectory and drift risk based on the flow field and material properties, so that the hydrodynamic effect is transformed from background interference into a structured input that can be directly used for decision-making and control, thus filling the gap in the existing technology for garbage drift modeling.

[0082] (3) This invention uses a dynamic programming module for prioritizing waste recycling tasks based on causal reasoning to construct a causal graph based on regional waste status, material statistics, local flow field characteristics and environmental variables. It models the cleaning tasks of a specific area as intervention variables and calculates the expected difference of the total waste load under different intervention scenarios within a given time window as the intervention utility of the area. This achieves task ranking based on future effects and specifically overcomes the shortcomings of existing technologies in the foresight of task decision-making.

[0083] (4) On the one hand, the present invention uses a pre-contact approach behavior prediction module to predict the capture probability and drift risk under different approach strategies at the target level based on relative pose, local flow field and material parameters, and outputs suggestions on approach direction, velocity range and collection port time window; on the other hand, it uses a fuzzy adaptive approach and collection control module to map information such as material properties, hydrodynamic disturbance, velocity error and lateral deviation into control quantities of propulsion and collection mechanisms, and performs adaptive adjustment within the safe range given by the pre-contact prediction, thereby overcoming the problems of rigidity of approach and collection strategies and insufficient adaptability to dynamic disturbances in the prior art at the execution level. Attached Figure Description

[0084] Figure 1 This is a flowchart of an unmanned vessel waste recycling method based on multi-source fusion and causal reasoning, according to an embodiment of the present invention. Detailed Implementation

[0085] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative effort are within the scope of protection of the present application.

[0086] This invention addresses the task of unmanned surface vessel (USV) waste collection in complex aquatic environments. It aims to achieve stable identification, rational scheduling, and adaptive collection of floating debris under conditions of strong reflection, surges, and complex flow fields. A complete information processing and control closed loop is constructed, comprising perception, flow field modeling, causal decision-making, pre-contact prediction, and fuzzy control. The entire system begins with hyperspectral-driven multi-source physical consistency fusion and deep learning-based recognition, and concludes with fuzzy adaptive approach and collection control. Intermediately, it incorporates local flow field estimation and drift modeling, causal reasoning-based regional task priority planning, and pre-contact approach behavior prediction. Clear data flow and functional division organically connect these modules.

[0087] At the perception and recognition layer, the hyperspectral-dominated multi-source physical consistency fusion and waste identification module receives synchronous observation data from hyperspectral imagers, lidar, water quality sensors, IMUs, and flow velocity sensors. It constructs spatial-spectral-physicochemical joint features using water body radiative transfer models and attitude compensation mechanisms, and completes the detection, category identification, and material parameter estimation of floating waste through a multimodal deep learning network. At the same time, it outputs regional waste distribution information divided by water area, providing a unified and physically interpretable data foundation for all subsequent modules.

[0088] In the hydrodynamic modeling layer, the local flow field estimation and drift modeling module takes the target-level position sequence and material parameters output by the identification module as input, and combines the hull motion state, IMU and flow velocity sensor data to separate the target's drift velocity relative to the water body. It then constructs a two-dimensional local flow field around the unmanned vessel using kernel-weighted interpolation. This module further extracts average flow velocity features at the regional scale to form a regional-level hydrodynamic description. Simultaneously, it provides short-term drift trajectory prediction and drift risk indicators at the target scale, providing dynamic driving information for the expected flow direction of debris to the causal task planning module, and also providing target-level drift trend references for pre-contact approach behavior prediction and fuzzy control.

[0089] At the task decision-making level, the dynamic programming module for prioritizing waste recycling tasks based on causal reasoning models regional waste quantity and type distribution, regional average material characteristics, local flow field characteristics, and environmental variables such as upstream emissions, rainfall, and wind fields, constructing a causal graph of waste generation, transport, and accumulation processes. Based on this, regional cleanup tasks are abstracted into regional intervention variables. By calculating the expected change in total waste load under different intervention scenarios within a given time window, the intervention utility of each region is obtained, and a regional task priority sequence is generated accordingly. This allows the system to prioritize operational areas that contribute more to reducing the overall waste load at the scheduling level.

[0090] In the prediction layer, the pre-contact approach behavior prediction module receives a set of high-priority regions from the causal task planning output, selects targets to approach from these regions, and constructs pre-contact state features by combining the target's relative pose, local flow field estimation results, and material parameters. This module, through a deep prediction network, provides the probability of the target entering the effective area of ​​the collection port and the risk of being pushed away by the flow field under different approach directions and velocity strategies. The output includes recommended parameters such as the approach direction angle, the approach velocity reference range, the collection port opening time window, the capture probability, and the drift risk level. This further refines the regional cleanup task into approach and contact control references specific to each target.

[0091] At the execution control layer, the fuzzy adaptive approach and collection control module comprehensively utilizes material parameters provided by the hyperspectral identification module, hydrodynamic indicators provided by the flow field estimation module, velocity and direction reference ranges provided by the pre-contact prediction module, and drift risk levels to construct fuzzy control inputs. Through a rule base, it achieves joint adjustment of velocity correction along the approach direction, lateral compensation propulsion, yaw angle adjustment, collection port opening angle, and collection intensity. Under the premise of satisfying the safety range constraints given by the pre-contact prediction module, the control output adaptively adjusts the approach and collection behavior for different waste materials and different flow field disturbances, thereby maintaining high recovery stability and robustness in complex hydrodynamic environments.

[0092] Through the above-mentioned top-down module division and bottom-up data transmission, this invention forms a complete closed loop of "perception-flow field modeling-causal task planning-approach prediction-adaptive execution", starting with hyperspectral multi-source fusion, taking causal reasoning as the decision core, and using pre-contact prediction and fuzzy control as execution support. This enables stable identification, rational scheduling and adaptive recycling of surface debris under complex flow field and strong interference conditions.

[0093] Example:

[0094] like Figure 1 As shown, this embodiment provides an unmanned surface vessel (USV) waste recycling method based on multi-source fusion and causal reasoning, including:

[0095] The water area operation area is divided into several non-overlapping sub-regions; observation data from hyperspectral imagers, lidar, water quality sensors, IMUs, and flow velocity sensors are collected and time-aligned; and environmental state vectors for hydrological and meteorological monitoring are obtained.

[0096] Feature extraction is performed on the observation data to construct a joint feature tensor, which is then input into a multimodal deep learning network to obtain the garbage target attributes, including: the location and contour parameter vectors, category probability vectors, and material parameter vectors of the garbage target; the joint feature tensor includes spatial geometric features, intrinsic reflectance estimation, water quality parameter vectors, and perturbation feature vectors; the multimodal deep learning network includes a spatial branch for convolutional encoding of spatial geometric features, a spectral branch for spectral convolutional encoding of the intrinsic reflectance estimation of floating objects, and a water quality and perturbation branch for fully connected encoding of the water quality parameter vectors and perturbation feature vectors;

[0097] The amount of garbage in each sub-region is calculated based on the garbage target attributes.

[0098] Based on the location and contour parameter vectors and material parameter vectors of the debris target, the drift velocity estimation vector of the debris target is calculated, and a two-dimensional local flow field function around the unmanned vessel is constructed to predict the drift trajectory of the debris target.

[0099] The flow field in each sub-region is spatially averaged based on the two-dimensional local flow field function to obtain the regional hydrodynamic characteristics;

[0100] A comprehensive state vector is constructed based on the regional state vector, regional hydrodynamic characteristics, and environmental state vector. The regional state vector includes: the quantity of different types of waste obtained by statistically analyzing the waste categories in each sub-region based on the category probability vector, and the regional-level material statistics obtained by selecting decision components based on the material parameter vector and averaging them in each sub-region.

[0101] By introducing action vectors and combining historical observation data, cleanup records, comprehensive state vectors, and corresponding waste load indicators, a causal graph model of waste load evolution is constructed, and a parameterized causal prediction model is trained. The expected reduction in the total waste load of the overall water area after cleanup of each sub-area is evaluated to obtain a task priority sequence.

[0102] Based on the relative position of the unmanned vessel and the debris target, the flow velocity at the location of the debris target, the material parameter vector of the debris target, and the comprehensive state vector, a pre-contact state feature vector is constructed and input into the pre-contact behavior prediction network; the mapping relationship of the pre-contact behavior prediction network is as follows: , For having a set of parameters Pre-contact behavior prediction network, For the first Each garbage target at any time The feature vector of the pre-contact state, The output of the contact pre-action prediction network includes the recommended approach direction angle range, the recommended approach velocity range, the collection port opening time window, the capture probability index, and the drift risk index.

[0103] Based on the task priority sequence, capture probability index, and drift risk index, candidate garbage targets are weighted and scored within the same sub-region to obtain the priority of the approaching object;

[0104] Using the material parameter vector of the highest priority approaching object, the flow velocity at its location, the wave disturbance intensity index calculated by the IMU, and the output of the pre-contact behavior prediction network, a set of fuzzy control rules is constructed based on fuzzy mathematics, and the control variables for unmanned surface vessel propulsion and collection are output.

[0105] Another aspect of this embodiment also provides an unmanned vessel waste recycling system based on multi-source fusion and causal reasoning, including the following modules:

[0106] I. Hyperspectral-based multi-source physical consistency fusion and waste identification module.

[0107] This module resides in the system's perception and recognition layer. Inputs are multi-source data collected by the unmanned surface vessel's hyperspectral imager, lidar, water quality sensor, IMU, and flow velocity sensor. Outputs include target-level debris information and regional debris distribution information, providing a unified data foundation for local flow field estimation, causal task prioritization planning, pre-contact approach behavior prediction, and fuzzy adaptive control. Addressing the challenges of strong reflections, surges, and turbidity in complex aquatic environments, this module primarily utilizes hyperspectral imaging, physically integrating spectral information with geometric, water quality, and hydrodynamic data. Furthermore, a multimodal deep learning network enables stable detection and material parameter estimation of floating debris.

[0108] In the time dimension, the system adopts a unified time base, aligning the outputs of each sensor into frame-level multi-source observations. Multi-source observations can be represented as:

[0109] ;

[0110] In the formula, Discrete time A single frame of multi-source observation data set; For hyperspectral imagers in pixel coordinates ,wavelength Spectral response value at; For lidar at any time The first collection Point cloud points, among which In three-dimensional coordinates, Echo intensity; Let be a vector of water quality parameters, where Turbidity The correlation coefficient for water body attenuation; This is the IMU attitude and acceleration vector, where the first three terms are attitude angles and the last three terms are acceleration components; The components of surface water velocity and direction measured by the flow velocity sensor; the superscript T indicates matrix transpose (with the same meaning in the following text).

[0111] For any pixel in a hyperspectral image The responses at multiple wavelengths are combined to form a spectral vector:

[0112] ;

[0113] In the formula, For pixels The hyperspectral response column vector; For the first Discrete wavelength sampling positions ; This represents the number of effective bands selected. (Based on analysis...) The shape characteristics can distinguish between the mirror reflection of the water surface and the response of a solid floating object.

[0114] To suppress strong reflections from the water surface, spectral shape parameters are constructed as follows:

[0115] ;

[0116] In the formula, For pixels Spectral shape index; For this pixel in the th Response value at each wavelength; and These represent the maximum and minimum values ​​in the responses of all bands, respectively. The sum of the response of the pixel across all bands; To prevent tiny positive numbers with a denominator of zero. When When the value is low and a narrow band peak appears in the local band response, the pixel is more likely to be caused by water surface specular reflection.

[0117] Based on spectral shape index Construct a water surface reflection mask:

[0118] ;

[0119] In the formula, For pixels A binary mask; The discrimination threshold is set for the spectral shape index; when When a pixel is considered a mirror-like reflection area on the water surface, it is suppressed in subsequent processing; when At this point, pixels are considered as effective regions for floating object analysis. This yields a hyperspectral image with suppressed reflection, providing a more stable spectral input for subsequent feature construction.

[0120] In terms of spatial geometry, IMU attitude information is used to perform coordinate compensation on the point cloud, unifying observations from different attitudes to the same ship coordinate system. The compensation relationship is written as:

[0121] ;

[0122] In the formula, For the first The three-dimensional coordinate vector of a point in a unified ship hull coordinate system; This is the rotation matrix obtained from the IMU attitude calculation; These are the original three-dimensional coordinates of the point in the sensor coordinate system; This is the translation vector determined by the relationship between the sensor's installation position and the ship's coordinate system. It represents the compensated point cloud over several consecutive frames. By performing time analysis, stable point clusters with small positional changes can be selected as candidate areas for floating objects, while scattered points that change rapidly with the surge can be filtered out, thereby determining the spatial range of the hyperspectral-point cloud joint analysis.

[0123] In the optical transmission modeling stage, a total water attenuation coefficient is introduced to describe the absorption and scattering of light by the water body. The total water attenuation coefficient is expressed as:

[0124] ;

[0125] In the formula, wavelength The total attenuation coefficient of the water body; The absorption coefficient; These are scattering coefficients, derived from the water quality parameter vector. (Including turbidity and parameters related to attenuation) are estimated through empirical relationships. Under the radiative transfer approximation, the relationship between the measured water surface reflectance and the intrinsic reflectance of floating objects can be written as:

[0126] ;

[0127] In the formula, For pixels At wavelength The water surface reflectance measured at that location can be obtained from... The calibration was obtained; This corresponds to the intrinsic reflectivity of the surface of the floating object; For pixels The equivalent propagation path length of the corresponding optical path in the water body can be estimated based on the point cloud height and the water surface position; Background water body at wavelength The average reflectance at that location.

[0128] Based on the above relationship, an approximate inversion form of intrinsic reflectivity is given:

[0129] ;

[0130] In the formula, For pixels At wavelength Estimation of the intrinsic reflectance of floating objects at a location; , , , The meaning is the same as the previous formula. Through this inversion, this module converts the measured spectrum affected by the water body into an approximate intrinsic spectral representation that reflects the characteristics of the target material.

[0131] On the spatial grid covering the candidate floating object region, this module combines spatial location, spectral characteristics, water quality parameters, and disturbance state into a unified spatial-spectral-physical-chemical (flow field and water quality, etc.) joint feature vector. For each spatial unit... Construct the joint eigenvector:

[0132] ;

[0133] In the formula, spatial unit The joint eigenvectors; The three-dimensional coordinates of the representative point of this unit in a unified coordinate system can be obtained from the compensated point cloud. Aggregation yields; For this unit at wavelength The intrinsic reflectance at a given location can be estimated from the intrinsic reflectance of pixels within the cell. Statistics show that; This refers to the aforementioned water quality parameter vector; For the attitude vector With velocity vector Through feature mapping function The obtained disturbance feature vector is used to describe the local water flow velocity, direction, and wave intensity.

[0134] Arrange the joint eigenvectors of all spatial units according to their spatial indices to obtain the joint feature tensor:

[0135] ;

[0136] In the formula, For a moment The joint eigenvectors; and These represent the number of spatial grid divisions in the two planar directions, respectively. The number of feature channels corresponds to the vector. The dimension of a tensor. It serves as input to a multimodal deep learning network for subsequent garbage target detection and material estimation.

[0137] Based on the feature tensor, a multimodal deep learning network is used to detect, classify, and estimate the material parameters of floating debris. The overall mapping relationship can be written as:

[0138] ;

[0139] In the formula, For having a set of parameters Multimodal deep learning networks; This is the target detection output, used to describe the spatial location and outline of each waste target; Output matrix for class probabilities; Output matrix for material parameters. In the network structure, spatial branch pairs... Convolutional encoding of relevant geometric features, spectral branch pairs The sequence is spectrally convolutionally encoded, and water quality and disturbance branch pairs are used. and Fully connected encoding is performed; the three features are concatenated and nonlinearly mapped at the feature fusion layer to form a joint representation, which is then generated by the detection head, classification head, and regression head respectively. , , .

[0140] To facilitate subsequent module calls, network output is organized in single-target format. Timing. The detected first The attributes of each garbage target are:

[0141] ;

[0142] In the formula, For the first The attribute triples of a garbage target; This is the location and contour parameter vector of the waste target, including center coordinates and scale parameters; This is the category probability vector of the garbage target; Let be the material parameter vector for this waste target, used to characterize physical properties such as buoyancy deviation, surface wettability, and water absorption and dissipation characteristics. The set of material parameters for all targets is denoted as . .

[0143] To address the requirements of causal task planning for regional-level waste status, this module employs a pre-defined set of water area sub-regions. The amount of garbage in each sub-region is counted based on the target location. The set of sub-regions satisfies:

[0144] ;

[0145] In the formula, This refers to the area where unmanned vessels are operating; For the first in this range A grid or sub-region; This represents the total number of sub-regions. Sub-regions The amount of garbage inside is expressed as:

[0146] ;

[0147] In the formula, For a moment subregion The target quantity of garbage within; As an indicator function, when garbage target Position parameters Fall into sub-region The value is 1 when the spatial range is specified, and 0 otherwise. Based on this, the quantity and proportion of different types of waste in each sub-region can be further statistically analyzed to form regional waste distribution characteristics, providing input for the causal task priority planning module.

[0148] Through the above processing flow, this module obtains data from multi-source observation frames. Starting with hyperspectral water surface interference suppression, attitude compensation point cloud unification, intrinsic spectral estimation based on water body attenuation model, and joint spatial-spectral-physicochemical feature construction, combined with a multimodal deep learning network, the system achieves this. Get the target attribute set and the amount of waste at the regional level The target-level output provides traceable spatial location and material parameters for local flow field estimation, pre-contact approach behavior prediction, and fuzzy control. The region-level output provides observations of the waste status in each region for causal task priority planning, thus providing a unified and physically interpretable input for subsequent links of the entire system at the perception level.

[0149] II. Local Flow Field Estimation and Drift Modeling Module.

[0150] This module, located in the middle layer of the system, takes as input the target-level location information and material parameters output by the hyperspectral-dominated multi-source fusion and waste identification module, as well as the status information from the IMU and flow velocity sensors. Its outputs are the local surface flow field estimation results and the drift prediction results of floating waste on a short timescale. This module provides regional-level hydrodynamic characteristics to the causal task priority dynamic programming module and target-level drift trend and offset risk indicators to the pre-contact approach behavior prediction and fuzzy adaptive control modules, enabling subsequent decision-making and control to explicitly consider changes in waste position under hydrodynamic influence.

[0151] In the time dimension, the system continuously samples the output of the hyperspectral and multi-source fusion identification module. Output target material parameter set Each garbage target Includes position and contour parameter vectors and material parameter vector To perform drift modeling, this module starts from... The center position of the garbage target in the water surface coordinate system is extracted and denoted as:

[0152] ;

[0153] In the formula, For discrete time garbage target Two-dimensional planar position vector; and These are the coordinate values ​​along the two coordinate axes in the water surface coordinate system. The ship's own translational speed and heading are derived from the navigation system and IMU, and are denoted as follows in this module. It is used to separate the drift component caused by water flow from the observed displacement.

[0154] At adjacent times and In the preceding steps, this module first calculates the observed displacement and observed velocity of the debris target. The observed displacement is expressed as:

[0155] ;

[0156] The observation velocity is expressed as:

[0157] ;

[0158] In the formula, For the goal of garbage time step Planar displacement vector within; This is the observed velocity vector relative to the global coordinate system; This represents the time interval between two adjacent observation frames. Since the observed velocity includes both the water flow's influence on the debris target and the coordinate changes caused by the ship's own motion, this module estimates the debris target's drift velocity relative to the water body by subtracting the ship's velocity.

[0159] ;

[0160] In the formula, For the goal of garbage At any moment Estimated drift velocity vector relative to the water body; The vector representing the ship's planar velocity in the same coordinate system is obtained by joint calculation of the navigation system and the IMU.

[0161] By analyzing multiple garbage targets and multiple time points over a period of time. By summarizing the data, a "drift sample field" around the unmanned vessel can be obtained. This sample field is spatially discrete. To provide a continuous local flow field description for causal programming and approach prediction, this module uses weighted interpolation to construct a two-dimensional local flow field function near the unmanned vessel. At a given planar position In this module, kernel-weighted interpolation is performed based on the drift velocity of surrounding debris targets:

[0162] ;

[0163] In the formula, For a moment In position The local velocity estimation vector at the location; For position relative to garbage targets The weighting coefficients are used to reflect the influence of spatial distance on the interpolation results. The weighting coefficients can be in Gaussian form:

[0164] ;

[0165] In the formula, It is the Euclidean norm; Scale parameters for controlling the spatial influence range. When the location Approaching a certain garbage target location When, corresponding weight The larger the value, the more significant the contribution of the drift velocity of the debris target to the flow field estimation.

[0166] Simultaneously, this module will use the local flow velocity vector measured by the flow velocity sensor. This is treated as a direct observation of the actual flow velocity near the ship's location, used to correct and constrain the interpolated flow field. Near the ship's location, this module sets larger weights or uses a fusion strategy to ensure that the interpolation results are consistent with the actual flow velocity in that sub-region. Maintaining consistency improves the reliability of local flow field estimation.

[0167] To provide a concise regional-level hydrodynamic feature for the causal task prioritization planning module, this module divides the aforementioned water areas. Based on this, the flow field within each sub-region is spatially averaged. Let sub-regions be... The average velocity vector within is ,in and This represents the average flow velocity components of the sub-region along the two coordinate axes. The average flow velocity is obtained by sampling at several points within the sub-region's grid. The average velocity vector is obtained by taking the arithmetic mean. It will be used as part of the regional state vector in subsequent modules to describe regional waste transport trends.

[0168] In terms of target-level drift modeling, this module uses the local flow field to predict the short-term drift trajectory of each waste target. For any waste target... At the current moment Its location is known and local flow field This module operates at a small time step. The following uses the first-order Euler method to predict the drift position at the next time step:

[0169] ;

[0170] In the formula, For the goal of garbage In time The predicted location; This represents the velocity estimation vector at the current location of the debris target. By repeating the above update over multiple time steps, a sequence of predicted drift trajectories of the debris target within a short time window can be obtained, which can be used to infer whether it will enter or leave the space where the unmanned vessel can effectively collect it.

[0171] Considering the uncertainties in local flow field estimation and material properties, this module also constructs a drift risk index during the prediction process to quantify the probability that waste targets will be pushed away from the collection channel by the water flow during the approach process. Specifically, for each waste target... The maximum offset distance relative to the expected collection path or collection port area is calculated on the predicted trajectory, and a risk level is assigned based on the offset distance and flow velocity. The drift risk index will serve as one of the inputs to the pre-contact approach behavior prediction module and the fuzzy control module, ensuring sufficient safety margin and compensation space when designing the approach strategy.

[0172] The output of this module includes two types of data: one is a set of regional hydrodynamic characteristics. Together with regional-level waste statistics, this serves as input to the causal task priority dynamic programming module, used to describe each sub-region. The dynamic driving conditions of waste transport and accumulation; another type is target-level drift prediction information. The module, along with its corresponding drift risk level, supports the design of the approach direction, speed, and control margin for waste targets in pre-contact approach behavior prediction and fuzzy control. Through this module, the system explicitly models and structures the previously implicit hydrodynamic effects, enabling subsequent modules to make decisions and controls based on the expected flow direction of waste under water flow, thus serving the overall goal of maintaining the stability and effectiveness of the unmanned vessel waste collection process under complex flow conditions.

[0173] III. Dynamic Programming Module for Prioritizing Waste Recycling Tasks Based on Causal Reasoning

[0174] This module is located in the system's task decision layer. Its inputs include: regional-level waste status and material statistics output by the hyperspectral-dominated multi-source fusion and waste identification module; regional-level hydrodynamic characteristics output by the local flow field estimation and drift modeling module; and environmental state vectors from upstream hydrological and meteorological monitoring. The outputs are the cleanup intervention utility for each water sub-region within a given time window, and a task priority sequence ranked based on this utility. This provides a basis for sub-region selection for the subsequent pre-contact approach behavior prediction module and the fuzzy adaptive approach and collection control module.

[0175] Regarding the sub-region division, the system adopts the aforementioned water area division results. The water area operation scope is denoted as... It is divided into several non-overlapping sub-regions. ,index The hyperspectral and multi-source fusion identification module is in real time. The amount of garbage in each sub-region has been given. It also provides a set of target-level material parameters. To characterize the structure of waste types, this module analyzes each sub-region. Perform detailed statistics along the category dimension. Let the system's preset number of waste categories be [number missing]. , record the The category label for each garbage target is For each sub-region and category index Define the number of categories:

[0176] ;

[0177] In the formula, For a moment subregion Inner The quantity of waste; As an indicator function, when garbage target Position parameters fall into sub-region The value is (1) when it is true, and (0) otherwise. This indicates that the target category label equals The value is 1 if it is true, and 0 otherwise. This allows us to obtain the garbage composition of each sub-region in terms of type.

[0178] In terms of materials, the hyperspectral-based identification module has provided a material parameter vector for each waste target. This module will The component most strongly correlated with subsequent decisions was selected as: buoyancy deviation index. Surface wettability index Water absorption characteristics Then, average the results within each sub-region to obtain region-level material statistics. Sub-region At any moment The average buoyancy deviation is expressed as:

[0179] ;

[0180] Average humidity is expressed as:

[0181] ;

[0182] The average water absorption characteristic index is expressed as follows:

[0183] ;

[0184] In the formula, , , Sub-regions At any moment The average buoyancy deviation, average wettability, and average water absorption characteristics; Used to avoid a denominator of zero when there is no garbage in the sub-region temporarily.

[0185] The local flow field estimation and drift modeling module has provided regional-level hydrodynamic feature vectors under the same sub-region division. ,in , sub-region The average velocity components in two planar directions are used to describe the main direction and intensity of waste transport along the water surface. This module directly... As part of the regional status.

[0186] At the environmental level, this module introduces environmental state vectors. ,in Rainfall intensity, This refers to the upstream inflow or outflow rate. For wind speed, The wind direction angle is used to characterize the external conditions that drive waste generation and migration over a time scale.

[0187] Based on the above information, this module in each sub-region Construct the region state vector:

[0188] ;

[0189] Based on this, a comprehensive state vector for causal modeling is constructed:

[0190] ;

[0191] In the formula, sub-region A combined vector of waste quantity, type distribution, and material statistics; Regional hydrodynamic characteristics; This represents the current global environment state vector. For the description region in subsequent causal inference models The input vector of the current state.

[0192] In terms of decision modeling, this module addresses the problem in existing technologies that only rank regions based on the current amount or density of waste, failing to reflect the impact of cleanup activities on future waste evolution. It incorporates the aforementioned comprehensive state vector... Based on this, regional-level action variables are introduced, and a causal decision-making model for waste load evolution is constructed. For each sub-region... Introducing action variables ,in This indicates the sub-region within the current scheduling period. Assign cleaning tasks, This indicates that cleanup will not be performed on this sub-region. The actions of all sub-regions constitute the action vector. .

[0193] This module focuses on a given time interval. The impact of internal cleaning activities on waste load. For time Time region The waste load index, in this embodiment, is selected as the weighted sum of the amount of waste in the sub-region and the waste risk weight; the entire water area in time Total waste load is defined as:

[0194] ;

[0195] During the offline phase, this module uses a comprehensive state vector based on historical observation data from multiple time periods and cleanup records. Action variables and corresponding Construct a causal graphical model of waste load evolution, and train a parameterized causal prediction model based on it. The causal diagram should include at least the causal effects of upstream water flow and rainfall on waste generation in each sub-region, the causal effects of sub-region flow velocity and material statistics on waste transport and accumulation, and the cleanup actions in each sub-region. Waste load in the current sub-region and related downstream regions The causal relationship is determined. By estimating the parameters of this causal structure, a causal prediction model for online decision-making is obtained:

[0196] ;

[0197] in This is the parameter set for the causal prediction model.

[0198] During the online decision-making phase, this module is constantly... Fixed observation of the overall state vector And with the goal of "minimizing total waste load," the effectiveness of interventions to clean up each sub-area was evaluated. Specifically, in calculating the sub-area... When considering the intervention effect, keep the actions of other sub-regions fixed. Only in the action vector Take different values ​​between 0 and 1, With the corresponding action vector Substitute as input , to obtain in "to "Not cleaning" and "to" Projected total future waste load under two scenarios: "Cleanup"

[0199] ;

[0200] ;

[0201] Based on this, this module will sub-regions The intervention utility is defined as:

[0202] ;

[0203] Characterizes the sub-region in the current state After scheduling a cleanup task, within the time window The expected reduction in the total garbage load of the entire inland waterway. The higher the value, the greater the contribution of prioritizing the cleanup of this sub-area to reducing the overall waste load in the future, given limited cleanup resources.

[0204] This module composes the intervention effectiveness of all sub-regions into a vector. , and according to Sort the sub-region indexes by size to obtain the task priority sequence. ,satisfy The scheduling layer, within the current scheduling cycle, from Select the target region set from the first few sub-regions, and combine the corresponding sub-region index, sub-region center position, and comprehensive state vector. The data is passed to the pre-contact approach behavior prediction module and the fuzzy adaptive approach and collection control module.

[0205] Through the above design, this module no longer uses the amount of garbage in the sub-region at the current moment as the sole sorting criterion. Instead, it quantifies the "impact of cleaning actions on different sub-regions on the overall garbage load within a future time window" based on a causal prediction model. This achieves a shift from correlation-driven to causal utility-driven decision-making at the task decision level. Compared to the sorting strategy, this module can prioritize the cleaning of key sub-regions that have a greater burden reduction effect on downstream sub-regions and the overall water area, avoiding the repeated consumption of cleaning resources on sub-regions that can be quickly replenished or naturally spread in the short term, thereby improving the overall governance efficiency and foresight of the unmanned vessel cluster in complex water environments.

[0206] IV. Pre-contact approach behavior prediction module.

[0207] This module sits between causal task priority dynamic programming and fuzzy adaptive approach and collection control. Its role is to predict the approach behavior under hydrodynamic forces and waste material constraints, given a defined current cleaning sub-region and target set within it. It provides a set of reference parameters for the control layer to execute, including the approach direction range, approach speed range, collection port opening timing, and risk indicators of being pushed away from the collection area by water flow. Through this module, the sub-region cleaning task formed by the decision-making layer is further transformed into quantitative references for approach and contact that can be directly invoked by the control layer.

[0208] The dynamic programming module for prioritizing causal tasks at time... Output area task priority sequence The scheduling layer is based on Select several high-priority regions as the target region set for the current batch, denoted as . In each sub-region Within the module, the hyperspectral-dominated multi-source fusion and waste identification system has provided a complete set of target material parameters. The spatial location and material parameters of each waste target are also provided; the local flow field estimation and drift modeling module has already provided the local flow field. This module estimates short-term drift behavior. Based on this, it constructs the target-level state, predicts the approach behavior, and outputs a recommended approach reference.

[0209] The unmanned ship's planar position in the global coordinate system at time... Recorded as:

[0210] ;

[0211] In the formula, These are the planar coordinates of the ship's hull in the water surface coordinate system; and These are the components along the two coordinate axes. For target-level locations, the notation used in the hyperspectral recognition module is followed to record the garbage target. At any moment The planar position is This module first transforms the location of the debris target to a coordinate system relative to the ship's hull, thus obtaining the relative pose of the debris target:

[0212] ;

[0213] In the formula, For the goal of garbage Position vector relative to the hull and These are the relative coordinate components. The relative pose reflects the garbage target. The distribution pattern in front of or to the side of the hull forms the basis for predicting subsequent approach behavior.

[0214] The local flow field estimation module has provided the results at any location. Velocity estimation at the location This module obtains the flow velocity vector at the current location of the garbage target:

[0215] ;

[0216] In the formula, For at any time garbage target The velocity estimation vector at the location is used to characterize the movement trend of the debris target, which is mainly driven by hydrodynamics in a short period of time. The material parameter vector of the debris target follows the aforementioned notation and is denoted as... ,in This is an index of buoyancy deviation. It is an index of surface wettability. This is an indicator of water absorption characteristics.

[0217] Based on the above quantities, this module constructs a pre-contact state feature vector for each candidate waste target:

[0218] ;

[0219] In the formula, For at any time target of waste The feature vector of the pre-contact state; Relative position; The flow velocity near the target garbage; This is the material parameter vector for the waste target; Sub-region where the garbage target is located The comprehensive state vector includes regional-level waste status, hydrodynamic characteristics, and environmental status. Feature vector This will serve as input to the pre-contact behavior prediction network, focusing on describing "where the garbage target is, in what flow field, what material it has, and what state its sub-region is in."

[0220] To describe the reference values ​​for the approaching motion, this module primarily outputs the approaching direction, approaching speed, and the collection port opening time window. The approaching direction is represented as a relative azimuth angle, denoted as [missing value]. , defined as the angle between the relative position and direction of the debris target and the ship's current heading; the approach speed is expressed in scalar form, denoted as . , defined as the desired velocity amplitude of the hull along the approach direction; the collection port opening time window is represented by the relative values ​​of the start and end times, denoted as . This indicates the time interval during which the collection port should remain open during the approach process. Additionally, this module outputs a capture probability index. With a drift risk indicator The former reflects the probability of a garbage target entering the effective collection area under the predicted approach strategy, while the latter reflects the risk level of a garbage target deviating from the collection area due to the flow field during the approach process.

[0221] The pre-contact behavior prediction network uses the following parameter set: The deep model implementation, whose mapping relationship can be written as:

[0222] ;

[0223] In the formula, For having a set of parameters Pre-contact behavior prediction network; For the goal of garbage The feature vector of the pre-contact state; This is the network output vector.

[0224] Specifically, This can be achieved by combining a feature encoder with a multi-task output head, with the input feature vector... First, it passes through the feature encoder. Mapping to low-dimensional latent variable representation Then, the multi-task output head Generate network output vector :

[0225] ;

[0226] In specific implementation, the feature encoder It can be used as a Transformer encoder to learn the nonlinear coupling relationship between relative pose, local flow field, material parameters, and region state; multi-task output head. It includes at least two branches: a probability branch for capturing probability and drift risk, and a regression branch for approach velocity, orientation angle and time window, thereby achieving joint prediction of classification and regression.

[0227] Apply consistency constraints to variables of different types at the output layer: capture probability Constrained to [0,1] by Sigmoid mapping; drift risk indicator Limited to [0,1] or a preset risk level range; approach direction angle The angle normalization function limits the range to (-π,π] or [0,2π]. The network is jointly trained offline using simulated environment data and data collected from actual ships; the training samples include the input features. And the monitoring signals obtained from the historical approach trajectory and collection results: whether the target was successfully captured or entered the effective collection area, the degree of deviation during the approach process, the recommended approach speed and direction, and the effective time interval for the collection port to be opened.

[0228] The training loss function is a multi-task weighted form:

[0229] ;

[0230] in, Cross-entropy loss is used for learning. ; For regression loss or hierarchical risk classification loss, used for learning ; , , The L1 / L2 / Huber regression loss is used to constrain the prediction errors of velocity, orientation angle, and time window, respectively. These are the weighting coefficients for each loss term. Through the above structure and training method, the network can output approach reference parameters and risk assessment results that the control layer can directly utilize, given the relative state of the garbage target and the flow field conditions.

[0231] Output vector According to the predetermined structure, it is divided into:

[0232] ;

[0233] In the formula, For capture probability indicators; As an indicator of drift risk; To recommend approximating the velocity scalar; Recommended approach angle; and The start and end times of the time window for the collection port, relative to the current time. Definition. During the offline phase, the network is jointly trained with simulated environments and real ship data to make the output capture probability and risk indicators close to the desired behavioral effect.

[0234] In practical applications, this module will not directly convert scalars. and Instead of forcibly issuing point values ​​to the control layer, the fuzzy adaptive control module is given adjustment space by constructing an allowable range for velocity and direction (i.e., a recommended velocity range) within the neighborhood of the predicted results. The allowable velocity range can be constructed based on the predicted values ​​and risk indicators, denoted as... ,in and respectively garbage target The lower and upper bounds of the approach speed are often given by:

[0235] ;

[0236] Given. In the formula, and A coefficient set according to the drift risk level, when At higher levels, and It can be adjusted to a more conservative value, shifting the speed range towards the lower speed side to reduce the uncertainty of relative motion during the approach process.

[0237] Similarly, in predicting the direction angle The given allowable range of direction angles nearby (i.e., the recommended range of direction angles to approach). Wherein, the lower bound and the upper bound of the direction angle are:

[0238] ;

[0239] In the formula, Allowable deviation in direction, This is an angle normalization function used to map direction angles to a preset angle range. It can be set according to the drift risk index and the intensity of local flow field disturbance; when When the flow is high or the flow direction changes strongly, reduce This reduces the range of directional adjustment and lowers the risk of lateral drift.

[0240] In the presence of multiple spam targets, this module scores candidate spam targets within the same sub-region based on the region priority output by the causal task priority module and the capture probability calculated internally by this module. (Target is recorded.) The overall score is:

[0241] ;

[0242] In the formula, For the goal of garbage At any moment Overall score; and This is a weighting coefficient used to balance the trade-off between high capture probability and low drift risk. For a given high-priority sub-region, the several garbage targets with the highest comprehensive scores are selected as the key convergence targets in the current scheduling cycle, and their corresponding... , , as well as It will be passed as a reference parameter to the fuzzy adaptive approach and collection control module.

[0243] In summary, the pre-contact approach behavior prediction module takes the relative pose of the waste target, local flow field, and material parameters as input. Through a depth prediction model, it provides recommended ranges for capture probability, drift risk, approach speed, direction, and collection port movement, and filters waste targets based on regional priority. The module outputs "approach behavior suggestions with safety margins," providing a quantitative reference for the fuzzy adaptive control module. This allows the control layer to both follow the upper-level planning intent and retain the ability to adjust to instantaneous disturbances and execution deviations, thereby improving the stability of the approach and recovery process in complex hydrodynamic environments.

[0244] V. Fuzzy Adaptive Approach and Collection Control Module.

[0245] This module resides in the system's execution control layer. It takes the target-level approach reference values ​​and multi-source environmental conditions output from the upstream module as input, and outputs control commands for the unmanned vessel's propulsion and collection mechanisms. By introducing fuzzy mathematics, it integrates hyperspectral material characteristics, hydrodynamic disturbances, drift risks, and recommended intervals provided by the pre-contact approach behavior prediction module. This allows for the adaptive adjustment of approach speed, lateral compensation, yaw correction, and collection port opening / closing methods in a rule-based manner, thereby maintaining the stability of approach and recovery actions in complex hydrodynamic environments.

[0246] Within a certain control cycle, the causal task priority dynamic programming module and the pre-contact approach behavior prediction module have selected the current highest priority approach object, denoted by index . The hyperspectral-dominated multi-source fusion and waste identification module has provided the material parameter vector of the highest priority approach object. ,in This is an index of buoyancy deviation. It is an index of surface wettability. This is an indicator of water absorption characteristics. The local flow field estimation and drift modeling module has provided the velocity vector at the location of the highest priority approaching object. The pre-contact approach behavior prediction module has output the recommended approach speed scalar for the highest priority approaching object. Approaching direction angle Collection port opening time window capture probability and drift risk indicators It also provides a permissible speed range (i.e., a recommended speed range to approach). .

[0247] The position of the ship's hull in the water surface coordinate system is denoted as:

[0248] ;

[0249] The planar positions of the highest priority objects to be approximated in the same coordinate system are denoted as:

[0250] ;

[0251] The relative position vector is defined as:

[0252] ;

[0253] In the formula, Describes the planar position of the highest priority approaching object relative to the hull. The hull's current heading angle is denoted as... The recommended approach angle given by the pre-contact prediction module is: The approach direction error is defined as:

[0254] ;

[0255] In the formula, This represents the angular deviation between the ship's current course and the recommended approach direction.

[0256] To describe the motion along the approach direction, this module obtains the hull velocity vector from the navigation and propulsion system. And calculate its component in the approach direction:

[0257] ;

[0258] In the formula, The projection velocity along the approach direction; From the angle Defined unit direction vector. Based on the velocity tolerance range (i.e., recommended approach velocity range) given by the pre-contact prediction module. The speed reference value for constructing the highest priority approaching object in this module is:

[0259] ;

[0260] In the formula, The convergence speed reference value for the highest priority convergence target is adjusted according to the drift risk index within the recommended range. Adjustments should be made when the risk of drift is high. To improve control margin, the speed is closer to the low-speed end. Speed ​​error is defined as:

[0261] ;

[0262] In the formula, This represents the error between the current approach speed and the speed reference of the highest priority approach object.

[0263] Lateral deviation describes the lateral position of the highest priority approaching object relative to the longitudinal centerline of the hull, denoted as . It can be derived from the relative position vector. The projection is obtained in the direction perpendicular to the approach direction. This quantity is used to guide the adjustment of the rudder angle and lateral thrust component, so that the highest priority approach target is gradually aligned with the center area of ​​the collection port over time.

[0264] At the input variable level, this module constructs the input vector of the fuzzy controller:

[0265] ;

[0266] In the formula, For the current control cycle, target the highest priority convergence object. fuzzy control input vector; For target material properties;

[0267] The velocity modulus at the location of the highest priority target to approach; The wave disturbance intensity index is calculated based on the IMU output; As an indicator of drift risk; For speed error; This is the lateral deviation; To approximate the directional error.

[0268] At the output variable level, this module needs to provide a set of control vectors for the propulsion and collection mechanisms:

[0269] ;

[0270] In the formula, To target the highest priority groups The control output vector; This is the speed correction amount along the approach direction; This is a lateral propulsion control variable used to reduce lateral deviation; This is a command to correct the yaw angle. To collect opening and closing angle control commands; It is used to collect power or strength control commands from the mechanism to adjust the drum speed.

[0271] The fuzzy control rules are constructed based on the aforementioned input and output variables, explicitly representing empirical control knowledge as a set of rules. Building upon this, this module constructs fuzzy control rules based on the input and output variables, explicitly representing the empirical control strategy as a set of rules in an "if-then" format. To improve feasibility, this embodiment uses triangular membership functions to linguistically classify the input variables. For example, drift risk is classified as "low / medium / high," lateral deviation as "large left / small left / center / small right / large right," and velocity error as "large negative / small negative / zero / small positive / large positive," etc. Output variables are similarly classified using linguistic values ​​such as "small / medium / large" or "left / right," facilitating direct mapping to propulsion power, rudder angle, and the action range of the collection mechanism's actuators.

[0272] Based on the membership functions mentioned above, the constructed fuzzy control rules may include, but are not limited to, the following exemplary rules:

[0273] Rule 1 (Conservative approach to high-current and high-risk scenarios):

[0274] like:

[0275] (1) The flow velocity is "large".

[0276] (2) Drift risk is "high".

[0277] (3) The speed error is either "small positive" or "large positive".

[0278] (4) The lateral deviation is "centered".

[0279] but:

[0280] (a) The speed correction along the approach direction is "negative" (moderate deceleration).

[0281] (b) Lateral thrust control value is "small".

[0282] (c) The yaw angle correction is "small".

[0283] (d) The opening angle of the collection port is "large".

[0284] (e) Collection intensity is "medium".

[0285] This rule ensures a high capture probability even under strong hydrodynamic disturbances.

[0286] Rule 2 (Strong correction when lateral drift and heading deviation are large):

[0287] like:

[0288] (1) The flow velocity is "medium".

[0289] (2) Drift risk is "medium".

[0290] (3) The lateral deviation is "larger on the left".

[0291] (4) The heading error is "large left deviation".

[0292] but:

[0293] (a) The velocity correction along the approach direction is "negatively small".

[0294] (b) The lateral thrust control value is "larger to the right".

[0295] (c) The yaw angle correction is "larger to the right".

[0296] (d) The opening angle of the collection port is "middle".

[0297] (e) Collection intensity is "medium".

[0298] This rule is used to quickly eliminate lateral offset of garbage targets in the later stages of convergence, thereby improving the speed of aligning with the collection window.

[0299] Rule 3 (Rapid approach in low-flow-rate and lightweight floating object scenarios):

[0300] like:

[0301] (1) The flow velocity is "small".

[0302] (2) Drift risk is "low".

[0303] (3) The buoyancy deviation of the garbage target is "positive" (good buoyancy, easy to float).

[0304] (4) The speed error is "negative and large".

[0305] (5) Lateral deviation is "centered".

[0306] but:

[0307] (a) The speed correction along the approach direction is "positive" (rapid approach).

[0308] (b) Lateral thrust control value is "small".

[0309] (c) The yaw angle correction is "small".

[0310] (d) The opening angle of the collection port is "middle".

[0311] (e) The collection intensity is "small".

[0312] This rule is beneficial for improving work efficiency in complex environments.

[0313] The above rules can be used for fuzzy inference through a min-max synthesis method. The resulting fuzzy output is then defuzzified using the center-of-gravity method and converted into a sharp control variable. To adapt to the mechanical structures of different types of unmanned surface vessels (e.g., propulsion configuration, maximum rudder angle, and collection mechanism type), more rules can be added or adjusted to match specific dynamic characteristics and mechanism action ranges, while keeping the above input-output framework unchanged, thereby further enhancing the flexibility and adaptability of the control strategy.

[0314] Through the above design, the fuzzy adaptive approach and collection control module unifies hyperspectral material information, local hydrodynamic disturbances, drift risks, and approach suggestions from upstream predictions into a fuzzy inference framework, outputting propulsion and collection control quantities with adaptive characteristics. On the one hand, this module explicitly receives velocity, direction, and time window suggestions from the pre-contact behavior prediction module at the input end, ensuring that the execution layer behavior is consistent with the upper layer planning. On the other hand, by comprehensively incorporating factors such as material parameters, water flow velocity, wave intensity, velocity error, and lateral deviation into fuzzy rules, the empirical control strategy is formalized into a rule base, achieving flexible adaptation to different types of waste and different hydrodynamic environments. This improves the overall stability and robustness of the unmanned surface vessel's approach and collection actions in complex waters, completing the closed loop of "perception-understanding-decision-prediction-execution" together with the aforementioned module.

[0315] It should be noted that the system provided in this embodiment is only an example of the above-described division of functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure can be divided into different functional modules to complete all or part of the functions described above. This system can be applied to an unmanned vessel waste recycling method based on multi-source fusion and causal reasoning in the above embodiment.

[0316] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0317] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A method for unmanned surface vessel waste recycling based on multi-source fusion and causal reasoning, characterized in that, Includes the following steps: The water area operation area is divided into several non-overlapping sub-regions; observation data from hyperspectral imagers, lidar, water quality sensors, IMUs, and flow velocity sensors are collected and time-aligned; and environmental state vectors for hydrological and meteorological monitoring are obtained. Feature extraction is performed on the observation data to construct a joint feature tensor, which is then input into a multimodal deep learning network to obtain the garbage target attributes. The garbage target attributes include: the garbage target's position and contour parameter vector, category probability vector, and material parameter vector. The joint feature tensor includes spatial geometric features, intrinsic reflectance estimation, water quality parameter vector, and perturbation feature vector. The multimodal deep learning network includes a spatial branch for convolutional encoding of spatial geometric features, a spectral branch for spectral convolutional encoding of the intrinsic reflectance estimation of floating objects, and a water quality and perturbation branch for fully connected encoding of the water quality parameter vector and perturbation feature vector. The amount of garbage in each sub-region is calculated based on the garbage target attributes. Based on the location and contour parameter vectors and material parameter vectors of the debris target, the drift velocity estimation vector of the debris target is calculated, and a two-dimensional local flow field function around the unmanned vessel is constructed to predict the drift trajectory of the debris target. The flow field in each sub-region is spatially averaged based on the two-dimensional local flow field function to obtain the regional hydrodynamic characteristics; A comprehensive state vector is constructed based on the regional state vector, regional hydrodynamic characteristics, and environmental state vector. The regional state vector includes: the quantity of different types of waste obtained by statistically analyzing the waste categories in each sub-region based on the category probability vector, and the regional-level material statistics obtained by selecting decision components based on the material parameter vector and averaging them in each sub-region. By introducing action vectors and combining historical observation data, cleanup records, comprehensive state vectors, and corresponding waste load indicators, a causal graph model of waste load evolution is constructed, and a parameterized causal prediction model is trained. The expected reduction in the total waste load of the overall water area after cleanup of each sub-area is evaluated to obtain a task priority sequence. Based on the relative position of the unmanned vessel and the debris target, the flow velocity at the location of the debris target, the material parameter vector of the debris target, and the comprehensive state vector, a pre-contact state feature vector is constructed and input into the pre-contact behavior prediction network; the mapping relationship of the pre-contact behavior prediction network is as follows: , For having a set of parameters Pre-contact behavior prediction network, For the first Each garbage target at any time The feature vector of the pre-contact state. The output of the contact pre-action prediction network includes the recommended approach direction angle range, the recommended approach velocity range, the collection port opening time window, the capture probability index, and the drift risk index. Based on the task priority sequence, capture probability index, and drift risk index, candidate garbage targets are weighted and scored within the same sub-region to obtain the priority of the approaching object; Using the material parameter vector of the highest priority approaching object, the flow velocity at its location, the wave disturbance intensity index calculated by the IMU, and the output of the pre-contact behavior prediction network, a set of fuzzy control rules is constructed based on fuzzy mathematics, and the control variables for unmanned surface vessel propulsion and collection are output.

2. The unmanned vessel waste recycling method based on multi-source fusion and causal reasoning according to claim 1, characterized in that, Feature extraction is performed on the observed data to construct a joint feature tensor, which is then input into a multimodal deep learning network to obtain the garbage target attributes, specifically: The observation data are denoted as: ; In the formula, Discrete time The set of observations; For hyperspectral imagers at position ,wavelength Spectral response value at; For lidar at any time The first collection Each point cloud point includes its three-dimensional coordinates and echo intensity; This is a vector of water quality parameters; The IMU attitude and acceleration vectors; The surface water flow velocity and direction components measured by the flow velocity sensor; The spectral response values ​​at different wavelengths are combined into spectral vectors. Based on the shape characteristics of the spectral vectors, a water surface reflection mask is constructed and reflection suppression is applied to the hyperspectral image. For each spatial unit Construct joint feature vectors ; In the formula, spatial unit The three-dimensional coordinates of a representative point in a unified coordinate system, i.e., the spatial geometric features, are determined by... The coordinate compensation was performed using IMU attitude information, and then aggregated to obtain the result. The superscript T indicates matrix transpose; spatial unit At wavelength The intrinsic reflectance at a given location is estimated by calibration and statistical analysis based on the spectral response values. The perturbation feature vector; Arrange the joint eigenvectors of all spatial units according to their spatial indices to obtain the joint feature tensor. ; and These represent the number of spatial grid divisions in the two planar directions, respectively. The number of feature channels corresponds to the joint feature vector. The dimension; The multimodal deep learning network concatenates and nonlinearly maps features extracted from the spatial branch, spectral branch, and water quality and perturbation branch at the feature fusion layer to form a joint representation. It then generates target detection outputs describing the location and contour parameter vectors of the waste target through the detection head, classification head, and regression head, respectively. Category probability output matrix and material parameter output matrix The mapping relationship is expressed as: In the formula, For having a set of parameters Multimodal deep learning networks; Timekeeping The detected first Garbage target attributes for: In the formula, For the first The location and contour parameter vector of each garbage target, including center coordinates and scale parameters; For the first The category probability vector of each garbage target; For the first The material parameter vector of each garbage target.

3. The unmanned vessel waste recycling method based on multi-source fusion and causal reasoning according to claim 1, characterized in that, The process involves calculating the drift velocity estimate vector of the debris target based on its position, contour parameter vector, and material parameter vector, constructing a two-dimensional local flow field function around the unmanned vessel, and predicting the drift trajectory of the debris target. Specifically: Location in a given plane At this location, a two-dimensional local flow field function is constructed by kernel-weighted interpolation based on the drift velocity of surrounding debris targets. In the formula, For a moment In position The local velocity estimation vector at the location; For position Compared to the first The weight coefficients of each garbage target, where exp represents the exponential function. Let T be the Euclidean norm, and let T denote the matrix transpose. To control the scale parameters of the spatial influence range In the first Each garbage target at any time The two-dimensional planar position vector, by the first The location and contour parameter vectors of each garbage target were extracted; For the first Each garbage target at any time Estimated drift velocity vector relative to the water body; For the first A garbage target, in time step The following uses the first-order Euler method to predict the drift position at the next time step: ； In the formula, For the first The garbage target in time The predicted location; To obtain the second-order flow from the two-dimensional local flow field function The velocity estimation vector at the current location of each garbage target.

4. The unmanned vessel waste recycling method based on multi-source fusion and causal reasoning according to claim 1, characterized in that, The construction of the comprehensive state vector based on the regional state vector, regional hydrodynamic characteristics, and environmental state vector is specifically as follows: In the formula, This is the region state vector, with the superscript T indicating matrix transpose; To characterize the regional hydrodynamic features, the two-dimensional local flow field function at several sampling points within the sub-region grid is analyzed. The arithmetic mean is obtained; This represents the current global environment state vector. To describe sub-regions The overall state vector of the current state; For each sub-region The region state vector is represented as: In the formula, sub-region The amount of garbage, The time obtained by performing waste category statistics for each sub-area subregion Inner First to The quantity of each type of waste, indicated by superscript 1 to... Indicates a category index; , and Sub-regions At any moment The average buoyancy deviation, average wettability, and average water absorption characteristics are known as material statistics.

5. The unmanned vessel waste recycling method based on multi-source fusion and causal reasoning according to claim 1, characterized in that, The construction of a causal graph model of waste load evolution and the training of a parameterized causal prediction model are specifically as follows: ； In the formula, For the entire water area in time Total waste load, For time Time region Waste load index, k=1,…,j,…J; For parametric causal prediction models, This is the combined state vector. Let T be the action vector, and the superscript T denotes matrix transpose. This indicates the sub-region within the current scheduling period. Perform the cleanup task. This indicates that the cleanup task will not be performed; For the parameter set of the parameterized causal prediction model; The process of separately assessing the expected reduction in the overall water area's garbage load after cleaning each sub-area yields a task priority sequence, specifically: Fix the action vectors of other sub-regions to 0 to obtain the sub-regions Forecast of future total waste load without cleanup tasks And the projected total future waste load for the cleanup mission. ; Define subregions The intervention effect is: ; For intervention utility, it represents the effect on the sub-region in the current state. After scheduling a cleanup task, within the time window The expected reduction in the total garbage load of the entire inland waterway; Combining the intervention utility of all sub-regions into a vector The superscript T indicates matrix transpose, and is followed by... Sort the sub-regions by index from largest to smallest to obtain the task priority sequence. .

6. The unmanned vessel waste recycling method based on multi-source fusion and causal reasoning according to claim 1, characterized in that, The construction of the pre-contact state feature vector is specifically as follows: ； In the formula, For time t, the first The feature vector of a garbage target before contact; Relative position For the first The planar position of a garbage target at time t. These are the planar coordinates of the ship's hull in the water surface coordinate system; For the first The flow velocity near the target garbage item; For the first The material parameter vector of each garbage target; For the first The sub-area where the garbage target is located The comprehensive state vector; the superscript T denotes matrix transpose; The pre-contact behavior prediction network adopts a structure combining a feature encoder and a multi-task output head, as follows: ,in, For feature encoder, Represented as low-dimensional latent variables, For multi-tasking output; The output of the pre-contact behavior prediction network is represented as follows: In the formula, For capture probability indicators; As an indicator of drift risk; To recommend approximating the velocity scalar; Recommended approach angle; and The start and end times of the time window for the collection port, relative to the current time. definition; Construct a recommended approach speed range The lower bound of velocity upper speed limit ; and A coefficient set according to the drift risk level; Construct recommended approach direction angle interval The lower bound of the direction angle upper bound of direction angle ; For directional allowable deviation, This is the angle normalization function.

7. The unmanned vessel waste recycling method based on multi-source fusion and causal reasoning according to claim 1, characterized in that, The process involves weighted scoring of candidate garbage targets within the same sub-region based on task priority sequence, capture probability index, and drift risk index to determine the priority of approaching objects. Specifically: From the task priority sequence Select the sub-region with the highest priority for performing the cleanup task; Weighted scoring ; In the formula, For the first Each garbage target at any time The higher the overall score, the higher the priority. and These are the weighting coefficients; For capture probability indicators; This is an indicator of drift risk.

8. The unmanned vessel waste recycling method based on multi-source fusion and causal reasoning according to claim 1, characterized in that, The material parameter vector of the highest priority approaching object, the flow velocity at its location, the wave disturbance intensity index calculated by the IMU, and the output of the pre-contact behavior prediction network are used to construct a set of fuzzy control rules based on fuzzy mathematics. The output of these rules determines the control variables for the unmanned surface vessel's propulsion and collection operations. Specifically: Construct a fuzzy controller, with the current control cycle targeting the highest priority approaching object. Fuzzy control input vector for: ； In the formula, For a specific moment; , and The highest priority targets are respectively The buoyancy deviation index, surface wettability index, and water absorption characteristic index; As the highest priority to approach The modulus of the flow velocity at the location; The wave disturbance intensity index is calculated based on the IMU output; As the highest priority to approach Drift risk indicators; The current convergence speed is relative to the highest priority convergence target. Speed ​​reference error; As the highest priority to approach Lateral position relative to the longitudinal centerline of the hull, i.e., lateral deviation; To represent the direction error of convergence; the superscript T indicates matrix transpose; The output variable is: ； In the formula, To target the highest priority groups The output variables that are controlled; This is the speed correction amount along the approach direction; This is the lateral propulsion control quantity; This is a command to correct the yaw angle. To collect opening and closing angle control commands; To collect power or intensity control commands from the mechanism.

9. The unmanned vessel waste recycling method based on multi-source fusion and causal reasoning according to claim 8, characterized in that, The specific details of constructing the fuzzy control rule set based on fuzzy mathematics are as follows: Based on the fuzzy control input vector and output variables, the empirical control strategy is explicitly represented as a set of rules in the form of "if-then". The fuzzy control input vector and output variables are linguistically divided using triangular membership functions, including: dividing the drift risk index into "low / medium / high", dividing the lateral deviation into "left large / left small / center / right small / right large", and dividing the speed reference error into "negative large / negative small / zero / positive small / positive large".

10. An unmanned vessel waste recycling system based on multi-source fusion and causal reasoning, characterized in that, The unmanned vessel waste collection method based on multi-source fusion and causal reasoning, applied to any one of claims 1-9, includes a data acquisition module, a hyperspectral-dominated multi-source physical consistency fusion and waste identification module, a local flow field estimation and drift modeling module, a causal reasoning-based waste collection task priority dynamic programming module, a pre-contact approach behavior prediction module, and a fuzzy adaptive approach and collection control module. The data acquisition module is used to divide the water area into several non-overlapping sub-regions; collect and time-align observation data from hyperspectral imagers, lidar, water quality sensors, IMUs, and flow velocity sensors; and obtain environmental state vectors for hydrological and meteorological monitoring. The hyperspectral-dominated multi-source physical consistency fusion and waste identification module is used to extract features from observation data, construct a joint feature tensor, and input it into a multimodal deep learning network to obtain waste target attributes. The waste target attributes include: location and contour parameter vectors, category probability vectors, and material parameter vectors of the waste target. The joint feature tensor includes spatial geometric features, intrinsic reflectance estimation, water quality parameter vectors, and perturbation feature vectors. The multimodal deep learning network includes a spatial branch for convolutional encoding of spatial geometric features, a spectral branch for spectral convolutional encoding of the intrinsic reflectance estimation of floating objects, and a water quality and perturbation branch for fully connected encoding of the water quality parameter vectors and perturbation feature vectors. The dynamic programming module for prioritizing waste recycling tasks based on causal reasoning is used to: count the amount of waste in each sub-region according to the waste target attributes; calculate the drift velocity estimation vector of the waste target based on the position and contour parameter vector and material parameter vector of the waste target; construct a two-dimensional local flow field function around the unmanned vessel to predict the drift trajectory of the waste target; spatially average the flow field in each sub-region based on the two-dimensional local flow field function to obtain the regional hydrodynamic characteristics; construct a comprehensive state vector based on the regional state vector, regional hydrodynamic characteristics, and environmental state vector; the regional state vector includes: the quantity of different types of waste obtained by counting waste categories in each sub-region based on the category probability vector, and the regional-level material statistics obtained by selecting decision components based on the material parameter vector and averaging them in each sub-region; introduce action vectors, combine historical observation data, executed cleanup records, comprehensive state vectors, and corresponding waste load indicators to construct a causal graph model of waste load evolution, and train a parameterized causal prediction model; evaluate the expected reduction in the total waste load of the overall water area after cleaning each sub-region to obtain a task priority sequence; The pre-contact approach behavior prediction module is used to construct a pre-contact state feature vector based on the relative position of the unmanned vessel and the debris target, the flow velocity at the location of the debris target, the material parameter vector of the debris target, and the comprehensive state vector, and input it into the pre-contact behavior prediction network; the mapping relationship of the pre-contact behavior prediction network is as follows: , For having a set of parameters Pre-contact behavior prediction network, For the first Each garbage target at any time The feature vector of the pre-contact state. The output of the contact pre-action prediction network includes the recommended approach direction angle range, the recommended approach speed range, the collection port opening time window, the capture probability index, and the drift risk index. Based on the task priority sequence, the capture probability index, and the drift risk index, the candidate garbage targets in the same sub-region are weighted and scored to obtain the priority of the approaching object. The fuzzy adaptive approach and collection control module is used to construct a set of fuzzy control rules based on fuzzy mathematics, according to the material parameter vector of the highest priority approach object, the flow velocity at its location, the wave disturbance intensity index calculated by the IMU, and the output of the pre-contact behavior prediction network, and output the control variables for the unmanned vessel to perform propulsion and collection.

Images