Multi-mode hierarchical fusion water surface unmanned equipment real-time obstacle avoidance path generation system and method

Abstract

The application provides a multi-mode layered fusion water surface unmanned equipment real-time obstacle avoidance path generation system and method. It includes: a collection registration module for registering visual data and laser point cloud data according to the calibration relationship and the collection time sequence to obtain registration data; a layered fusion module for fusing target features of images and point clouds according to a depth mapping relationship to obtain fused target features and generate candidate obstacle objects; an interference filtering module for determining the water surface interference state and screening effective obstacle objects; a risk prediction module for determining collision risk and generating a predicted motion trajectory; a path generation module for determining a local safe feasible region under global flight lines and motion constraints and generating a continuous target obstacle avoidance path; and an execution correction module for converting the target obstacle avoidance path into a navigation control instruction. The application can improve fusion perception accuracy, reduce water surface interference misjudgment rate, and improve the real-time and continuity of obstacle avoidance path generation.

Description

Technical Field

[0001] This application relates to the field of intelligent perception and autonomous obstacle avoidance technology for unmanned surface equipment, and in particular to a multi-mode hierarchical fusion system and method for generating real-time obstacle avoidance paths for unmanned surface equipment. Background Technology

[0002] Unmanned surface vessels are widely used in waterway inspection, environmental monitoring, and emergency operations. Their autonomous navigation requires continuous perception of the type, location, and movement of surrounding obstacles, and timely generation of obstacle avoidance paths based on the perception results. The water surface environment is characterized by strong reflectivity, abundant water mist, frequent changes in ripples and waves, and random obstacle movement, which places high demands on perception accuracy and real-time path planning.

[0003] Current methods for detecting obstacles on the water surface typically employ lidar or visual cameras. LiDAR can acquire the distance and spatial contour of obstacles, but its semantic recognition capabilities are limited. It is prone to missing low-reflectivity floating objects and may misidentify discrete points formed by water mist or waves as obstacles. Visual cameras can identify target categories and contours, but they are easily affected by reflections, backlighting, low illumination, and water mist, and it is difficult to consistently obtain accurate spatial depth.

[0004] Some existing solutions fuse visual data with laser point clouds, but these typically employ raw data overlay or single-layer feature stitching. Deviations in sensor acquisition timing, field of view, and coordinate reference can easily lead to target mismatch, and visual semantic information and point cloud spatial information are difficult to fully correlate. Furthermore, existing solutions lack effective filtering mechanisms to adapt to changes in water surface disturbances, and the data connection between perception results and path planning is insufficient. This makes it difficult to predict the encounter status of dynamic obstacles in a timely manner, resulting in delayed local path updates, abrupt trajectory changes, or deviations from the unmanned equipment's movement capabilities, thus affecting the reliability of autonomous obstacle avoidance in complex and dynamic waters. Summary of the Invention

[0005] In view of this, embodiments of this application provide a multi-modal hierarchical fusion system and method for generating real-time obstacle avoidance paths for unmanned surface equipment, in order to solve the problems of inaccurate multi-modal perception fusion, weak water surface interference filtering capability, and delayed and discontinuous dynamic obstacle avoidance path generation in the prior art.

[0006] The first aspect of this application provides a multi-modal hierarchical fusion real-time obstacle avoidance path generation system for unmanned surface equipment, comprising: a registration module for simultaneously acquiring visual data, laser point cloud data, and navigation status, registering visual data and laser point cloud data according to calibration relationships and acquisition time sequences to obtain registration data; a hierarchical fusion module for establishing a depth mapping relationship between the image and the point cloud based on the registration data, fusing target features of the image and the point cloud according to the depth mapping relationship to obtain fused target features, and generating candidate obstacle objects with semantic attributes and spatial state associations based on the fused target features; and an interference filtering module for filtering candidate obstacle objects based on their spatial state. The system uses a cross-modal consistent state and a time-series stable state to determine the surface disturbance state and screen for effective obstacle objects. A risk prediction module determines collision risk based on the relative motion relationship between the effective obstacle object and the navigation state, and generates a predicted trajectory based on the time-series state of the effective obstacle object. A path generation module determines a local safe and feasible region based on the collision risk and predicted trajectory, under global route and motion constraints, and generates a continuous target obstacle avoidance path within the local safe and feasible region. An execution correction module converts the target obstacle avoidance path into navigation control commands and corrects the navigation control commands based on the deviation between the actual trajectory and the target obstacle avoidance path.

[0007] The second aspect of this application provides a method for generating real-time obstacle avoidance paths for multi-modal hierarchical fusion unmanned surface equipment, comprising: simultaneously acquiring visual data, laser point cloud data, and navigation status; registering visual data and laser point cloud data according to calibration relationships and acquisition time sequences to obtain registration data; establishing a depth mapping relationship between the image and the point cloud based on the registration data; fusing target features of the image and the point cloud according to the depth mapping relationship to obtain fused target features; generating candidate obstacle objects with semantic attributes and spatial state associations based on the fused target features; determining the surface interference state based on the cross-modal consistent state and time-series stable state of the candidate obstacle objects, and filtering effective obstacle objects; determining the collision risk based on the relative motion relationship between the effective obstacle objects and the navigation status; generating a predicted motion trajectory based on the time-series state of the effective obstacle objects; determining a local safe and feasible region under global flight path and motion constraints based on the collision risk and the predicted motion trajectory, and generating a continuous target obstacle avoidance path within the local safe and feasible region; converting the target obstacle avoidance path into navigation control commands, acquiring the actual trajectory, and correcting the navigation control commands based on the deviation between the actual trajectory and the target obstacle avoidance path.

[0008] The above-described technical solutions adopted in the embodiments of this application can achieve the following beneficial effects: The system employs a registration module to simultaneously acquire visual data, laser point cloud data, and navigation status. It registers the visual data and laser point cloud data according to calibration relationships and acquisition time sequences to obtain registration data. A layered fusion module establishes a depth mapping relationship between the image and point cloud based on the registration data. It then fuses the target features of the image and point cloud according to the depth mapping relationship to obtain fused target features. Based on these fused target features, it generates candidate obstacle objects with semantic attributes and spatial state associations. An interference filtering module determines the surface interference state based on the cross-modal consistent state and time-series stable state of the candidate obstacle objects, filtering out effective obstacle objects. A risk prediction module determines the collision risk based on the relative motion relationship between the effective obstacle objects and the navigation status, and generates predicted motion trajectories based on the time-series states of the effective obstacle objects. A path generation module determines the local safe and feasible region under global flight path and motion constraints based on the collision risk and predicted motion trajectory, and generates continuous target obstacle avoidance paths within the local safe and feasible region. An execution correction module converts the target obstacle avoidance path into navigation control commands and corrects the navigation control commands based on the deviation between the actual trajectory and the target obstacle avoidance path. This application can improve the accuracy of fusion perception, reduce the misjudgment rate of water surface interference, and enhance the real-time and continuous nature of obstacle avoidance path generation. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 This is a schematic diagram of the structure of the multi-modal hierarchical fusion real-time obstacle avoidance path generation system for unmanned surface equipment provided in this application embodiment; Figure 2 This is a flowchart illustrating the method for generating real-time obstacle avoidance paths for multi-mode hierarchical fusion unmanned surface equipment provided in this application embodiment. Detailed Implementation

[0011] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0012] Existing unmanned surface vessels typically employ a single lidar or visual camera for obstacle perception. Lidar can acquire the distance and spatial contour of obstacles, but struggles to accurately identify obstacle semantics and has limited ability to perceive low-reflectivity floating objects. Visual cameras can extract target categories and contour features, but are easily affected by water surface reflections, water mist, waves, and low-light environments, and their spatial depth measurement accuracy is insufficient. While some existing solutions integrate visual data and lidar point cloud data, they often employ raw data overlay or single-layer feature stitching methods.

[0013] Due to differences in acquisition timing, coordinate references, and fields of view among different sensors, existing fusion schemes are prone to mismatches between image targets and point cloud targets, making it difficult to accurately correlate visual semantic information with point cloud spatial information. Furthermore, existing schemes lack effective discrimination mechanisms to adapt to changes in water surface interference, making it difficult to distinguish between real obstacles and false targets formed by water reflections, spray, and mist. The connection between perception results and path planning is also insufficient; the movement trends of dynamic obstacles cannot be promptly applied to path generation, easily leading to delayed path updates, abrupt trajectory changes, or generated paths that do not conform to the movement capabilities of unmanned equipment.

[0014] To address the aforementioned technical issues, this application provides a multi-modal hierarchical fusion real-time obstacle avoidance path generation system for unmanned surface equipment. This system simultaneously acquires visual data, laser point cloud data, and navigation status. Based on calibration relationships and acquisition timing, it registers the visual data and laser point cloud data; establishes a depth mapping relationship between the image and the point cloud; sequentially completes data correspondence, target feature fusion, and semantic attribute-spatial state association to generate candidate obstacle objects; and then identifies surface interference based on the cross-modal consistency and temporal stability of the candidate obstacle objects, filtering out valid obstacle objects.

[0015] After obtaining valid obstacle objects, this application determines the collision risk based on the relative motion relationship between the valid obstacle objects and the navigation state, and generates a predicted motion trajectory based on continuous observation. Combining the collision risk, predicted motion trajectory, global route, and motion constraints, a local safe and feasible domain is determined, from which a continuous target obstacle avoidance path is generated. The target obstacle avoidance path is converted into navigation control commands, and feedback correction is performed based on the deviation between the actual trajectory and the target obstacle avoidance path. Through the above technical solution, the accuracy of the fusion perception of vision and point clouds can be improved, misjudgments caused by interference from the water surface environment can be reduced, and the real-time performance, continuity, and executability of dynamic obstacle avoidance path generation can be enhanced.

[0016] The following will describe in detail the specific components and functions of the multi-mode hierarchical fusion real-time obstacle avoidance path generation system for unmanned surface equipment provided in this application, with reference to the accompanying drawings and specific embodiments. Figure 1 This is a schematic diagram of the structural composition of the multi-modal hierarchical fusion real-time obstacle avoidance path generation system for unmanned surface equipment provided in the embodiments of this application, as shown below. Figure 1 As shown, the system may specifically include the following components: The acquisition and registration module 101 is used to simultaneously acquire visual data, laser point cloud data and navigation status, and register visual data and laser point cloud data according to calibration relationship and acquisition time sequence to obtain registration data; The layered fusion module 102 is used to establish a depth mapping relationship between the image and the point cloud based on the registration data, fuse the target features of the image and the point cloud according to the depth mapping relationship, obtain the fused target features, and generate candidate obstacle objects with semantic attributes and spatial state associations based on the fused target features. The interference filtering module 103 is used to determine the water surface interference state based on the cross-modal consistent state and time-series stable state of the candidate obstacle objects, and to filter the effective obstacle objects. The risk prediction module 104 is used to determine the collision risk based on the relative motion relationship between the effective obstacle object and the navigation state, and to generate a predicted motion trajectory based on the temporal state of the effective obstacle object. The path generation module 105 is used to determine the local safe and feasible region under the global flight path and motion constraints based on the collision risk and predicted motion trajectory, and to generate a continuous target obstacle avoidance path in the local safe and feasible region. The execution correction module 106 is used to convert the target obstacle avoidance path into navigation control commands and correct the navigation control commands based on the deviation between the actual track and the target obstacle avoidance path.

[0017] In some embodiments, visual data, laser point cloud data, and navigation status are acquired simultaneously, and the visual data and laser point cloud data are registered according to the calibration relationship and acquisition time sequence to obtain registration data, including: The visual acquisition unit and the lidar are jointly calibrated to determine the spatial transformation relationship between the visual coordinate system and the point cloud coordinate system. Based on the unified clock marking of the acquisition time of visual data and laser point cloud data, candidate frame pairs are formed according to the time matching status. Based on the acquisition time difference of the candidate frame pairs in the navigation status compensation, the target frame pairs that meet the timing consistency condition are determined. According to the spatial transformation relationship, the laser point cloud in the target frame pair is projected onto the image plane to establish the spatial relationship between the image region and the point cloud data, and registration data is generated.

[0018] Specifically, this embodiment uses a small to medium-sized unmanned surface vessel as a carrier. A 16-line array lidar and a high-definition binocular wide dynamic range camera are coaxially mounted on the front of the hull, ensuring that the main sensing directions of the two types of equipment are consistent and the field of view overlap rate is no less than 95%. The hull is also equipped with a positioning unit and an inertial measurement unit. The positioning unit outputs the position and speed, while the inertial measurement unit outputs the roll, pitch, heading, and angular velocity. An embedded industrial control computer receives data from each device through a unified data bus. The visual data uses the left eye image as the point cloud projection reference, and the right eye image is used to verify the depth continuity of the near-field area. The lidar point cloud data is organized according to the scanning cycle, and the scanning time, echo intensity, and radar coordinates are retained for each point.

[0019] During system initialization, the internal parameters of the camera are calibrated first. The calibration board is placed at multiple distances and attitude positions within a common field of view, and multiple sets of binocular images are acquired. The focal length, principal point, distortion parameters, and relative pose of the binoculars are calculated. Subsequently, geometric markers capable of forming stable laser echoes are set on the calibration board, and images and point clouds are acquired simultaneously. Marked corner points are extracted from the images, and corresponding planes and edges are fitted from the point cloud. A joint calibration target is constructed using the image reprojection deviation and the distance deviation from the point cloud to the calibration plane. The rotation matrix and translation vector from the radar coordinate system to the visual coordinate system are iteratively solved. After the solution is completed, the spatial transformation relationship is verified using calibration frames not involved in the calculation. When the reprojection deviation meets the preset calibration conditions, the calibration parameter version is saved; otherwise, attitude samples with larger errors are reacquired to avoid parameter degradation caused by a single calibration attitude.

[0020] Considering that hull vibration and temperature differences may cause slow drift in the installation relationship, during navigation, rigid targets with stable contours and continuous existence are selected from the common field of view. The image edges of the target are locally matched with the point cloud projection edges to calculate the online reprojection residuals. When the residuals continuously exceed the drift threshold over multiple acquisition cycles, only minor attitude deviations in the spatial transformation relationship are incrementally corrected, and the amount of correction per instance is limited. When stable targets are lacking, the most recently valid calibration parameters are maintained to avoid transient targets such as waves from participating in calibration updates.

[0021] During the data acquisition phase, the industrial control computer sends hardware trigger signals to the camera and LiDAR, and uses a unified clock to mark the acquisition time for image frames, point cloud frames, positioning data, and inertial data. The system uses the point cloud frame time as the retrieval benchmark, searching the image buffer queue for the image frame with the smallest time difference. Candidate frame pairs are formed when the time difference is within the frame pair tolerance window. When multiple image frames meet the conditions, the image frame whose exposure completion time is closest to the median scan time of the point cloud is selected. If no image frame meets the conditions, the current point cloud frame is terminated from participating in the fusion process to prevent cross-period erroneous association.

[0022] Since point cloud scanning and image exposure are not absolutely simultaneous, motion compensation is required using the navigation status. Between the two acquisition times of a candidate frame pair, the system performs temporal interpolation on the positioning position, heading, and attitude, calculates the pose change of the hull coordinates from the point cloud time to the image time, and then applies the pose change to the point cloud coordinates. For point clouds that record scanning times point by point, each laser point is first compensated to the point cloud reference time, and then compensated to the image exposure time. After compensation, the temporal and attitude residuals of the candidate frame pairs are recalculated, and candidate frame pairs that simultaneously satisfy both temporal and attitude consistency conditions are determined as target frame pairs.

[0023] For example, an unmanned surface vessel (USV) travels at a speed of 3 m / s and turns at an angular velocity of 4° / s. The median time of a certain point cloud frame differs from the most recent image exposure time by 12 ms. The system uses 100Hz inertial data and 10Hz positioning data to interpolate the translation and rotation angles for the corresponding time period, and performs inverse pose compensation on the point cloud to unify the obstacle positions represented by the point cloud to the image exposure time. If the compensated time residual is not greater than a preset synchronization threshold and the attitude interpolation is valid, the target frame pair is retained; if the inertial data is missing in the corresponding time period, the frame pair quality is reduced or registration data generation is stopped.

[0024] For a target frame pair, the system first uses a rotation matrix and translation vector to transform each laser point to the visual coordinate system, and then calculates the pixel position by combining the camera's internal parameters and distortion parameters. Points located behind the camera, outside the common field of view, or falling into invalid imaging areas are excluded. When multiple laser points are projected onto the same pixel neighborhood, representative depth is retained by combining the distance continuity state and the local point cloud clustering state to prevent distant points from penetrating and covering nearby obstacles. The system uses the image region as an index to record the depth, spatial coordinates, echo quality, and point cloud index of the associated point cloud, and encapsulates the target image frame, compensated point cloud, pixel association table, unified timestamp, navigation attitude, and frame pair quality state into registration data for subsequent depth mapping and hierarchical feature fusion.

[0025] Through the above implementation methods, joint calibration provides a stable basis for cross-sensor spatial conversion, unified clock and candidate frame screening limit asynchronous acquisition errors, navigation state compensation further eliminates target displacement caused by ship motion, and pixel-level projection association enables the visual region to obtain the corresponding point cloud depth, thereby improving the registration accuracy of images and point clouds and the reliability of subsequent fusion data in dynamic water surface scenes.

[0026] In some embodiments, a depth mapping relationship between an image and a point cloud is established based on registration data; target features of the image and point cloud are fused according to the depth mapping relationship to obtain fused target features; and candidate obstacle objects with semantic attributes and spatial state associations are generated based on the fused target features, including: The point cloud projection units in the registration data are mapped to the image regions, and depth values ​​are associated with each image region to establish a depth mapping relationship. Visual target features and spatial target features of the corresponding point cloud are extracted from the image region, respectively. The feature contribution is determined based on the data quality and cross-modal matching status of the two types of target features. The two types of target features are then cross-fused and feature-selected to obtain fused target features. Semantic recognition is performed on the fused target features, the spatial state is determined based on the deep mapping relationship, and the semantic attributes and spatial state are associated with the same target to generate candidate obstacle objects.

[0027] Specifically, the registration data includes motion-compensated point clouds, target image frames, pixel association tables, unified timestamps, flight attitude, and frame pair quality status. The layered fusion module first performs brightness equalization and edge-preserving enhancement on the target image frames, performs radius outlier filtering and local density correction on the compensated point clouds, and then reads the projection position of each laser point according to the pixel association table. The system organizes adjacent pixels with consistent image boundaries into image regions, and forms point cloud projection units by grouping laser points whose projection positions fall into the corresponding image regions, thus giving the same spatial target a unified region index in both types of data.

[0028] When establishing a depth mapping relationship, the system reads the distance of each laser point in the point cloud projection unit along the visual optical axis. When multiple distance layers exist in the neighborhood of a pixel, the distance layer closest to the camera and supported by local point cloud is first retained using the front-to-back occlusion relationship. Then, the representative depth and depth confidence state are calculated based on point cloud density, echo quality, and neighborhood distance continuity. Pixels not directly covered by laser points are only filled in using adjacent effective depths within the same image boundary, without crossing significant edges to propagate depth and avoid filling background water depth into obstacle areas. This forms a depth mapping relationship with image region identifiers as indexes, associating representative depth, depth range, point cloud index, and confidence state.

[0029] The mid-level feature extraction employs a lightweight dual-branch network suitable for embedded industrial control computers. The vision branch extracts edge responses, texture distributions, and contour representations from image regions of different resolutions, and simultaneously calculates the region's exposure state, sharpness state, and occlusion state to form visual target features and visual quality states. The point cloud branch groups associated point clouds according to spatial proximity, extracts shape, scale, and orientation representations from the point cloud distribution, and forms spatial target features and point cloud quality states based on the number of effective points, echo stability, and spatial integrity. Both branches share image region identifiers, enabling target-level alignment between visual target features and spatial target features before fusion.

[0030] The system calculates the cross-modal matching state of two types of target features based on the depth mapping relationship. Specifically, the degree of overlap between the visual contour and the point cloud projection boundary, the projection deviation between the visual target center and the point cloud spatial center, and the consistency of the two scale representations are converted into matching scores. These scores are then combined with the visual quality state, point cloud quality state, and depth reliability state to determine the feature contribution of the two types of target features. The contribution is normalized and used as a fusion gating parameter: the visual branch contribution is reduced when the image is overexposed or blurred, and the point cloud branch contribution is reduced when the point cloud is sparse or the echo is unstable, but the effective information supported by the other branch is retained.

[0031] During the cross-fusion process, corresponding point cloud spatial features are retrieved using the region response from the visual target features, and the distance and orientation relationships in the point cloud spatial features are used to inversely constrain the visual region response, resulting in bidirectional enhanced features. Subsequently, duplicate representations are merged based on feature response intensity and cross-modal consistency, suppressing weak responses that appear only at a single scale and lack target support, thus forming fused target features. The fusion result simultaneously retains the contribution of each branch, cross-modal matching score, and deep confidence state for subsequent interference filtering, rather than directly deleting low-matching targets at this stage.

[0032] The high-level semantic recognition unit calculates the target category and category confidence state based on the fused target features, classifying the target into water obstacle categories related to navigation and obstacle avoidance. The spatial state is recovered based on depth mapping: first, the target's 3D center is calculated using representative depth and pixel position; then, the target's occupied space and relative distance are determined by combining the region boundary and associated point cloud range, and converted to the ship's navigation coordinate system. The system uses region index and acquisition time as object identification basis, employs category and category confidence state as semantic attributes, and 3D position, occupied space, and distance status as spatial states, associating them with the same candidate obstacle object.

[0033] For example, in a highly reflective water area during the day, a small floating object approximately 28m away exists. Its image outline is relatively complete, but there are only 7 effective laser points. Based on high visual quality, limited point cloud density, and good boundary overlap, the system assigns a contribution of 0.62 to the visual target features and 0.38 to the spatial target features. It also obtains a representative depth of 27.8m from the stable distance layer of the effective laser points to generate candidate obstacle objects for the small floating object. Although adjacent reflective areas have strong image responses, they lack stable point cloud support and have low boundary matching scores. Therefore, the system still generates candidate objects with low cross-modal matching, which are then processed by the subsequent surface interference filtering module in conjunction with continuous frame status for discrimination. At night, when image quality deteriorates but point cloud echoes stabilize, the fusion gating parameters automatically increase the contribution of spatial target features.

[0034] Through the above implementation methods, the depth mapping relationship binds pixel semantics and point cloud distance to a unified target region. The quality status and cross-modal matching status enable the two types of features to participate in the fusion according to their effectiveness. The objectified association between semantic attributes and spatial status provides candidate obstacle data with a unified structure and retains reliable information for interference filtering, risk assessment and trajectory prediction.

[0035] In some embodiments, determining the water surface disturbance state based on the cross-modal consistent state and time-series stable state of candidate obstacle objects, and filtering valid obstacle objects, includes: Determine the target support state of each candidate obstacle object in visual data and laser point cloud data, and generate a cross-modal consistent state based on the matching degree of the two types of target support states; For the same candidate obstacle object in consecutive frames, a temporally stable state is generated based on the continuous changes in the corresponding spatial position and target features; Candidate obstacle objects are matched with water surface disturbance feature models, and the water surface disturbance state is determined by combining cross-modal consistent state and temporal stable state. Filter out candidate obstacle objects that meet the interference determination criteria, and determine the remaining candidate obstacle objects as valid obstacle objects.

[0036] Specifically, the water surface interference adaptive filtering module receives candidate obstacle objects, each carrying visual target features, spatial target features, depth confidence state, cross-modal matching score, semantic attributes, spatial state, and acquisition time. The system evaluates the target support degree of the candidate objects in the image and point cloud respectively. The visual target support state can be determined by the integrity of the region contour, the stability of the internal texture, the target response intensity, and the image quality; the point cloud target support state can be determined by the effective point density, echo continuity, spatial clustering, and depth confidence state. Each indicator is first normalized according to the corresponding data quality to avoid directly interpreting low response as no target when exposure decreases or the point cloud is naturally sparse.

[0037] When generating a cross-modal consistent state, the system projects the point cloud support region onto the image plane using depth mapping, compares the spatial overlap of the two types of support regions, and combines the center position deviation, boundary correspondence, and scale variation to form a target matching value. The target matching value is then jointly calculated with the visual target support state and the point cloud target support state. A high cross-modal consistent state occurs when both types of data form stable support and spatial correspondence; a low cross-modal consistent state occurs when only strong image bright spots exist without point cloud aggregation, or when only discrete echoes exist without continuous image contours. For small, point-like floating objects, the system adopts support conditions adapted to the target distance and the expected number of point clouds, preventing them from being directly reduced to interference objects simply because the number of laser points is small.

[0038] In the temporal processing stage, the unmanned surface vessel's own motion is compensated first, and then candidate obstacle objects in adjacent acquisition cycles are associated. The system predicts the current search area based on the spatial state of the previous frame. Within the search area, the same object is matched according to semantic compatibility, spatial proximity, and similarity of fused target features to establish object trajectory identifiers. For continuously associated objects, the effective duration is statistically analyzed, and the deviation of the spatial position from the predicted position, the degree of change of fused target features, and the continuity of depth changes are calculated to form a temporally stable state. Even if real dynamic obstacles are continuously moving, as long as the positional changes conform to the continuous motion relationship, a relatively stable state can still be obtained; wave and water mist noise usually appear suddenly, change positions abruptly, and dissipate rapidly, thus forming a lower stable state.

[0039] The water surface disturbance feature model is established using historical water area data and on-site initialization data, preserving cross-modal support patterns, spatial morphology patterns, and temporal evolution patterns according to the source of disturbance. Reflective areas typically exhibit high-brightness saturated image responses, weak point cloud support, and region drift that changes with the viewing angle; spray typically forms short-term discrete point clouds and irregular edges; water mist scattering usually presents large-scale low-density echoes and low-contrast images; ripple jitter manifests as a periodic response close to the water surface and lacking stable space occupation. The system inputs the fused target features, cross-modal consistent states, and temporal stable states of candidate objects into the disturbance feature model to calculate the matching state of the corresponding disturbance modes.

[0040] The interference determination boundary is dynamically adjusted based on the current visual quality, point cloud quality, and water surface disturbance level. When strong reflections cause a decrease in visual quality, the contribution of point cloud stability and temporal continuity to the determination is increased; when water mist causes point cloud echo attenuation, the contribution of stable image contours and continuous motion relationships is increased. The system integrates the interference mode matching state, cross-modal consistency state, and temporal stability state to generate a reliable state of water surface interference. When the interference matching degree reaches the determination boundary and the real target support is insufficient, the candidate object is marked as interference; candidate objects near the determination boundary enter a short-term pending confirmation state, retaining object records and continuing to accumulate observations within a limited number of frames to prevent the instantaneous data quality degradation from causing the real target to be missed.

[0041] For example, in a series of consecutive frames in a brightly lit water area, the visual target support value of a certain bright area is 0.84, the point cloud target support value is 0.13, and the cross-modal consistency value is 0.21. Furthermore, the location of this area drifts irregularly with changes in the ship's viewing angle. The matching value between this object and the reflective interference mode reaches 0.88, and no stable depth is formed in three consecutive frames, thus meeting the interference criteria and being filtered out. Although the plastic buoys in the adjacent water area only form a few laser points, the image outline is continuous, the point cloud distance is concentrated around 18.6m, and the compensated positional change conforms to the water flow drift trend. The cross-modal consistency value and temporal stability value reach 0.72 and 0.83 respectively, thus being identified as valid obstacle objects. The wave noise appearing on the side of the ship disappears after two acquisition cycles. The system only filters the local candidate objects corresponding to the waves, maintaining the continuous trajectory identification of the ship object.

[0042] Furthermore, after completing the judgment, the system deletes the candidate objects and their corresponding temporary trajectories that meet the interference judgment conditions, while retaining their judgment records for parameter verification; the remaining candidate objects are converted into valid obstacle objects, and the object trajectory identifiers, semantic attributes, spatial states, and perception credibility states are updated for use in collision risk calculation and motion trajectory prediction.

[0043] Through the above implementation methods, cross-modal target support can identify false responses generated by a single sensor, time-series stability judgment can distinguish between transient surface noise and continuously moving targets, and adaptive judgment boundary can adapt to changes in lighting and water mist, thereby reducing false detections and missed detections in complex water surface environments.

[0044] In some embodiments, matching candidate obstacle objects with a water surface disturbance feature model and determining the water surface disturbance state by combining cross-modal consistent states and temporally stable states includes: The interference feature matching degree is determined based on the target features of the candidate obstacle object, and the true target support degree is determined based on the cross-modal consistent state and the temporal stable state. The interference judgment boundary is adjusted according to the quality status of visual data and laser point cloud data. The interference feature matching degree and the real target support degree are matched with the adjusted interference judgment boundary to determine the water surface interference status. When candidate obstacle objects contain local regions with different water surface disturbance states, the local regions are divided according to the continuity of the feature space, and the local regions that meet the support conditions of the real target are retained.

[0045] Specifically, the water surface interference feature model includes multiple interference prototypes formed from historical inland river and near-shore data. Each interference prototype stores correlation parameters between image representation, point cloud representation, spatial morphology, and temporal variations. The system does not determine interference based on a single brightness or point cloud density, but rather extracts interference representations adapted to the current observation quality from the fused target features of candidate obstacle objects. For the visual part, it calculates the degree of abrupt change in the region's brightness distribution relative to the surrounding water surface, the regularity of edges, and the persistence of texture; for the point cloud part, it calculates the echo discrete state, depth clustering state, and the integrity of occupied space; and then combines the region's position relative to the water surface and continuous frame variations to form the features to be matched.

[0046] The system calculates the normalized distance between the feature to be matched and each interference prototype, converts the distance into a similarity state of the corresponding prototype, and selects the interference prototype with the greatest support as the candidate interference type. To avoid the influence of different feature dimensions on the matching results, each feature is first standardized using the background statistical range of the current water area; features outside the reliable range are not included in this matching. The system adjusts the similarity state based on the proportion of effective features participating in the matching to obtain the interference feature matching degree. If there are too few effective features, the interference is not directly confirmed; instead, the matching confidence state is reduced and the system waits for subsequent frames to supplement the information.

[0047] In some examples, the true target support is formed jointly based on cross-modal consistent state and temporally stable state. The system compresses the overlap between the visual target support region and the point cloud support region, the continuity of the depth mapping, and the matching state of the two types of target features into a cross-modal support component, and compresses the object's duration, the predictability of positional changes, and the retention of target features into a temporal support component. When both components are high, the true target support increases accordingly; when one component decreases, compensation is made based on the target distance, the category confidence state, and the expected sensor visibility. For example, small floating objects at a distance are expected to form only a few laser points, but as long as the point cloud depth is concentrated and the image contour is continuous, a true target support that meets the conditions can still be formed.

[0048] The interference determination boundary includes the interference matching boundary and the true support boundary, with initial values ​​determined by on-site initialization data. Within a sliding time window, the system statistically analyzes visual exposure, sharpness, effective point cloud ratio, and echo stability to form visual quality and point cloud quality states. When visual overexposure occurs, the contribution of simple highlight features to the interference matching degree is reduced, while the determination weights for point cloud spatial aggregation and temporal continuity are increased. When water mist degrades point cloud quality, the point cloud support requirements are relaxed, while the requirements for image contour continuity and cross-frame predictability are increased. The adjustment process sets the maximum variation range and recovery rate to prevent frequent fluctuations in the determination boundary caused by short-term splashes.

[0049] When the interference feature matching degree reaches the adjusted interference matching boundary, and the actual target support degree is lower than the adjusted actual support boundary, the surface interference state is determined to be interference; when the actual target support degree meets the retention condition, it is determined to be non-interference; otherwise, it enters the pending confirmation state. The object to be confirmed maintains its trajectory identification within the limited observation period, and the system recalculates the two types of metrics after accumulating new observations. If the same judgment result is achieved continuously, the state confirmation is completed; if it still cannot be confirmed after the limited period, the object is retained according to the navigation safety level, and the perception credibility state is reduced to avoid uncertain small targets being directly deleted.

[0050] In some examples, candidate obstacle objects may simultaneously cover both the real target and water surface interference, such as a ship's edge being connected to a reflective spot, or the lower part of a buoy being covered by a wave area. To address this, the system uses image pixel blocks and associated point cloud clusters as region nodes, establishing feature space connectivity based on spatial adjacency, target feature similarity, and depth continuity. Region nodes with strong connectivity and similar interference matching states are merged into a single local region; significant image boundaries, depth jumps, or abrupt changes in matching states serve as partitioning boundaries. The system recalculates the interference feature matching degree and real target support for each local region, rather than using the uniform judgment result for the entire candidate object.

[0051] For example, the side area of ​​a vessel 22m away from the unmanned surface vessel is connected to the bright reflection on the water surface. Approximately 30% of the image area in the overall candidate objects is a saturated bright spot without a stable point cloud, with a local interference feature matching degree of 0.91 and a true target support degree of 0.16. The remaining areas have a continuous hull outline and a stable depth layer, with an interference feature matching degree of 0.24 and a true target support degree of 0.87. The system divides the image into two local regions along the brightness abrupt change boundary and the depth interruption location, filters out the high-matching, low-support regions, retains the hull area, and recalculates the 3D center, occupied space, and category confidence state using the image boundaries and point cloud range of the retained regions. For buoys that are briefly occluded by waves, if the unoccluded part still meets the true target support condition, the buoy object and trajectory identifier are maintained, only the spatial integrity state of the current frame is reduced.

[0052] Through the above implementation methods, the interference feature matching degree characterizes the degree to which candidate objects conform to the water surface interference pattern, the real target support degree characterizes the degree of support of cross-modal and time-series evidence for the real target, the quality adaptive boundary makes the judgment conditions adjust with the effectiveness of the sensor, and the local region division avoids the deletion of complete obstacles due to partial reflection or wave coverage, thus taking into account both water surface interference suppression and real obstacle preservation.

[0053] In some embodiments, determining the collision risk based on the relative motion relationship between the effective obstacle object and the navigation state, and generating a predicted trajectory based on the temporal state of the effective obstacle object, includes: For the same valid obstacle object in consecutive frames, estimate its current motion state according to the temporal changes of its corresponding spatial state; Transform the current motion state and navigation state into a unified motion coordinate system to determine the approach state and predicted encounter state of effective obstacle objects relative to the unmanned equipment on the water surface. The safety boundary is determined based on the semantic attributes and spatial state of the effective obstacle object. The predicted encounter state is then matched with the safety boundary to determine the collision risk. Based on continuous frame observations, the current motion state is recursively corrected, and the future spatial state of effective obstacle objects is deduced within the target prediction time domain to generate a predicted motion trajectory.

[0054] Specifically, the risk prediction module receives valid obstacle objects and synchronous navigation status. The system arranges consecutive frame objects according to the acquisition time and performs continuation verification on existing trajectory identifiers; when trajectory identifiers are missing, the current search range is predicted based on the motion state of the previous frame. Within the search range, objects are associated according to spatial proximity, semantic compatibility, and similarity of fused target features to prevent trajectory swapping between adjacent targets.

[0055] For valid obstacle objects that have completed continuous association, the system transforms the spatial position of each frame to a unified geographic reference coordinate system, eliminating the apparent motion caused by the displacement of the unmanned vessel itself, and then calculates the position change according to the time interval. The motion state can be composed of the target position, motion speed, motion direction, and state uncertainty range. When the trajectory is formed for the first time, the initial velocity is estimated using adjacent observation positions; after continuous observations reach a preset number, the predicted state at the previous moment is fused with the current observation state through a recursive state estimator. When the perceived confidence state is high, the contribution of the current observation to the state correction is increased; when the point cloud is sparse or the target is partially occluded, the contribution of the current observation is reduced to avoid sudden velocity changes caused by single-frame position jitter.

[0056] A unified motion coordinate system can be selected, with the current position of the unmanned surface vessel (USV) as the origin and the current heading as the longitudinal direction. The system uses the positioning position and inertial attitude to transform the current motion state of the effective obstacle object to this coordinate system, and simultaneously converts the USV's velocity to the same directional reference to calculate the target's relative position and relative velocity. The velocity component along the relative position direction is used to determine whether the two sides are continuously approaching, and combined with the relative motion straight line or short-time curve, the expected nearest encounter time, the nearest encounter distance, and the encounter direction relationship are obtained, forming the approach state and the predicted encounter state. When the nearest encounter time is outside the predicted range or the distance between the two sides continues to increase, the approach state is correspondingly reduced.

[0057] In some examples, the safety boundary is not a fixed radius, but is determined based on the semantic attributes, occupied space, and motion state of the effective obstacle object. Large ships have a large occupied space and motion inertia, so the system expands the longitudinal safety range along its direction of motion; small floating objects occupy less space, but the boundary still needs to be expanded when the perceived uncertainty range is large. The system also increases the execution margin based on the unmanned vessel's current speed, permissible turning capability, and braking distance, and expands the boundary outward based on the target state uncertainty range, forming a dynamic safety boundary that changes with the target direction and credible state.

[0058] Collision risk is determined by matching the predicted encounter state with the dynamic safety boundary. The system judges whether the predicted nearest encounter position enters the safety boundary and forms a risk value based on the expected nearest encounter time, the degree of boundary intrusion, and the relative approach trend. The shorter the expected time, the greater the degree of intrusion, and the more obvious the approach trend, the higher the risk value. The risk value is compared with the preset risk boundary to form safe, low-risk, medium-risk, and high-risk states. When the perceived trust state decreases but the target may enter the safety boundary, the system adopts a more safe risk state and retains the source of risk and the range of state uncertainty for the path generation module to set the safety interval.

[0059] The trajectory prediction employs a combination of recursive prediction and observation correction. The system determines the degree of motion change based on the target's most recent temporal state. When the motion direction and velocity are stable, a uniform velocity propagation relationship is used; when a continuous turning or acceleration trend occurs, changes in angular velocity or acceleration are incorporated into the state propagation relationship. For each new observation frame acquired, the predicted position is first compared with the actual observed position to form a state residual. Then, the position, velocity, and uncertainty range are corrected according to the reliable observation state. After correction, starting from the current acquisition time, the future spatial state is progressively extrapolated at fixed time intervals within the target prediction time domain from 1 to 3 seconds. The predicted positions, corresponding occupied spaces, and uncertainty ranges are then connected in chronological order to form the predicted motion trajectory.

[0060] When a target observation is lost for a short period, the system continues to extrapolate along the most recent valid motion state within a limited period, expanding the uncertainty range. After the same object is reacquired, the predicted trajectory is corrected using the new observation. If the target is not re-observed after the trajectory holding period, the corresponding predicted motion trajectory is stopped from being output, and the last risk record is retained for smooth exit in adjacent planning periods.

[0061] For example, an unmanned surface vessel (USV) travels along the course at 3 m / s, while a small motorized vessel 20 m to its right approaches laterally at 2 m / s. Six consecutive frames of observation show the target's course is stable, and unified coordinate calculations yield a predicted closest encounter time of 2.8 s and a closest encounter distance of 2.4 m. Based on the target size, the USV's turning capability, and the predicted uncertainty range, a safety boundary of 5 m is established in the encounter direction. If the predicted encounter position enters this boundary, the system classifies the object as high-risk and outputs a time-sharing predicted position for the next 3 seconds. If the relative velocity of a distant buoy is close to zero and the closest encounter distance is greater than the corresponding safety boundary, the system maintains a low-risk or safe status.

[0062] Through the above implementation methods, continuous object association and recursive state estimation can maintain the stability of the dynamic target motion state, unified coordinate transformation can eliminate the influence of the unmanned vessel's own motion on the relative relationship, dynamic safety boundary and predicted encounter state can incorporate target type, space occupation and motion uncertainty into risk assessment, and time-sharing trajectory extrapolation provides continuous spatiotemporal basis for the subsequent generation of forward obstacle avoidance paths.

[0063] In some embodiments, based on collision risk and predicted trajectory, a local safe and feasible region is determined under global flight path and motion constraints, and a continuous target obstacle avoidance path is generated within the local safe and feasible region, including: The planning time domain and safety interval are determined based on collision risk, and a dynamic obstacle occupancy domain is constructed by combining the predicted motion trajectory. A set of reachable motion states is generated based on the current navigation state and motion constraints, and local trajectory deduction is performed on the motion states in the set of reachable motion states to obtain candidate navigation trajectories; Candidate navigation trajectories with spatiotemporal overlap exist in the filtering and dynamic obstacle occupancy domains. A local safe and feasible domain is formed based on the remaining candidate navigation trajectories. The evaluation weights of global flight path consistency constraints and path continuity constraints are adjusted according to collision risk. The target trajectory is determined from the local safe and feasible domain and continuity correction is performed to generate the target obstacle avoidance path.

[0064] Specifically, the real-time obstacle avoidance path generation module receives the current navigation status, the global route, the collision risk of each effective obstacle, and the predicted trajectory. The global route consists of several route positions and target headings, while the current navigation status includes the unmanned vessel's position, heading, speed, and turning changes. Motion constraints are pre-configured based on the vessel's hydrodynamic characteristics and actuator capabilities, limiting at least the speed variation range, allowable acceleration / deceleration range, heading angular velocity range, minimum turning radius, and control response delay, ensuring that the planned results fall within the actual range of motion achievable by the unmanned vessel.

[0065] The system determines the planning time domain and update cycle for this local plan based on the highest current collision risk. For no or low risk scenarios, a shorter planning time domain is used and the frequency of repeated planning is reduced to maintain overall route stability. For medium risk scenarios, the planning time domain is extended to search for detour spaces in advance. For high risk scenarios, the path update cycle is shortened, and the planning time domain covers the expected encounter time. The safety interval is based on the dynamic safety boundary of obstacles, and is extended by combining the predicted uncertainty range, the current speed of the unmanned vessel, and the control response distance. The higher the risk or the larger the predicted uncertainty range, the larger the safety interval.

[0066] When constructing the dynamic obstacle occupancy domain, the system reads the time-division position and occupied space in the predicted motion trajectory according to a uniform time step, and superimposes the safety interval onto the outside of the occupied space at each predicted time, forming obstacle occupancy units that move over time. Obstacle occupancy units at adjacent time steps are connected according to the target's reachable motion range to prevent candidate trajectories from crossing between discrete predicted positions. Static obstacles maintain fixed occupancy units within the planning time domain, while dynamic obstacles form gradually changing occupancy units based on their predicted positions and uncertain ranges. All time-division occupancy units together constitute the dynamic obstacle occupancy domain.

[0067] The set of reachable motion states is generated using a lightweight dynamic window approach. Centered on the current speed and heading angular velocity, the system determines the speed and turning windows based on the allowable acceleration, deceleration, and steering variation ranges within the planning period. Motion states are then generated according to sampling intervals adapted to the risk level. In high-risk scenarios, sampling closer to safe turning and deceleration directions is increased; in low-risk scenarios, sampling significantly deviating from the overall course is reduced. For each motion state, starting from the current ship position, the system time-by-time extrapolates the ship's position, heading, and speed, taking into account control response delays and turning inertia, to form a candidate navigation trajectory with a time index.

[0068] In some examples, candidate trajectories must also satisfy the boundaries of navigable waters and the allowable range of the global mission. The system establishes a navigation envelope adapted to the ship's shape along each candidate trajectory and compares the navigation envelope with the dynamic obstacle occupancy domain at the same prediction time. If spatial overlap occurs at any time, or if the minimum interval is lower than the corresponding safety interval, the candidate trajectory is determined as infeasible and filtered out. The remaining candidate trajectories, together with their corresponding motion states, control sequences, and safety margins, constitute a local safe feasible region. When the local safe feasible region is empty, the system prioritizes generating a controlled deceleration trajectory that meets the motion constraints and triggers a replanning with a shortened cycle.

[0069] The system evaluates the proximity of each candidate navigation trajectory within the local safe feasible domain to the global route, as well as the continuity of the candidate navigation trajectory with the target trajectory of the previous planning period in terms of position, heading, and speed changes. Global route consistency constraints limit unnecessary detours, while path continuity constraints limit trajectory jumps between planning periods; simultaneously, trajectory safety margin serves as the basis for safety evaluation within the feasible domain. In low-risk scenarios, the evaluation weight of global route consistency constraints is increased, allowing the unmanned vessel to maintain its original route or make only minor corrections; in medium-risk scenarios, the contributions of path continuity constraints and safety margins are increased; in high-risk scenarios, avoiding dynamically occupied obstacle domains is prioritized, and route deviations are allowed to increase within the motion constraints.

[0070] The system selects the target trajectory based on the weighted evaluation results. If the difference between the new target trajectory and the target trajectory of the previous planning cycle does not exceed the continuity boundary, the new target trajectory is directly continued; if the difference exceeds the continuity boundary, it is first determined whether the previous target trajectory is still within the local safe and feasible region. If the previous target trajectory is still safe and the evaluation improvement of the new target trajectory is insufficient, the previous target trajectory is maintained; if the previous target trajectory is no longer safe, a transition segment is established between the old and new trajectories, starting from the current actual navigation state, and the curvature and speed changes of the transition segment are restricted. After the transition is completed, curve smoothing is performed on local inflection points to generate a continuous target obstacle avoidance path.

[0071] For example, for the small motorized vessel expected to enter the safety boundary in 2.8 seconds, the system sets the local planning time domain to 3 seconds and constructs a dynamic obstacle occupancy domain with a time step of 0.1 seconds. Candidate navigation trajectories are generated based on the unmanned vessel's current speed of 3 m / s, permissible acceleration / deceleration states, and turning capabilities, eliminating trajectories that overlap with the motorized vessel's occupancy domain within 2.2 to 3 seconds. If maintaining the global route is no longer feasible, the system selects a detour trajectory from the remaining trajectories with sufficient safety margin and low cost of connecting with the previous trajectory, gradually changing the course through transition sections. Once the obstacle is reduced to low risk, the consistency constraint weight of the global route is increased, causing subsequent paths to gradually approach the original route.

[0072] Through the above implementation methods, the time-sharing dynamic obstacle occupancy domain can incorporate the future movement of obstacles into the collision screening, the reachable motion state set can ensure that the candidate trajectory conforms to the actual movement capability of the unmanned vessel, the risk adaptive evaluation weight can take into account both safe detour and global navigation efficiency, and the cross-cycle continuous correction can reduce the sudden changes in heading and speed jitter caused by local path switching.

[0073] In some embodiments, a dynamic obstacle occupancy domain is constructed, and a target trajectory is determined from a local safe and feasible domain, including: Based on the predicted uncertainty range and collision risk of the predicted motion trajectory at each prediction time, the spatial range of the corresponding effective obstacle object is expanded in time to construct a dynamic obstacle occupancy domain. The trajectory evaluation status is determined based on the safety status of the candidate navigation trajectory relative to the dynamic obstacle-occupied domain, the consistency status relative to the global route, and the continuity status relative to the target trajectory of the previous planning cycle. Candidate target trajectories are determined by evaluating the contribution of each state in the trajectory assessment based on collision risk. When the difference between the candidate target trajectory and the target trajectory of the previous planning period exceeds the continuity threshold, the candidate target trajectory will be determined as the target trajectory if the target trajectory of the previous planning period does not meet the safety conditions or the trajectory evaluation status meets the switching conditions.

[0074] Specifically, the path generation module represents the predicted trajectory of each valid obstacle object as a time-ordered sequence of predicted position, occupied space, and predicted uncertainty range. The predicted uncertainty range can be formed by the position error range output by the recursive state estimator and gradually increases with the extension of the prediction time. The system first transforms the occupied space of the obstacle at the current moment to the local planning coordinate system, then moves the occupied space according to the position at each prediction moment, and superimposes the corresponding predicted uncertainty range along the outer edge of the space to form a basic time-sharing occupancy unit.

[0075] Collision risk is used to further refine the time-sharing expansion. Low-risk objects are expanded by an amount close to the basic prediction error to avoid leaving too much space for targets far from the route. Medium-risk objects have an expansion amount adapted to the unmanned vessel's control response distance. High-risk objects have their safety range expanded on the side facing the unmanned vessel based on relative approach speed, expected encounter time, and state confidence level. For objects with obvious directional change trends or missing short-term observations, the system also adds asymmetric expansion along the possible direction of motion. Expansion regions at adjacent prediction times are connected according to the target's maximum reachable displacement, and the time-sharing expansion regions of multiple objects together form a dynamic obstacle occupancy domain according to time indexes.

[0076] The safety status of candidate navigation trajectories is determined through time-sharing collision checks. The system establishes a hull navigation envelope at each trajectory time point, compares this envelope with the dynamic obstacle occupancy domain at the same predicted time, and calculates the minimum spatial interval, the time to enter the expansion region, and the trend of safety margin changes. Trajectories with spatiotemporal overlap are directly marked as infeasible; trajectories without overlap are assigned a safe status based on the minimum interval and the stability of the safety margin. Trajectories with continuously decreasing safety margins receive a lower safety rating even if they do not currently intersect, preventing the selection of edge trajectories that are about to enter high-risk areas.

[0077] In some examples, the global course consistency status is used to evaluate whether candidate trajectories maintain the mission's course. The system projects each position of the candidate trajectory onto the global course, calculates the forward movement status along the course, the lateral deviation status, and the terminal heading recovery status, and then summarizes these according to the planning time domain. Trajectories that can continuously advance towards the global target and easily return to the original course after obstacle avoidance have a high consistency status. The continuous status compares the position, heading, speed, and curvature changes of candidate trajectories within overlapping time periods with the current actual course status and the target trajectory of the previous planning period as references, and focuses on evaluating the initial segment of the trajectory to be executed, avoiding excessive influence of minor differences at distant points on the current path selection.

[0078] The system normalizes the safety state, global course consistency state, and continuous state into a unified evaluation range, and determines the contribution of each state based on collision risk. In the absence of risk or with low risk, the global course consistency state and continuous state contribute significantly, ensuring the unmanned vessel maintains its original course. In the medium-risk state, the contribution of the safety state is increased, while the limitations of the continuous state on path smoothing are retained. In the high-risk state, the safety state becomes the primary evaluation criterion; as long as the candidate trajectory meets the motion constraints, a large course deviation is acceptable. The system also sets a minimum safety threshold; any trajectory below this threshold will not be selected despite higher evaluations for other states. After weighting, the trajectory with the optimal evaluation result and meeting the minimum safety threshold is determined as the candidate target trajectory.

[0079] In some examples, to avoid repeated switching between feasible trajectories on both sides of an obstacle in adjacent planning cycles, the system performs a switching determination on candidate target trajectories. First, it compares the trajectory distance, heading changes, and control state changes between the candidate target trajectory and the target trajectory of the previous planning cycle during the common planning period, and the comparison results form a trajectory difference. If the trajectory difference does not exceed the continuity threshold, the candidate target trajectory is directly adopted; if the trajectory difference exceeds the continuity threshold, the previous target trajectory is re-examined using the latest dynamic obstacle occupancy domain. If the previous target trajectory has spatiotemporal overlap, falls below the minimum safety threshold, or no longer meets motion constraints, a switch is performed immediately.

[0080] When the previous target trajectory remains safe, the system calculates the improvement in the candidate target trajectory relative to the previous target trajectory. Switching is only confirmed when the improvement reaches the switching threshold corresponding to the current collision risk; otherwise, the executable portion of the previous target trajectory is maintained, and only the unexecuted far-end trajectory is updated. A higher switching threshold is used in low-to-medium risk scenarios to suppress path swaying caused by minor perceptual fluctuations; the switching threshold is lowered in high-risk scenarios to respond promptly to sudden changes in obstacle direction. After switching, the curvature and velocity of the new trajectory's initial segment are continuously corrected using the current actual motion state as the connection starting point, and the target trajectory version is updated to prevent old control commands from continuing to apply.

[0081] For example, if the uncertainty range of a high-risk vessel's position increases from 0.4m to 1.2m in the next 3 seconds, the system, considering the approach speed, will extend the unmanned vessel's position by an additional 1m. The left-hand bypass trajectory selected in the first planning cycle maintains a minimum safety margin of 3.5m. In the next cycle, due to a slight shift in target observation, the overall evaluation of the right-hand trajectory only improves by 0.03, failing to reach the 0.08 switching threshold, and the system maintains the left-hand trajectory. In the cycle following that, the target clearly changes direction, and the left-hand trajectory enters the updated dynamic obstacle occupancy domain. The system no longer compares the evaluation improvement amount but directly switches to the right-hand trajectory that meets the safety threshold, and continuously corrects to limit changes in the initial heading.

[0082] Through the above implementation methods, the predicted uncertainty range and collision risk jointly control the time-sharing expansion of the obstacle space range, so that the dynamic obstacle occupancy domain covers the area that may be encountered; the safety state, the global route consistency state and the continuous state form a layered trajectory evaluation; the switching mechanism with safety priority and evaluation improvement threshold can take into account both the response to sudden risks and the stability of cross-cycle paths, and reduce the heading jitter caused by repeated left and right detours.

[0083] In some embodiments, the target obstacle avoidance path is converted into navigation control commands, and the navigation control commands are corrected based on the deviation between the actual trajectory and the target obstacle avoidance path, including: Discretize the target obstacle avoidance path according to the control cycle to generate a target motion state sequence corresponding to the path position; Based on the tracking relationship between the navigation state and the target motion state sequence, the target control state of the actuator is determined through motion control mapping, and navigation control commands are generated. The actual motion state collected is aligned with the target motion state sequence to determine the tracking deviation of the actual trajectory relative to the target obstacle avoidance path. The path tracking parameters are corrected based on the tracking deviation, and the navigation control commands are updated under the constraints of the change in actuators; When the obstacle avoidance conditions are met, a connecting path is generated based on the actual flight path and the global route, and the navigation control instructions are updated according to the connecting path.

[0084] Specifically, the navigation execution and feedback correction module receives the target obstacle avoidance path, current navigation status, global route, and obstacle risk status. The target obstacle avoidance path includes the path position, target heading, target speed, and local curvature arranged according to the planning time. The system resamples the path at fixed control cycles of the controller. When the distance between adjacent planning points is too large, intermediate positions are inserted according to the curvature continuity. When adjacent planning points are too close, key positions that can characterize changes in heading and curvature are retained, forming a sequence of target motion states corresponding to the execution time. Each target motion state is also associated with the cumulative path length and the target arrival time for subsequent progress matching.

[0085] The motion control mapping is established based on the unmanned surface vessel's (USV) maneuver response model, mapping changes in target heading and speed to target servo steering angles and target propulsion motor speeds. The maneuver response model can be obtained through still-water calibration navigation, recording the heading response caused by servo angle changes at different speeds and the speed response caused by motor speed changes, thus forming segmented control parameters. During the execution phase, the system reads the target state corresponding to the current control cycle from the target motion state sequence, compares the target state with the real-time navigation state, calculates the required heading and speed changes, and then determines the target control state of the actuators by combining the current speed, control response delay, and hull steering inertia.

[0086] The navigation control command is formed by a combination of feedforward and feedback control variables. The feedforward control variable pre-determines the base rudder angle and base rotational speed based on the local curvature of the target path and the target speed, while the feedback control variable corrects for the deviation of the current position relative to the target path. For path segments with large curvature, the system applies a rudder angle in advance that matches the steering trend, reducing lag correction after entering the curve; for path segments requiring deceleration and obstacle avoidance, the propulsion motor speed is reduced in advance to match the actual speed change with the path's time state. After combination, the target rudder angle, target rotational speed, command activation time, and path version are encapsulated into a navigation control command.

[0087] The actual motion state is synchronously acquired by the positioning unit and the inertial measurement unit, including the actual position, actual heading, actual speed, and actual angular velocity. To avoid jumps in progress caused by using only the shortest path point, the system uses the cumulative path length of the previous control cycle as the starting point, projects the actual position onto the target obstacle avoidance path within a limited forward search range, and determines the current path progress by combining the target arrival time and the actual distance traveled. Subsequently, the target motion state at the corresponding progress position is read, and the lateral position deviation, path direction deviation, heading deviation, and speed deviation are calculated to form the tracking deviation of the actual trajectory relative to the target obstacle avoidance path.

[0088] In some examples, the system adjusts the path tracking parameters based on the magnitude, direction, and duration of the tracking deviation. When the lateral deviation continues to increase, the contribution of heading feedback to rudder angle correction is increased; when the heading deviation has decreased but the lateral deviation still exists, the instantaneous rudder angle change is reduced and the smooth return distance is extended; when the speed deviation persists, the speed mapping parameters are adjusted. If the actual angular velocity and the target angular velocity are inconsistent for a long period, the steering compensation amount for the current control segment can be updated according to the response difference at the corresponding speed to adapt to changes in water flow and load, but the basic maneuvering response model is not directly rewritten.

[0089] When updating navigation control commands, constraints related to actuator changes must also be met. The system limits the changes in rudder angle, motor speed, and command reversal frequency within a single control cycle, and performs saturation processing on control quantities approaching mechanical boundaries. When the correction required for tracking deviation exceeds the actuator's achievable range, the allowable correction is allocated according to safety priorities, and the unachieved deviation is fed back to the next path planning cycle, allowing the path generation module to adjust subsequent trajectories instead of continuously outputting unexecutable commands. After each command is issued, the target control status, actual response status, and path version are recorded to prevent older command versions from continuing to execute after a path switch.

[0090] For example, an unmanned surface vessel (USV) executes a right-turn obstacle avoidance path at a target speed of 3 m / s. Due to lateral currents, its actual trajectory deviates 0.8 m outside the target path within five control cycles, resulting in a heading deviation of 3°. The system gradually increases the rudder angle correction based on the progress alignment results, but limits the rudder angle change within a set range for each cycle, while slightly reducing the propulsion speed to avoid high-speed sharp turns. As the lateral deviation decreases, the system reduces the feedback control input and restores the target speed. If the deviation continuously exceeds the correctable boundary, it reports an insufficient execution capability to the path generation module, triggering a local path replanning.

[0091] Obstacle avoidance completion conditions can be jointly determined based on obstacle collision risk, relative position, and the progress of the target obstacle avoidance path. When the relevant obstacle remains in a safe state for several consecutive cycles, has passed the most recent encounter position, and the unmanned vessel has reached the return interval of the target obstacle avoidance path, the system selects an accessible position ahead of the global route from the end of the actual track as the return anchor point.

[0092] Based on the current actual position, heading, speed, and the corresponding route status of the return anchor point, a curvature-continuous connecting path is generated. The path is then checked to ensure it enters a dynamic obstacle-occupied domain and meets motion constraints. If the check passes, the connecting path is discretized into a new sequence of target motion states, and navigation control commands are updated. If the check fails, navigation continues along the current safe path, and the return is delayed.

[0093] Through the above implementation methods, the target motion state sequence with time and path progress can stably connect planning and control, the motion control mapping combining feedforward and feedback can reduce steering and speed response lag, the progress alignment and constrained correction can limit trajectory deviation caused by water flow disturbance, and the connecting path generated based on the actual trajectory can enable the unmanned vessel to smoothly return to the global route after obstacle avoidance.

[0094] The above embodiments have described in detail the specific components and functions of the multi-mode hierarchical fusion real-time obstacle avoidance path generation system for unmanned surface equipment of this application. The implementation process of the multi-mode hierarchical fusion real-time obstacle avoidance path generation method for unmanned surface equipment of this application will be described in detail below with reference to specific embodiments. Figure 2 This is a flowchart illustrating the real-time obstacle avoidance path generation method for multi-modal hierarchical fusion unmanned surface equipment provided in this application embodiment, as shown below. Figure 2 As shown, the method may specifically include the following steps: S201, simultaneously acquires visual data, laser point cloud data and navigation status, and registers visual data and laser point cloud data according to calibration relationship and acquisition time sequence to obtain registration data; S202, establish the depth mapping relationship between the image and the point cloud based on the registration data, fuse the target features of the image and the point cloud according to the depth mapping relationship to obtain the fused target features, and generate candidate obstacle objects with semantic attributes and spatial state associations based on the fused target features. S203, determine the water surface disturbance state based on the cross-modal consistent state and time-series stable state of the candidate obstacle objects, and filter the effective obstacle objects; S204, determine the collision risk based on the relative motion relationship between the effective obstacle object and the navigation state, and generate a predicted motion trajectory based on the temporal state of the effective obstacle object; S205: Based on collision risk and predicted trajectory, determine local safe and feasible domain under global flight path and motion constraints, and generate continuous target obstacle avoidance paths within the local safe and feasible domain. S206 converts the target obstacle avoidance path into navigation control commands, obtains the actual trajectory, and corrects the navigation control commands based on the deviation between the actual trajectory and the target obstacle avoidance path.

[0095] Specifically, in S201, after the unmanned surface vessel enters autonomous navigation mode, it simultaneously acquires images of the water area and 3D point clouds through a front-mounted visual acquisition unit and lidar, and acquires position, heading, speed, and attitude through a positioning unit and inertial measurement unit. The system pre-determines the spatial transformation relationship between the visual coordinate system and the point cloud coordinate system using joint calibration, and uses a unified clock to mark the acquisition time of various data.

[0096] The image frames with the smallest time difference are matched based on the point cloud frame time to form candidate frame pairs. Then, based on the changes in navigation status during the acquisition of candidate frame pairs, pose compensation is performed on the point cloud to determine the target frame pairs that meet the time consistency condition. The target point cloud is projected onto the image plane according to the spatial transformation relationship to form registration data that includes the target image, compensated point cloud, pixel association information, unified timestamp, and frame pair quality status.

[0097] S202: Based on the projection position of the laser points in the registration data, the point cloud projection units are associated with the corresponding image regions, assigning representative depth and depth confidence states to the image regions, and establishing a depth mapping relationship between the image and the point cloud. The system extracts visual target features of the image regions and spatial target features of the associated point clouds through a lightweight dual-branch network, and then determines the contribution of the two types of features based on visual quality, point cloud quality, and cross-modal matching status.

[0098] The two types of target features are cross-fused and redundancy-filtered according to their contribution to form fused target features. The high-level semantic recognition unit determines the target category based on the fused target features, and uses the depth mapping relationship to recover the target's three-dimensional position and occupied space, associating semantic attributes and spatial state with the same target to generate candidate obstacle objects.

[0099] S203 evaluates the target support state of candidate obstacle objects in visual data and laser point cloud data respectively, and generates a cross-modal consistent state based on the overlap, boundary correspondence, and depth continuity of the two types of support regions. After compensating for the motion of the unmanned equipment, the system associates the same candidate object in consecutive frames and generates a temporally stable state based on the object's duration, spatial position changes, and fused target feature changes.

[0100] The candidate objects are then matched with the water surface interference feature model, and the water surface interference state is determined by combining cross-modal consistent state, temporal stable state, and current sensor quality. Reflections, water mist, spray, and ripple responses that meet the interference judgment conditions are filtered out, and the remaining objects are identified as valid obstacle objects. When the real target is connected to the interference area, the local area is divided according to the continuity relationship of the feature space, and only the local area lacking real target support is filtered out.

[0101] S204: Based on the object trajectory identifier, associate the same valid obstacle object in consecutive frames, and recursively estimate the current position, velocity, direction of motion, and uncertainty range of the state according to the temporal changes of the spatial state. The system transforms the obstacle motion state and the unmanned equipment navigation state into a unified motion coordinate system, calculates the relative position, relative velocity, expected nearest encounter time, and nearest encounter distance, and forms the approach state and the predicted encounter state.

[0102] Then, based on the semantic attributes of obstacles, occupied space, motion state, and the steering and braking capabilities of unmanned equipment, a dynamic safety boundary is determined. The predicted encounter state is matched with the dynamic safety boundary to determine the collision risk. The target motion state is recursively corrected based on continuous frame observations, and the future spatial state is extrapolated hourly within the 1 to 3s prediction time domain to generate a predicted motion trajectory with a time index and uncertainty range.

[0103] S205 determines the planning time domain, path update cycle, and safety interval based on the current highest collision risk. It then expands the predicted trajectory's position, occupied space, and uncertainty range at each prediction time in a time-sharing manner to construct a dynamic obstacle occupancy domain. The system generates a set of reachable motion states based on the current navigation status and motion constraints such as speed changes, steering capability, and turning radius. It then performs local trajectory extrapolation for each motion state to obtain candidate navigation trajectories.

[0104] The ship's navigation envelope and dynamic obstacle occupancy domain of candidate trajectories are compared at the same prediction time to filter out trajectories with spatiotemporal overlap or insufficient safety intervals, forming a local safe and feasible domain. Then, based on the evaluation contributions of collision risk adjustment safety state, global course consistency state, and cross-cycle continuous state, the target trajectory is selected from the local safe and feasible domain, and the target obstacle avoidance path is generated through switching determination and curvature continuous correction.

[0105] S206: Discretize the target obstacle avoidance path according to the control cycle to generate a target motion state sequence containing the target position, target heading, target speed, and path progress. Based on the tracking relationship between the current navigation state and the target motion state, the system converts changes in target heading and target speed into servo motor target angle and propulsion motor target speed through a manipulation response model, and generates navigation control commands.

[0106] During navigation, the system continuously collects actual position, heading, and speed data, aligns the actual motion state with the target motion state sequence, and determines the tracking deviation of the actual trajectory relative to the target obstacle avoidance path. Based on the tracking deviation, the path tracking parameters are corrected, and navigation control commands are updated under constraints of rudder angle and motor speed changes. Once the obstacle remains safe and the unmanned equipment has passed the nearest encounter position, a continuous curvature connection path is generated based on the actual trajectory and the global route, allowing the unmanned equipment to smoothly return to the global route.

[0107] Through the cyclic execution of S201 to S206, visual semantic information and point cloud spatial information can be fused in layers under a unified spatiotemporal reference. Water surface environmental interference can be adaptively filtered based on cross-modal and temporal evidence. The future motion state of obstacles can be applied to local path planning in advance, and the execution instructions can be continuously corrected through trajectory feedback, thereby forming a real-time obstacle avoidance closed loop in which perception, prediction, planning, execution and correction are interconnected.

[0108] It should be understood that the sequence number of each step in the above method embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0109] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although the technical solutions of this application have been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A multi-modal hierarchical fusion real-time obstacle avoidance path generation system for unmanned surface equipment, characterized in that, include: The acquisition and registration module is used to simultaneously acquire visual data, laser point cloud data and navigation status, and register the visual data and laser point cloud data according to the calibration relationship and acquisition sequence to obtain registration data; The layered fusion module is used to establish a depth mapping relationship between the image and the point cloud based on the registration data, fuse the target features of the image and the point cloud according to the depth mapping relationship to obtain fused target features, and generate candidate obstacle objects with semantic attributes and spatial state associations based on the fused target features. The interference filtering module is used to determine the water surface interference state based on the cross-modal consistent state and time-series stable state of the candidate obstacle objects, and to filter out valid obstacle objects. The risk prediction module is used to determine the collision risk based on the relative motion relationship between the effective obstacle object and the navigation state, and to generate a predicted motion trajectory based on the temporal state of the effective obstacle object. The path generation module is used to determine a local safe and feasible region under global flight path and motion constraints based on the collision risk and the predicted motion trajectory, and to generate a continuous target obstacle avoidance path in the local safe and feasible region. The execution correction module is used to convert the target obstacle avoidance path into navigation control commands and correct the navigation control commands based on the deviation between the actual trajectory and the target obstacle avoidance path.

2. The system according to claim 1, characterized in that, The process of simultaneously acquiring visual data, laser point cloud data, and navigation status, and registering the visual data and laser point cloud data according to calibration relationships and acquisition timing to obtain registration data includes: The visual acquisition unit and the lidar are jointly calibrated to determine the spatial transformation relationship between the visual coordinate system and the point cloud coordinate system. Based on a unified clock to mark the acquisition time of the visual data and the laser point cloud data, candidate frame pairs are formed according to the time matching status. The acquisition time difference of the candidate frame pairs is compensated based on the navigation status to determine the target frame pairs that meet the timing consistency condition; According to the spatial transformation relationship, the laser point cloud in the target frame pair is projected onto the image plane to establish the spatial association between the image region and the point cloud data, and the registration data is generated.

3. The system according to claim 1, characterized in that, The step of establishing a depth mapping relationship between the image and the point cloud based on the registration data, fusing target features of the image and the point cloud according to the depth mapping relationship to obtain fused target features, and generating candidate obstacle objects with semantic attributes and spatial state associations based on the fused target features includes: The point cloud projection units in the registration data are mapped to image regions, and depth values ​​are associated with each image region to establish the depth mapping relationship. The visual target features and the spatial target features of the corresponding point cloud of the image region are extracted respectively; The feature contribution is determined based on the data quality and cross-modal matching status of the two types of target features. The two types of target features are then cross-fused and feature-filtered to obtain the fused target features. Semantic recognition is performed on the fused target features, the spatial state is determined according to the deep mapping relationship, the semantic attributes and the spatial state are associated with the same target, and the candidate obstacle object is generated.

4. The system according to claim 1, characterized in that, The step of determining the water surface disturbance state based on the cross-modal consistent state and time-series stable state of the candidate obstacle objects, and screening valid obstacle objects, includes: Determine the target support state of each candidate obstacle object in visual data and laser point cloud data, and generate the cross-modal consistent state based on the matching degree of the two types of target support states; The temporally stable state is generated by associating the same candidate obstacle object in consecutive frames based on the continuous changes in the corresponding spatial position and target features. The candidate obstacle object is matched with the water surface disturbance feature model, and the water surface disturbance state is determined by combining the cross-modal consistent state and the temporal stable state. Filter out candidate obstacle objects that meet the interference determination conditions, and determine the remaining candidate obstacle objects as the valid obstacle objects.

5. The system according to claim 4, characterized in that, The step of matching the candidate obstacle object with the water surface disturbance feature model and determining the water surface disturbance state by combining the cross-modal consistent state and the temporal stable state includes: The interference feature matching degree is determined based on the target features of the candidate obstacle object, and the true target support degree is determined based on the cross-modal consistent state and the temporal stable state. The interference determination boundary is adjusted according to the quality status of the visual data and the laser point cloud data. The interference feature matching degree and the real target support degree are matched with the adjusted interference determination boundary to determine the water surface interference state. When the candidate obstacle object contains local regions with different water surface disturbance states, the local regions are divided according to the continuity relationship of the feature space, and the local regions that meet the support conditions of the real target are retained.

6. The system according to claim 1, characterized in that, The step of determining the collision risk based on the relative motion relationship between the effective obstacle object and the navigation state, and generating a predicted trajectory based on the temporal state of the effective obstacle object, includes: For the same valid obstacle object in consecutive frames, estimate its current motion state according to the temporal changes of its corresponding spatial state; The current motion state and the navigation state are converted to a unified motion coordinate system to determine the approach state and predicted encounter state of the effective obstacle object relative to the unmanned equipment on the water surface. The safety boundary is determined based on the semantic attributes and spatial state of the effective obstacle object, and the predicted encounter state is matched with the safety boundary to determine the collision risk. The current motion state is recursively corrected based on continuous frame observations, and the future spatial state of the effective obstacle object is deduced within the target prediction time domain to generate the predicted motion trajectory.

7. The system according to claim 1, characterized in that, The step of determining a local safe and feasible region under global flight path and motion constraints based on the collision risk and the predicted motion trajectory, and generating a continuous target obstacle avoidance path within the local safe and feasible region, includes: The planning time domain and safety interval are determined based on the collision risk, and a dynamic obstacle occupancy domain is constructed by combining the predicted motion trajectory. A set of reachable motion states is generated based on the current navigation state and the motion constraints, and local trajectory deduction is performed on the motion states in the set of reachable motion states to obtain candidate navigation trajectories; Filter out candidate navigation trajectories that overlap with the dynamic obstacle-occupied domain in time and space, and form the local safe and feasible domain based on the remaining candidate navigation trajectories; The evaluation weights of global flight path consistency constraints and path continuity constraints are adjusted according to the collision risk, the target trajectory is determined from the local safe and feasible domain and continuity correction is performed to generate the target obstacle avoidance path.

8. The system according to claim 7, characterized in that, Constructing the dynamic obstacle occupancy domain and determining the target trajectory from the local safe and feasible domain includes: Based on the predicted uncertainty range of the predicted motion trajectory at each prediction time and the collision risk, the spatial range of the corresponding effective obstacle object is expanded in a time-division manner to construct the dynamic obstacle occupancy domain; The trajectory evaluation status is determined based on the safety status of the candidate navigation trajectory relative to the dynamic obstacle occupancy domain, the consistency status relative to the global route, and the continuity status relative to the target trajectory of the previous planning cycle. The contribution of each state in the trajectory evaluation state is adjusted according to the collision risk to determine the candidate target trajectory; When the difference between the candidate target trajectory and the target trajectory of the previous planning period exceeds the continuity threshold, the candidate target trajectory is determined as the target trajectory if the target trajectory of the previous planning period does not meet the safety conditions or the trajectory evaluation status meets the switching conditions.

9. The system according to claim 1, characterized in that, The step of converting the target obstacle avoidance path into navigation control commands and correcting the navigation control commands based on the deviation between the actual trajectory and the target obstacle avoidance path includes: The target obstacle avoidance path is discretized according to the control cycle to generate a target motion state sequence corresponding to the path position; Based on the tracking relationship between the navigation state and the target motion state sequence, the target control state of the actuator is determined through motion control mapping, and the navigation control command is generated. The actual motion state collected is aligned with the target motion state sequence to determine the tracking deviation of the actual trajectory relative to the target obstacle avoidance path; The path tracking parameters are corrected based on the tracking deviation, and the navigation control commands are updated under the constraints of actuator changes; When the obstacle avoidance completion conditions are met, a connection path is generated based on the actual flight path and the global route, and the navigation control command is updated according to the connection path.

10. A method for real-time obstacle avoidance path generation of unmanned surface equipment based on a multi-mode hierarchical fusion system as described in any one of claims 1 to 9, characterized in that, include: Simultaneously acquire visual data, laser point cloud data, and navigation status; register the visual data and laser point cloud data according to the calibration relationship and acquisition sequence to obtain registration data; A depth mapping relationship between the image and the point cloud is established based on the registration data. The target features of the image and the point cloud are fused according to the depth mapping relationship to obtain fused target features. Candidate obstacle objects with semantic attributes and spatial state associations are generated based on the fused target features. The water surface disturbance state is determined based on the cross-modal consistent state and time-series stable state of the candidate obstacle objects, and the effective obstacle objects are screened. The collision risk is determined based on the relative motion relationship between the effective obstacle object and the navigation state, and a predicted motion trajectory is generated based on the temporal state of the effective obstacle object. Based on the collision risk and the predicted motion trajectory, a local safe and feasible region is determined under the global flight path and motion constraints, and a continuous target obstacle avoidance path is generated in the local safe and feasible region. The target obstacle avoidance path is converted into navigation control commands, the actual trajectory is obtained, and the navigation control commands are corrected based on the deviation between the actual trajectory and the target obstacle avoidance path.

Images