Intelligent control method and system for unmanned ship based on visual inspection

Abstract

The invention discloses an intelligent control method and system for an unmanned ship based on visual inspection, and relates to the technical field of visual inspection. A plurality of object areas are determined based on image recognition of a cruise dynamic graph; and determining corresponding object features according to the region position of each object region, the corresponding region form and the image recognition model, and determining each piece of cruise early warning information based on the relative position of the object features and the unmanned ship and the corresponding feature information. Determining an optimal cruise route of the unmanned ship according to the feature position of the target object feature, the cruise path of the unmanned ship and the cruise early warning information of the multiple non-target object features, and determining multiple target cruise items based on the identification of the target cruise event, and an intelligent control system of the unmanned ship is determined according to the item contents of the multiple target cruise items, the corresponding priorities and the work task table of the unmanned ship, and accurate interaction of the unmanned ship relative to the target object features is ensured.

Description

Technical Field

[0001] This invention relates to the technical field of visual inspection, and in particular to an intelligent control method and system for unmanned vessels based on visual inspection. Background Technology

[0002] With the development of technology, unmanned surface vessels (USVs) are gradually being applied to the field of object navigation. They navigate relative to the features of target objects. Under autonomous control, USVs perform dynamic inspections and move from their current position to the features of the target object. In existing technologies, USVs collect preset navigation paths and navigate dynamically along these paths, ignoring the influence of non-target object features and corresponding navigation warning information. This affects the accuracy of the USV's optimal navigation route and leads to the inaccuracy of the USV's multi-dimensional work content relative to the features of the target object. Summary of the Invention

[0003] The purpose of this invention is to overcome the shortcomings of the prior art. This invention provides an intelligent control method and system for unmanned vessels based on visual detection.

[0004] This invention provides an intelligent control method for unmanned surface vessels based on vision detection, comprising: When the unmanned vessel is cruising on the water, the camera installed on the unmanned vessel performs visual detection of the surrounding environment to collect multiple images of the surrounding environment. Based on the multiple images of the surrounding environment and the overall shape of the unmanned vessel, the corresponding cruise dynamic map is determined. Based on the image recognition of the cruise dynamic map, multiple object regions are identified. According to the regional location, corresponding regional shape and image recognition model of each object region, the corresponding object features are determined. Based on the relative position of the object features and the unmanned vessel and the corresponding feature information, each cruise warning information is determined. The cruise warning information not only includes the warning type and risk level, but also specifies the target ID, key parameters and specific recommended actions. If multiple object features are respectively target object features and multiple non-target object features, the optimal cruise route of the unmanned vessel is determined based on the feature position of the target object features, the cruise path of the unmanned vessel, and the cruise warning information of the multiple non-target object features; the optimal cruise route includes a complete path with detailed waypoints and speed commands. The unmanned vessel cruises along the optimal cruise route, and the target cruise event of the unmanned vessel is determined based on the dynamic information of the unmanned vessel's attitude parameter combination, the corresponding current position and the characteristics of the target object. Based on the identification of the target cruise event, multiple target cruise projects are determined. According to the project content, corresponding priority and unmanned vessel's work task list of multiple target cruise projects, the intelligent control system of the unmanned vessel is determined, and the multi-dimensional work content of the unmanned vessel relative to the characteristics of the target object is determined.

[0005] This invention provides an intelligent control system for an unmanned surface vessel (USV) based on vision detection, which is applied to the aforementioned intelligent control method for USVs based on vision detection.

[0006] Compared with the prior art, the beneficial effects of the present invention are: (1) When the unmanned vessel is in a state of surface cruise, the camera configured on the unmanned vessel performs visual detection of the surrounding environment to collect multiple surrounding environment images. Based on the multiple surrounding environment images and the overall shape of the unmanned vessel, the corresponding cruise dynamic map is determined. Based on the image recognition of the cruise dynamic map, multiple object regions are determined. Based on the regional position of each object region, the corresponding regional shape and the image recognition model, the corresponding object features are determined. Based on the relative position of the object features and the unmanned vessel and the corresponding feature information, each cruise warning information is determined. The cruise dynamic map is introduced and the image recognition of the cruise dynamic map is realized, which improves the accuracy of each cruise warning information.

[0007] (2) If multiple object features are target object features and multiple non-target object features respectively, the optimal cruise route of the unmanned vessel is determined based on the feature position of the target object features, the cruise path of the unmanned vessel and the cruise warning information of multiple non-target object features. This further controls the target object features and multiple non-target object features, takes into account the feature position of the target object features, the cruise path of the unmanned vessel and the cruise warning information of multiple non-target object features, and improves the accuracy of the optimal cruise route of the unmanned vessel.

[0008] (3) The unmanned vessel cruises along the optimal cruise route. The target cruise event of the unmanned vessel is determined based on the combination of attitude parameters of the unmanned vessel, the corresponding current position and the dynamic information of the target object features. Based on the identification of the target cruise event, multiple target cruise items are determined. The intelligent control system of the unmanned vessel is determined based on the item content, corresponding priority and the work task table of the unmanned vessel. The multi-dimensional work content of the unmanned vessel relative to the target object features is determined. The overall consideration of the item content, corresponding priority and work task table of multiple target cruise items is realized, which improves the accuracy of the intelligent control system of the unmanned vessel. The multi-dimensional work content of the unmanned vessel relative to the target object features is determined, which ensures the accurate interaction of the unmanned vessel relative to the target object features. Attached Figure Description

[0009] Figure 1 This is a flowchart illustrating the intelligent control method for an unmanned vessel based on visual detection in an embodiment of the present invention. Figure 2 This is a flowchart illustrating step S11 in the intelligent control method for an unmanned vessel based on visual detection in an embodiment of the present invention. Figure 3 This is a flowchart illustrating step S12 in the intelligent control method for an unmanned vessel based on visual detection in an embodiment of the present invention. Figure 4 This is a flowchart illustrating step S13 in the intelligent control method for an unmanned vessel based on visual detection in an embodiment of the present invention. Figure 5 This is a flowchart illustrating step S14 in the intelligent control method for an unmanned vessel based on visual detection in this embodiment of the invention. Figure 6 This is a flowchart illustrating step S15 in the intelligent control method for an unmanned vessel based on visual detection in an embodiment of the present invention. Figure 7 This is a schematic diagram of the structural composition of the intelligent control system for an unmanned vessel based on vision detection in an embodiment of the present invention. Detailed Implementation

[0010] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0011] Please see Figures 1 to 7 An intelligent control method for unmanned surface vessels (USVs) based on vision detection is applied to vision detection scenarios. The intelligent control method for USVs based on vision detection includes: Step S11: When the unmanned vessel is in the water surface cruising state, the camera configured on the unmanned vessel performs visual detection of the surrounding environment to collect multiple surrounding environment images, and determines the corresponding cruising dynamic map based on the multiple surrounding environment images and the overall shape of the unmanned vessel. Step S12: Based on the image recognition of the cruise dynamic map, determine multiple object regions, determine the corresponding object features according to the region location, corresponding region shape and image recognition model of each object region, and determine each cruise warning information based on the relative position of the object features and the unmanned vessel and the corresponding feature information. Step S13: If multiple object features are target object features and multiple non-target object features, determine the optimal cruise route of the unmanned vessel based on the feature position of the target object features, the cruise path of the unmanned vessel, and the cruise warning information of the multiple non-target object features. Step S14: The unmanned vessel cruises along the optimal cruise route, and the target cruise event of the unmanned vessel is determined based on the dynamic information of the unmanned vessel's attitude parameter combination, the corresponding current position, and the characteristics of the target object. Step S15: Based on the identification of the target cruise event, determine multiple target cruise items, determine the intelligent control system of the unmanned vessel according to the project content, corresponding priority and the unmanned vessel's work task table of the multiple target cruise items, and determine the multi-dimensional work content of the unmanned vessel relative to the characteristics of the target object.

[0012] refer to Figure 2 In step S11, the specific steps are as follows: S111: The unmanned vessel floats on the water surface and cruises relative to the water surface. Based on the detection of the unmanned vessel, the corresponding cruise information is determined. Based on the identification of the cruise information, multiple key detection areas are determined. Based on the regional location of each key detection area and the camera configured on the unmanned vessel, the visual detection mode of the camera is determined, and the camera is triggered to perform visual detection of the surrounding environment along the visual detection mode. S112: Determine multiple surrounding environment images based on the camera's visual detection of the surrounding environment, and mark the corresponding image content. Determine the corresponding cruise dynamic map based on the multiple surrounding environment images, the corresponding image content, and the overall shape of the unmanned vessel.

[0013] In the embodiments of this application, the unmanned surface vessel floats on the water surface and performs surface patrol relative to the water surface. Based on the detection of the unmanned surface vessel, corresponding patrol information is determined. Based on the identification of the patrol information, multiple key detection areas are determined. Based on the regional location of each key detection area and the camera configured on the unmanned surface vessel, the visual detection mode of the camera is determined. The camera is triggered to perform visual detection of the surrounding environment along the visual detection mode. This approach takes into account the regional location of each key detection area and the overall consideration of the camera configured on the unmanned surface vessel, ensuring the accuracy of the camera's visual detection mode.

[0014] At this point, the unmanned vessel floats on the water and cruises relative to the water surface. State perception is not just about obtaining simple coordinates, but about constructing a six-degree-of-freedom state vector containing precise position, velocity, and attitude through the fusion of sensors such as RTK-GPS and IMU. This is the benchmark for all subsequent calculations and control. Task information parsing is even more critical. It transforms a macroscopic "inspection" command into a structured, executable data packet, which includes the global path, the digital twin model of the target, historical defect data, and specific detection procedures.

[0015] Through a spatiotemporal indexing mechanism, the system dynamically selects the "critical detection areas" most relevant to the current position and trajectory of the unmanned vessel from the global tasks. This is equivalent to centrally allocating attention resources. The system dynamically prioritizes these critical areas based on the urgency of the task, the importance of the area, and historical data (such as the defect recurrence rate). This prioritization mechanism ensures that the unmanned vessel can always prioritize the areas with the most critical problems, greatly improving the efficiency and targeting of the inspection.

[0016] The system has a built-in parameterized visual pattern library. Each pattern is a sensor configuration scheme optimized for a specific task (such as wide-area search, fine imaging, and 3D scanning). The decision model integrates the input of "key detection area attributes" (such as material, size, and detection type) and "the relative pose of the unmanned vessel and the area" (such as distance and angle). Through a rule engine or machine learning model, it selects the optimal solution from the pattern library. This adaptive decision-making capability enables the unmanned vessel to always work with the best sensor configuration under different distances, different lighting conditions, and different task requirements.

[0017] The control system sends a precise, coordinated sequence of commands to the visual perception unit (camera, gimbal, light source, etc.), rather than a simple "take a picture" command. These commands specify the gimbal's locking mode, the camera's resolution and exposure, the light source's brightness, etc. After completing the configuration, the execution unit returns a confirmation signal, forming a closed-loop control to ensure that the commands are executed accurately. Once the mode is successfully triggered, the data stream begins to be continuously generated according to the predetermined parameters, and each frame of data is accompanied by a precise spatiotemporal label.

[0018] Specifically, the unmanned vessel departs autonomously from the dock, and its RTK-GPS and IMU systems output its precise position on the water surface and its stable navigation attitude in real time. At the same time, the onboard computer loads a high-precision 3D model of the No. 3 lock station and parses the core objectives from the mission instructions: to perform high-resolution imaging of the upstream surfaces of lock piers A, B, and C, and to perform a 3D morphological scan of the underwater part of lock pier B. At this moment, the unmanned vessel clearly knows where it is and what specific task it is about to complete.

[0019] As the unmanned vessel navigated, it was 200 meters away from dam A, while dam B was even further away. Through spatiotemporal queries, the system immediately identified the "water-facing side of dam A" as the next critical area to enter the work area. At the same time, the system retrieved historical database information and found that dam A had a 0.5mm crack in a report from last year. Based on this, the system automatically set the detection priority of the "water-facing side of dam A" to "high" and specifically marked that the location of the crack needed to be carefully re-examined.

[0020] When the unmanned surface vessel (USV) continued to approach to within 150 meters of gate pier A, the system determined that it was still in the approach phase but needed to begin locking onto the target. Instead of directly activating the highest power consumption fine imaging mode, it chose a variant of "wide-area surveillance": activating the gimbal and using a telephoto lens to continuously track the outline of gate pier A at a lower resolution, preparing for subsequent precision operations. When the USV smoothly entered the designated 20-meter working area, the system triggered another decision: the distance had met the fine imaging requirements, and it immediately switched to "fine imaging mode." The camera switched to a high-resolution industrial camera, the frame rate decreased, the LED fill light was turned on, and the gimbal adjusted the viewing angle to be perpendicular to the gate pier surface to ensure image quality.

[0021] Upon entering the 20-meter working area, the main control computer immediately issued a series of precise instructions to the vision perception unit: commanding the gimbal to lock onto gate pier A, switch to the high-resolution industrial camera, set the resolution, frame rate, and exposure parameters, and adjust the brightness of the LED fill light to 80%; after receiving the "OK" confirmation signal from the vision unit, the unmanned vessel's camera began to collect data on the surface of gate pier A, which was stably locked by the gimbal, at a rate of 2 frames per second; each frame of the image was precisely stamped with RTK position, IMU attitude, and UTC timestamp, forming a high-quality data stream that was continuously sent to the subsequent analysis module.

[0022] Furthermore, multiple surrounding environment images are determined based on the camera's visual detection of the surrounding environment, and the corresponding image content is marked. The corresponding cruise dynamic map is determined based on the multiple surrounding environment images, the corresponding image content, and the overall shape of the unmanned vessel. This comprehensive consideration of multiple surrounding environment images, corresponding image content, and the overall shape of the unmanned vessel ensures the accuracy of the corresponding cruise dynamic map.

[0023] At this point, the system uses hardware triggering to ensure that multiple cameras are exposed synchronously in an instant, and immediately encapsulates each frame of image into a standardized data packet. This data packet not only contains the image itself, but more importantly, it includes a timestamp with microsecond precision, a spatial stamp (optical center coordinates) with centimeter precision, a precise attitude stamp, and a sensor ID. This approach completely transforms a static image into a data point that is precisely anchored in a four-dimensional spatiotemporal coordinate system, providing an indispensable alignment benchmark for subsequent multi-source information fusion.

[0024] The system utilizes deep learning models deployed on embedded GPUs to perform multi-level information mining on the original pixel matrix; object detection models (such as YOLO) can identify and outline specific object instances in the image; semantic segmentation models (such as U-Net) can classify each pixel and finely distinguish different material regions such as concrete, water, and rust; at the same time, specific algorithms for inspection tasks (such as crack detection models) will specifically search for specific patterns; all these extracted semantic information and geometric features are used as metadata and reattached to the data packet generated in the first step, transforming it from a bunch of pixels into a detailed visual content.

[0025] A vision-inertial tightly coupled SLAM (VIO) algorithm is introduced, which takes high-frequency IMU data (providing motion prediction) and low-frequency image data (providing absolute position constraints) as input. Through methods such as extended Kalman filtering or graph optimization, the advantages of the two sensors are optimally fused and their errors are mutually corrected. While accurately estimating the unmanned vessel's own trajectory, the algorithm calculates the 3D coordinates of the stably tracked feature points through triangulation, thereby constructing a 3D point cloud map of the local environment. The semantic information extracted in the second step is projected onto this 3D map, forming a multi-layered "cruising dynamic map" that includes not only geometric structure, but also object categories, defect markers, and even dynamic object trajectories.

[0026] Specifically, the unmanned vessel navigates smoothly at a distance of 15 meters from the water-facing surface of gate pier A. At T=10:35:01.500, the main control computer sends a hardware trigger signal, and the wide-angle and telephoto cameras are exposed simultaneously. The system immediately constructs a data packet for the image IMG_789.jpg acquired by the telephoto camera. This data packet not only contains an 8192x5460 resolution image of the gate pier surface, but also accurately records the UTC time of the acquisition, the RTK-GPS coordinates of the camera's optical center, the attitude angle measured by the IMU, and the sensor ID "Cam_Tele". This data packet is immediately sent to the processing queue.

[0027] After receiving the data packet IMG_789.jpg, the inference engine immediately began working; the object detection model identified a "concrete gate pier" object in the center of the image with a confidence of 99.5%; the semantic segmentation model generated a mask image, accurately dividing the pixels into concrete, water stains, and shadow areas; and the specialized crack detection algorithm marked a "suspected crack" with a length of 80 pixels and a width of 2 pixels at the image coordinates (4000, 3000); all this information was packaged back into the data packet, and now the image comes with a detailed description.

[0028] The unmanned surface vessel (USV) continued its journey along pier A, with its VIO system operating at full speed. It continuously tracked thousands of concrete spot features on the pier's surface. When the IMU reported a slight rightward acceleration of the hull due to the water flow, the vision system verified and precisely quantified this motion by observing the overall leftward shift of all feature points on the image. After EKF fusion, the system output a trajectory that was smoother and more accurate than using GPS or IMU alone. Meanwhile, the "suspected crack" detected in IMG_789.jpg was converted into a three-dimensional world coordinate system by combining its image coordinates with the camera pose. This was then permanently added to the "cruise animation" as a 3D point labeled "crack." In the USV's autonomous control system, S112 outputs a real-time, semantically labeled 3D point cloud world.

[0029] refer to Figure 3 In step S12, the specific steps are as follows: S121: Perform image recognition on the cruise dynamic image, and determine multiple regions during the recognition process. Based on the image recognition of each region, determine the corresponding object region to identify multiple object regions. Based on the detection of each object region, determine the region position and corresponding region shape of each object region. S122: Based on the detection of the unmanned vessel, a model database is determined. Based on the matching of the model database and the cruise dynamic map, the corresponding image recognition model is determined. Based on the regional position of each object area, the corresponding regional shape, and the image recognition model, the corresponding object features are determined. At the same time, the relative position of the object feature and the unmanned vessel is marked. Based on the relative position, the feature information of the object feature, and the corresponding feature shape, each cruise warning information is determined. In the embodiments of this application, image recognition is performed on the cruise dynamic image, and multiple regions are determined during the recognition process. Based on the image recognition of each region, the corresponding object region is determined, thereby determining multiple object regions. Based on the detection of each object region, the regional position and corresponding regional shape of each object region are determined, which is compatible with the overall consideration of the detection of each object region and ensures the accuracy of the regional position and corresponding regional shape of each object region.

[0030] At this point, the system analyzes the input "cruise dynamic map," which is a collection of 3D point clouds, semantic labels, and dynamic information. The system performs semantic-driven filtering; for example, in an inspection task, all point clouds marked as "water bodies" can be temporarily ignored to focus on potential structures. The core geometric clustering algorithm (such as Euclidean clustering) begins to work, which aggregates point clouds that are close to each other in space into a cluster, thereby forming one or more spatially separated "object region" proposals. The output of this step is not the final identified object, but a series of candidate regions composed of point clouds that need to be further analyzed.

[0031] The system extracts two main categories of key features for each candidate region: location and morphology. Location is typically determined by calculating the centroid of all point clouds in the region, representing its geometric center. Morphology is quantified in a more refined manner, with the core being the calculation of its minimum bounding box (especially the orientation-aligned OBB) to obtain the object's precise length, width, height, and orientation in space. In addition, the system estimates its surface area and volume, and determines its primary geometric orientation through principal component analysis (PCA). A blurry point cloud is transformed into a set of precise, standardized geometric parameters.

[0032] Specifically, the VIO system of the unmanned vessel outputs the current "cruising dynamic map," which contains millions of point clouds. The system performs semantic filtering, removing 95% of the "water body" point clouds that constitute the water surface environment from the current processing flow, significantly reducing the computational load. For the remaining point clouds, the system applies the Euclidean clustering algorithm. Since the "concrete" point cloud clusters on the left side of the hull are spatially continuous, they are successfully aggregated into a large region, which the system names Region_01. At the same time, about 100 meters to the right front of the hull, a smaller moving point cloud cluster marked as "unknown" is aggregated into another independent region, named Region_02, due to its small internal point spacing and large distance from Region_01. Thus, S121 outputs two clear "object region" proposals: one is a huge static structure, and the other is a small moving object.

[0033] Upon receiving Region_01 and Region_02, the system immediately begins geometric quantization. For Region_01 (i.e., the gate pier), the system calculates the centroid of all its point clouds, obtaining a precise three-dimensional coordinate (x1, y1, z1) as its position. The system calculates its orientation-aligned minimum bounding box (OBB), determining its dimensions to be 15 meters long, 3 meters wide, and 20 meters high, and accurately determines that its long side is parallel to the waterway. For Region_02 (i.e., another vessel), the system similarly calculates its centroid coordinates (x2, y2, z2), and through OBB analysis, determines its dimensions to be 5 meters long, 2 meters wide, and 1.5 meters high, and that it is moving in a northwest direction.

[0034] Furthermore, a model database is established based on the detection of the unmanned vessel. A corresponding image recognition model is determined by matching this model database with the cruise dynamic map. The corresponding object features are determined based on the regional location, corresponding regional shape, and image recognition model of each object region. Simultaneously, the relative position of the object feature and the unmanned vessel is marked. Each cruise warning message is determined based on this relative position, the feature information of the object feature, and the corresponding feature shape. This comprehensive consideration of relative position, feature information of the object feature, and corresponding feature shape ensures the accuracy of each cruise warning message. Additionally, the cruise dynamic map is introduced, and image recognition of the cruise dynamic map is implemented, further improving the accuracy of each cruise warning message.

[0035] At this time, the system maintains a modular model database containing specialized models for different tasks, such as those specifically for recognizing hydraulic structures, vessels on the water, and floating objects. An intelligent decision engine takes the geometric features (size, speed, shape) of the object region output by S121 as input, and matches and loads the most suitable recognition model for each region through preset rules or learned strategies. This on-demand calling mechanism ensures that each target can receive the most professional analysis, greatly improving the accuracy of recognition and computational efficiency.

[0036] The loaded specialized models (such as CNN or PointNet++) perform deep reasoning on the corresponding object regions, and their output far exceeds simple classification labels. For hydraulic structures, the models can perform pixel-level or point-level semantic segmentation, accurately identify specific defects such as cracks, spalling, and seepage, and quantify their attributes such as length, width, and area. For dynamic objects, the models can not only identify their categories (such as "fishing boats"), but also calculate their colors, features, and even kinematic parameters (speed, heading). All this information is integrated into a structured "object feature" object.

[0037] The system transforms the world coordinates of all object features extracted in the previous step with the real-time pose of the unmanned vessel, and calculates the precise distance, azimuth, and pitch angle of each feature relative to the unmanned vessel. A multi-factor risk assessment algorithm comprehensively considers relative position, motion information (such as CPA / TCPA), object attributes, and mission context to dynamically rate the risk of each target. The system generates structured "cruise warning information", which not only includes the warning type and risk level, but also specifies the target ID, key parameters, and specific recommended actions (such as "path replanning" or "preparing for fine scanning").

[0038] Specifically, the unmanned surface vessel's system receives two object regions output by S121. For Region_01, whose shape is "huge, static, and vertical," the decision engine immediately matches the rules, determines that it is a hydraulic structure, and loads Model_Water_Structure_v3.2, which is specifically used for hydraulic structure recognition, from the model database. For Region_02, whose shape is "small, mobile, and elongated," the system loads Model_Marine_Vehicle_v2.1, which is specifically used for recognizing water vehicles, in parallel. The unmanned surface vessel ensures that it observes different targets with the most professional "eyes."

[0039] Model_Water_Structure_v3.2 performed a deep analysis on Region_01, ultimately identifying it as “No. 3 Gate Pier A”, and precisely identifying a 25cm long “crack” and a 0.1㎡ “concrete spalling area” on its surface. This information was integrated into the feature object of Region_01. At the same time, Model_Marine_Vehicle_v2.1 analyzed Region_02, identifying it as a “blue fishing boat”, and calculated that it was moving northwest at a speed of 2 knots. This information was also integrated into the feature object of Region_02.

[0040] The system begins risk assessment; for the gate pier, it calculates that the distance between the gate pier and the unmanned vessel is 80 meters and the azimuth angle is 90 degrees; since this is the mission target and has entered the work area, the system generates an information-level warning: {Warning Type: "Target Acquisition", Target ID: "Pier_A", Suggested Action: "Prepare to switch to fine scan mode"}, and generates a recording instruction for the discovered crack; for the fishing vessel, the system calculates its trajectory and finds that the CPA (closest encounter distance) is only 20 meters and the TCPA (time to nearest encounter point) is 60 seconds; since the CPA is less than the safety threshold of 30 meters, the system immediately generates a high-priority warning: {Warning Type: "Collision Risk", Target ID: "Fishing_Boat_01", Risk Level: "High", CPA: 20m, TCPA: 60s, Suggested Action: "Immediately execute path replanning"}.

[0041] refer to Figure 4 In step S13, the specific steps are as follows: S131: Identify multiple object features and mark target object features and multiple non-target object features during the identification process. The target object feature serves as the final interaction object of the unmanned vessel. Based on the early warning detection of each non-target object feature, determine the corresponding cruise early warning information to mark the cruise early warning information of multiple non-target object features. S132: Collect the cruise path of the unmanned vessel, determine the first set of cruise routes based on the cruise path of the unmanned vessel and the feature position of the target object, and determine the second set of cruise routes based on the cruise path of the unmanned vessel and the cruise warning information of multiple non-target object features. S133: Based on the matching of the first group of cruise routes and the second group of cruise routes, determine each priority cruise route and mark the route cruise coefficient of each priority cruise route to determine the optimal cruise route for the unmanned vessel.

[0042] In the embodiments of this application, multiple object features are identified, and target object features and multiple non-target object features are marked during the identification process. The target object features serve as the final interaction object of the unmanned vessel. Based on the early warning detection of each non-target object feature, the corresponding cruise early warning information is determined to mark the cruise early warning information of multiple non-target object features. This approach takes into account the overall consideration of early warning detection of each non-target object feature and ensures the accuracy of the corresponding cruise early warning information.

[0043] At this point, the system loads and parses the current task context to identify the final interaction object for this patrol. A logical classification engine compares all object features output by S12 with the task target. Any object that matches the task instructions is immediately marked as a "target object feature," becoming the core of all subsequent planning. All other objects are classified as "non-target object features," and are usually further subdivided into dynamic obstacles, static obstacles, or environmental noise based on their dynamic attributes for differentiated management.

[0044] Once the characteristics of the target object are determined, the system locks its position and state in the "cruise dynamic map" as the core reference frame for all subsequent calculations, making the behavior of the entire unmanned vessel revolve around this "gravitational center". This locking behavior usually triggers the switching of the mission state machine, such as switching from the "cruise search" state to the "target approach and operation" state. This switching will bring about a series of adjustments to control parameters, such as reducing the cruise speed, improving the attitude control accuracy, activating specific sensors, etc., to prepare for the subsequent interactive operation. At the same time, the key attributes of the target (such as size and material) will be solidified as direct inputs for generating specific operation paths.

[0045] The system employs a multi-factor risk assessment model that comprehensively considers the kinematic parameters (velocity, CPA, TCPA), spatial parameters (distance, size), task context (current speed) and environmental factors of each non-target object. Through weighted calculation or fuzzy logic, the model assigns a quantified risk level (such as "high", "medium", "low") to each object. The system generates a standardized "cruise warning message" for each non-target object, which includes the confirmed and quantified risk level, key data, and suggested actions. This message serves as the direct basis for generating avoidance routes in subsequent path planning.

[0046] Specifically, the system employs a multi-factor risk assessment model that comprehensively considers the kinematic parameters (speed, CPA, TCPA), spatial parameters (distance, size), task context (current speed) and environmental factors of each non-target object. Through weighted calculation or fuzzy logic, the model assigns a quantified risk level (such as "high", "medium", or "low") to each object. The system generates a standardized "cruise warning message" for each non-target object, which includes the confirmed and quantified risk level, key data, and suggested actions. This message serves as the direct basis for generating avoidance routes in subsequent path planning.

[0047] The unmanned surface vessel (USV) locks "Gate A" as the final interactive object, and its three-dimensional coordinates and dimensions become the center around which the path planner must revolve. The system's mission state switches from "Transit to Area" to "Inspection Approach". The USV's cruising speed automatically decreases from 5 knots to 2 knots, and the attitude controller also switches to high-precision mode to ensure the stability of subsequent imaging. The dimensions of Gate A (15 meters long and 3 meters wide) and the required detection distance (15 meters) are extracted.

[0048] The system began quantifying the threat; for the fishing boat, S12 reported a CPA of 20 meters and a TCPA of 60 seconds; since the unmanned vessel was currently operating at low speed, the safety threshold was a CPA greater than 30 meters, and the 20-meter distance constituted a significant threat; the model rated its risk level as "high" and generated a final warning: {ID: "Fishing_Boat_01", Risk Level: "High", Recommended Action: "Forced Avoidance"}; for the floating tree branch, although it was only 5 meters away from the route, due to its small size and lack of speed, the risk level was rated as "low", and the generated warning suggestion was only "Schedule fine-tuning".

[0049] Furthermore, the unmanned surface vessel's (USV) cruise path is collected. Based on the USV's cruise path and the characteristic positions of target objects, a first set of cruise routes is determined. Based on the USV's cruise path and cruise warning information of multiple non-target object characteristics, a second set of cruise routes is determined. This comprehensive consideration of the USV's cruise path and cruise warning information of multiple non-target object characteristics ensures the accuracy of the second set of cruise routes.

[0050] At this point, the system does not generate this path in real time, but loads a global path pre-set by the mission planning system before the mission begins. The cruise path of the unmanned vessel usually consists of a series of sparse waypoints, which define the general flow of the mission, such as from the starting point to the work area and then to the return point. After the system collects this path, it will analyze it and extract the current mission segment, the approximate geometric direction, and constraint information such as speed limits.

[0051] The planner completely ignores all non-target objects and only plans the path centered on the target object established in S131. It optimizes the path through a complex cost function, which includes ensuring the optimal sensor viewpoint, maintaining the best detection distance from the target surface, ensuring coverage integrity, and maintaining path smoothness. Using local path planning algorithms such as RRT, the system generates one or more theoretically optimal fine-grained operation paths, such as vertical scanning paths or zigzag paths with high overlap. These paths represent the best solution for completing the task in an interference-free environment.

[0052] For each threat and its warning information marked in S131, a corresponding evasive maneuver is generated. For dynamic obstacles (such as ships), the system predicts their trajectory and generates multiple evasive options (such as turning left to detour, turning right to detour, slowing down and waiting) based on speed obstacle or COLREGs model. For static obstacles, a simple geometric obstacle avoidance algorithm is used to generate detour paths. This set of outputs is not a complete path from A to B, but a series of pluggable "maneuver modules" or "behavioral fragments", which constitute a "safety contingency plan library" to deal with emergencies.

[0053] Specifically, after the unmanned vessel departs from the dock, its main control computer loads the pre-planned global mission path. Currently, it has passed the "area entrance" waypoint and is in the "gate inspection of gate No. 3" mission segment. The system analyzes that the current global path is a straight line segment that travels roughly along the parallel line of the gate, with a speed limit of 5 knots. This macro path provides the benchmark and context for all subsequent fine planning.

[0054] Centered on "gate pier A" locked by S131, the planner initiated the RRT* algorithm with the primary objective of "maintaining a 15-meter distance from the surface of gate pier A and a perpendicular viewpoint". It generated two candidate operation paths: Path_Task_1 is a vertical scanning path from top to bottom, passing through once; Path_Task_2 is a zigzag path that achieves higher coverage of the entire gate pier surface through two round trips. These two paths constitute the "first set of cruise routes", which is a perfect solution for the unmanned vessel to complete the task in an ideal world, completely ignoring the existence of the fishing boat and the floating tree branches.

[0055] Based on the warning information from S131, the system begins to generate evasion plans. For high-threat fishing vessels, the planner generates two evasion maneuver modules based on their CPA and TCPA: Maneuver_Avoid_Fishing_Starboard (turns 30 degrees to the right for 30 seconds) and Maneuver_Avoid_Fishing_Port (turns 45 degrees to the left and returns). For low-threat floating branches, the planner generates a simple maneuver module: Maneuver_Shift_Left_2m (shifts 2 meters to the left). These three maneuver modules constitute the "second set of cruising routes," which is the "plan library" for unmanned vessels to deal with emergencies. At this point, S132 has produced an ideal mission path and a set of safe evasion plans, which will be sent to S133 for final fusion and decision-making.

[0056] Therefore, based on the matching of the first set of cruise routes and the second set of cruise routes, each priority cruise route is determined, and the route cruise coefficient of each priority cruise route is marked to determine the optimal cruise route of the unmanned vessel. This method takes into account the overall consideration of matching the first set of cruise routes and the second set of cruise routes, ensuring the accuracy of each priority cruise route. At the same time, it further controls the characteristics of target objects and multiple non-target objects, taking into account the feature positions of target objects, the cruise path of the unmanned vessel, and the cruise warning information of multiple non-target objects, thereby improving the accuracy of the optimal cruise route of the unmanned vessel.

[0057] At this point, the system identifies the "trigger points" on the ideal mission path where evasion needs to be performed. These points are typically calculated based on TCPA or safe distance thresholds relative to obstacles. A spatiotemporal stitching algorithm "transforms" each available evasion maneuver module to these trigger points. This "transformation" process calculates a complete transition trajectory from smooth deviation from the main path, evasion, and smooth return, ensuring that it conforms to the kinematic and dynamic constraints of the unmanned vessel. Each successful transformation generates a brand new candidate route from the starting point to the end point, thus forming a set of multiple candidate routes.

[0058] The system employs a multi-objective cost function, which is typically a weighted sum of various costs. These cost items include: efficiency costs determined by path length and time; task quality costs measuring deviations from the ideal path; safety costs inversely proportional to obstacle distances; and energy consumption costs determined by acceleration and deceleration frequencies. The weighting coefficients of each cost are not fixed but can be dynamically adjusted based on the task stage (e.g., increasing safety weights in complex waters) and the system's own state (e.g., increasing energy consumption weights when battery is low). The function outputs a single "route cruise coefficient," the lower of which indicates better overall route performance.

[0059] The system searches for the route with the smallest "route cruise coefficient" among all candidate routes. This lowest cost value represents the best balance between multiple objectives such as efficiency, quality, safety, and energy consumption under the current weight settings, and determines the optimal cruise route. At this time, the optimal cruise route contains a complete path with detailed waypoints and speed commands, which will be formally issued to the next level motion control and execution module (S14) as the action guidelines that the unmanned vessel must follow.

[0060] Specifically, the system obtained the output of S132: an ideal mission path for parallel navigation along the lock pier, and two evasive maneuver modules for the fishing boat: "left large detour" and "right small detour". The system calculated on the mission path that evasion needs to be performed 50 meters ahead. The fusion algorithm transformed the "left large detour" module to this point and generated the complete Candidate_Path_1 by calculating the smooth turning and return arc. It then transformed the "right small detour" module to the same position, generating Candidate_Path_2. At this point, the system had two complete candidate routes available for evaluation.

[0061] The system calculates the cost of two candidate routes and assigns weights based on current safety priority (w1=0.2, w2=0.2, w3=0.5, w4=0.1). For Candidate_Path_1 (left detour), the path is long and the detection quality is affected during the detour, but the closest distance to the fishing boat is 50 meters, making it very safe, and the turn is also gentle; its total coefficient is calculated to be 33. For Candidate_Path_2 (right detour), the path is short and has little impact on detection, but the closest distance to the fishing boat is only 25 meters and it is close to the shore, making it a higher risk; its total coefficient is calculated to be 53.

[0062] The system directly compares the two total coefficients: 33 < 53; since Candidate_Path_1 has a lower overall cost, the system officially determines it as the "best cruising route"; after receiving the detailed path instruction of "making a large detour to the left", the motion controller of the unmanned ship immediately begins to execute the turn precisely, and proceeds along this longer but much safer route, and continues to complete the inspection task of gate pier No. 3 in an orderly manner while ensuring its own absolute safety.

[0063] refer to Figure 5 In step S14, the specific steps are as follows: S141: Input the optimal cruise route into the route analysis module of the unmanned vessel, and determine multiple inspection nodes under the calculation of the route analysis module. Based on the node positions of multiple cruise nodes, the corresponding node areas and the current position of the unmanned vessel, trigger the unmanned vessel to carry out targeted cruise work relative to the characteristics of the target object. S142: In this targeted patrol operation, multiple attitude parameters of the unmanned vessel are collected. Based on the multiple attitude parameters, the current position of the unmanned vessel, and the water surface state, the combination of attitude parameters of the unmanned vessel is determined. At the same time, dynamic information of the target object features is collected. Based on the combination of attitude parameters of the unmanned vessel and the dynamic information of the target object features, the first layer of target patrol content is determined. S143: Determine the second level of target cruise content based on the current position of the unmanned vessel and the dynamic information of the target object characteristics, and determine the target cruise event of the unmanned vessel based on the first level of target cruise content and the second level of target cruise content.

[0064] In the embodiments of this application, the optimal cruise route is input into the route analysis module of the unmanned vessel, and multiple inspection nodes are determined by the calculation of the route analysis module. Based on the node positions of the multiple cruise nodes, the corresponding node areas, and the current position of the unmanned vessel, the unmanned vessel is triggered to perform targeted cruise operations relative to the characteristics of the target object. This approach is compatible with the overall considerations under the calculation of the route analysis module and ensures the accuracy of the multiple inspection nodes.

[0065] At this point, the system analyzes the continuous path (such as a B-spline curve or a dense point sequence) output by S13, and generates nodes using a hybrid sampling strategy based on its geometric characteristics and task requirements. Equidistant sampling is used on smooth road sections to ensure efficiency, curvature-based sampling is used on sharp bends to improve accuracy, and event-driven sampling is performed at locations with special tactical significance (such as directly facing the target center point). A continuous path is transformed into a set of ordered, finite "inspection node" sequences.

[0066] Each "inspection node" is defined as a structured data object with attributes far exceeding three-dimensional coordinates. It contains a "node region" (such as a sphere) that defines the "arrival" tolerance range to ensure the robustness of control. It also contains the expected kinematic parameters such as attitude and velocity when arriving at the node. Most importantly, it encapsulates a "trigger command" that defines the action that should be performed immediately when entering the region, such as starting a scan, taking a picture, or switching modes.

[0067] In a high-frequency control loop, the system continuously compares the real-time position of the unmanned vessel with the "node region" of the current target node. This "node region" is actually a geofence. Once the unmanned vessel enters the fence, the system immediately performs two operations: first, it triggers the instruction encapsulated in the node, and second, it updates the task state machine, switching from the "navigation" state to the "execute task" state, thereby unlocking the execution permission for subsequent fine control modules. At the same time, the system sets the next node in the sequence as the new target and starts a new round of navigation and triggering loop.

[0068] Specifically, the S13 module of the unmanned vessel outputs an optimal operating path that is 50 meters long and parallel to "No. 3 gate pier A". The route analysis module identifies that the main body of the path is a straight line with transition arcs at both ends. It uses equidistant sampling in the middle 45-meter straight section, generating a node every 0.5 meters. In the arc sections at both ends, it uses curvature-based sampling to increase the node density to ensure smooth turning. At the very center of the path, through event-driven sampling, an additional special node is generated to perform the core photo-taking task. The module generates an ordered sequence containing 101 "inspection nodes".

[0069] The system encapsulates instructions for key nodes in the node sequence; the starting node Node_1 is defined as: {position:(x_start,y_start,z), area: radius 1.5 meters, desired speed: 2 knots, trigger instruction: "Start_Inspection_Mode"}; the center image capture node Node_51 is defined as: {position:(x_center,y_center,z), area: radius 0.5 meters, desired attitude: bow facing the center of the lock pier, desired speed: 0.5 knots, trigger instruction: "Execute_Image_Capture_Sequence"}; the instruction for the ending node Node_101 is to end the operation mode and resume cruise.

[0070] The unmanned surface vessel (USV) is heading towards the first critical node, Node_1, at a speed of 2 knots. In the background, the trigger continuously calculates the distance between the USV and Node_1. When the USV enters the geofence 1.5 meters from the center point of Node_1, the judgment condition is immediately met. The system then executes the trigger command "Start_Inspection_Mode" from Node_1, and the task state machine switches from "cruising" to "targeted cruise operation". The controller automatically limits the speed to within 2 knots and activates the image stabilization and gimbal tracking systems, making all preparations for the precise operations of S142 and S143. At the same time, Node_2 is set as the new navigation target, and the entire execution process begins to operate in an orderly manner.

[0071] Furthermore, in this targeted patrol operation, multiple attitude parameters of the unmanned surface vessel (USV) are collected. Based on these multiple attitude parameters, the USV's current position, and the water surface condition, the combination of USV's attitude parameters is determined. Simultaneously, dynamic information of the target object's features is collected. Based on the USV's attitude parameter combination and the target object's dynamic information, the first layer of target patrol content is determined. This comprehensive consideration of both the USV's attitude parameter combination and the target object's dynamic information ensures the accuracy of the first layer of target patrol content.

[0072] At this point, the system collects the three-axis angular velocity and linear acceleration of the unmanned vessel through a high-frequency IMU, which is the basis for calculating the real-time attitude. At the same time, it generates a high-frequency, high-precision global pose estimate by fusion of RTK-GPS and IMU sensors. In order to achieve platform stability, the system also collects the state information of water surface waves through dedicated sensors or vision algorithms. In addition, for dynamic targets, the system continuously updates their relative velocity and predicted trajectory. These multi-dimensional data together constitute a complete, real-time state snapshot.

[0073] The system employs the advanced strategy of Model Predictive Control (MPC). In each control cycle, MPC predicts the attitude response under different control inputs over a short period of time based on the built-in unmanned vessel dynamics model. Through rolling optimization, it solves for a sequence of control commands that minimizes the error between the predicted attitude and the desired attitude, while also taking into account physical constraints such as the thrusters. The final output of MPC is a synergistic "attitude parameter combination", such as the thrust distribution of the left and right thrusters. This set of commands can accurately generate torques to counteract disturbances and actively stabilize the hull in the desired attitude.

[0074] The system is set with a trigger that will activate data acquisition only when multiple conditions such as attitude stability, position accuracy and target alignment are met simultaneously. Once triggered, the system will execute a preset data acquisition sequence, the output of which is the "first layer of target cruise content". This is usually high-fidelity, high-resolution raw data, such as industrial camera photos without motion blur and high-density 3D point clouds, which is the basis for all subsequent depth analysis.

[0075] Specifically, the unmanned vessel entered "targeted patrol operation" mode, facing a suspected cracked area on the gate pier; the IMU detected that the hull had rolled 2 degrees and pitched 1 degree due to the waves; the RTK-GPS / IMU fusion system output its centimeter-level precise position and forward speed of 0.5 knots; the vision system analyzed the water surface ripples and estimated that the current short-wave swell was 0.3 meters high; since the target was static, its dynamic information was zero.

[0076] After receiving the above data, the MPC controller started working. It predicted that if no intervention was made, the roll would increase to 4° in the next second. Through optimization calculations, the MPC determined that a corrective torque needed to be applied to the left and output the command: {left thruster thrust: 52%, right thruster thrust: 48%}. This tiny thrust difference generated a torque opposite to the wave torque. Within 1 second, the ship's roll angle was quickly corrected to within ±0.1 degrees, reaching an extremely stable state.

[0077] After stabilizing the hull, the system detected a roll angle of 0.05° and a pitch angle of -0.02°, both less than the threshold of 0.1°. At the same time, the gimbal had locked the crack in the center of the image. The "Acquisition Ready" trigger was immediately activated, and the system sent an exposure command to the industrial camera. The camera took a 50-megapixel RAW format photo, named IMG_Crack_001PierA.tif. This image is the "first target cruise content" of this execution. It clearly records the shape of the crack and provides irrefutable high-quality raw data for the subsequent "deformation detection of hydraulic structures in locks and pumping stations".

[0078] Therefore, the second level of target cruise content is determined based on the current position of the unmanned vessel and the dynamic information of the target object characteristics. The target cruise event of the unmanned vessel is determined based on the first level of target cruise content and the second level of target cruise content. This approach takes into account both the first level of target cruise content and the second level of target cruise content, ensuring the accuracy of the target cruise event of the unmanned vessel.

[0079] At this point, the system acquires high-precision six-DOF pose, including position, attitude, and velocity, output by the S142 sensor fusion module; it acquires dynamic information or fixed coordinates of the target object's features; through pure geometric and kinematic calculations, the system calculates key relative state parameters, such as relative distance, azimuth, and pitch angle; simultaneously, the system also records environmental parameters (water temperature, illumination) and platform status (battery power, sensor operating status) during the acquisition; all these calculated parameters together constitute the "second layer of target cruise content," a set of structured metadata describing the acquisition environment.

[0080] Timestamp synchronization is performed to ensure that the metadata accurately describes the state of the original data at the moment it was collected. The system encapsulates the first-level target cruise content and the second-level target cruise content into a standardized, self-interpreting data structure, namely the "target cruise event". This event data package typically contains a unique event identifier, a precise timestamp, a task context, a reference to the original data (first-level content), and all metadata (second-level content). This complete event package is the smallest and most complete unit for subsequent intelligent diagnosis and digital twin updates.

[0081] Specifically, at the same moment that S142 takes a picture of the crack in the gate pier, S143 works in parallel; the system acquires the real-time pose of the unmanned vessel: position (x_u, y_u, z_u), attitude (roll=0.05°, pitch=-0.1°, yaw=90.2°); at the same time, it retrieves the world coordinates (x_c, y_c, z_c) of the crack from the database; through calculation, the system finds that the crack is 15.2 meters away from the unmanned vessel, the relative azimuth is 90.3°, and the relative pitch is -1.2°; the system also records that the current water flow speed is 0.5 m / s, the light intensity is 50,000 lux, and the battery charge is 85%; all of this data is packaged into "secondary target cruise content".

[0082] After S142 and S143 are completed, the system performs final encapsulation; the capture timestamp of the photo IMG_Crack_001.tif is 2023-10-27T10:35:00.123Z, and all metadata is also locked at this moment; the system creates a Cruise_Event object, which packages all information, including the event ID, timestamp, task ID, reference to the photo file, and complete metadata such as relative pose, hull status, and environmental parameters. This complete Cruise_Event object is written to the ship's onboard storage system or transmitted in real time to the shore-based control center. This structured "target cruise event" is the most basic and reliable data unit required for the research on "deformation detection of lock and pumping station hydraulic structures".

[0083] refer to Figure 6In step S15, the specific steps are as follows: S151: Dynamically identify the target cruise event and determine multiple project nodes during the identification process. Based on the tracing of each project node, determine the corresponding target cruise project to identify multiple target cruise projects. S152: Determine the priority of each target cruise project based on the comparison of multiple target cruise projects. At the same time, collect the work task list of the unmanned vessel. Determine the first level of intelligent control content based on the priority of multiple target cruise projects and the work task list of the unmanned vessel. Determine the second level of intelligent control content based on the project content of multiple target cruise projects and the work task list of the unmanned vessel. S153: Determine the intelligent control system of the unmanned vessel based on the first level of intelligent control content, the second level of intelligent control content, and the current battery level of the unmanned vessel; determine multiple working contents based on the identification of the intelligent control system of the unmanned vessel; and determine the multi-dimensional working contents of the unmanned vessel relative to the characteristics of the target object based on the multiple working contents, the corresponding working dimensions, and the current dynamic behavior of the target object characteristics.

[0084] In the embodiments of this application, the target cruise event is dynamically identified, and multiple project nodes are determined during the identification process. The corresponding target cruise project is determined based on the tracing of each project node, thereby determining multiple target cruise projects. This approach takes into account the overall consideration of tracing each project node and ensures the accuracy of the corresponding target cruise project.

[0085] At this point, the system receives the "target cruise event" containing raw data and metadata, and hands it over to the knowledge graph and inference engine for processing. The inference engine performs visual analysis on the images or point clouds in the event and extracts low-level features. These features are then semantically matched with conceptual patterns (such as "cracks" and "seepage points") in the knowledge graph. The event's metadata (such as distance and camera parameters) is used for contextual verification, and the pixel information in the image is converted into the physical dimensions of the real world to confirm the recognition result. This system is dynamic and can continuously learn and optimize through feedback, making the recognition increasingly accurate.

[0086] Each successful semantic recognition triggers the system to create a new "project node" instance. This new node is immediately populated with rich attributes: some are physical attributes inherited from the recognition results (such as the length and width of the crack); others are "birth certificates" inherited from the event metadata, including its precise spatial location, discovery time, and the environment and platform status at the time of discovery. Each node is assigned a globally unique identifier (GUID) to ensure that it can be accurately indexed and traced throughout the entire system lifecycle.

[0087] The system utilizes the relationship definitions in the knowledge graph to automatically trace newly created project nodes to higher-level tasks or research objectives. For example, a newly discovered "structural crack" node will be traced to the parent concept of "deformation signs of hydraulic structures" and ultimately linked to the top-level task "deformation detection method of gate and pumping station hydraulic structures". The system will dynamically aggregate or generate one or more business-oriented "target cruise projects" based on the type, location, and association of nodes. For example, it can generate a "refined modeling and trend analysis" project for a single critical crack, or aggregate multiple small cracks into a "crack cluster survey" project.

[0088] Specifically, the unmanned surface vessel generated a Cruise_Event containing a photo of a suspected crack; the inference engine detected a clear linear feature in the photo and matched it with the concept of "crack" in the knowledge graph; the engine extracted {relative distance: 15.2m, camera focal length: 25mm, image pixel length: 150px} from the event metadata, and calculated the actual length of the crack to be approximately 25cm through geometric conversion; the system confirmed this identification result as "Class I structural crack", which is a semantic label with clear engineering significance.

[0089] Based on the identification results from the previous step, the system created a new instance node in the knowledge graph and assigned it a GUID. This node was populated with rich attributes: the concept type was "Class I structural crack", the physical attributes were {length: "25cm", width: "1mm"}, the spatial location was {x:12345.67, y:23456.78, z:-5.5}, ​​the discovery time was "2023-10-27T10:35:00.123Z", and it was associated with the original Cruise_EventID.

[0090] The system automatically traces the newly created crack node back to the parent concept of "deformation signs of hydraulic structures" through the knowledge graph, and finally establishes a link with the top-level task "deformation detection method of gate and pumping station hydraulic structures". Since this is the first such crack found on "No. 3 gate pier A", the system does not aggregate it, but directly generates a new and specific target cruise project for it: "refined modeling and trend analysis of key cracks (ID:GUID-123...) of gate pier A". This project is passed to S152 as an independent, high-priority task unit for subsequent intelligent control decision-making.

[0091] Furthermore, the priority of each target cruise project is determined by comparing multiple target cruise projects. At the same time, the work task list of the unmanned vessel is collected. The first level of intelligent control content is determined based on the priority of multiple target cruise projects and the work task list of the unmanned vessel. The second level of intelligent control content is determined based on the project content of multiple target cruise projects and the work task list of the unmanned vessel. This overall consideration of the project content of multiple target cruise projects and the work task list of the unmanned vessel ensures the accuracy of the second level of intelligent control content.

[0092] At this point, the system uses a multi-factor weighted scoring model to score each "target cruise project". The evaluation factors include: risk level quantified according to defect type, task relevance to the main task, estimated time and power consumption (resource cost), and urgency of the project (time sensitivity). Each factor has a configurable weight, and a comprehensive priority score is finally calculated. The system sorts all projects according to this score, generating a clear priority queue to ensure that the most critical tasks are processed first.

[0093] The system selects the highest-scoring item from the priority queue as a candidate; it then verifies this candidate against the "work task list" (a set of high-level rules and constraints) issued by the shore-based system to ensure that it does not conflict with the established strategy; once the item is finally determined, the system will generate an atomic, immediately executable action instruction based on its specific requirements.

[0094] The system is no longer limited to the highest priority projects, but examines the "content" of all projects from a global perspective and understands all the current task requirements. It will aggregate multiple related project contents into a higher-level strategic goal, such as aggregating multiple "crack modeling" projects into "completing the crack survey of the entire gate pier". Aligning this aggregated goal with the long-term goals in the "work task list", this second layer of content is medium- to long-term, goal-oriented content, which provides a clear direction for the unmanned vessel.

[0095] Specifically, S151 output two projects: Project_1 (refined crack modeling) and Project_2 (biomarker assessment). The system performed priority calculations (weights w1=0.5, w2=0.3, w3=0.1, w4=0.1). Project_1 received a high score of 86 due to its high risk (90 points) and high task relevance (100 points). Project_2, on the other hand, received only 34 points due to its low risk (20 points) and moderate relevance (50 points). The ranking results clearly show that Project_1 has a much higher priority than Project_2.

[0096] The system selects the highest priority project_1 as a candidate; the task table contains a rule {rule: "prioritize handling all Level I risks"}, and project_1 is completely consistent with this rule; therefore, based on the "refined modeling" requirement of project_1, the system generates the first layer of intelligent control content: {action: "refined scanning", target: "Crack_PierA_01", execution method: "use airborne LiDAR to perform high-density 3D point cloud acquisition"}, which is the most urgent task that the unmanned vessel will immediately execute.

[0097] From a global perspective, the system recognizes that the core task revolves around "cracks," and the top-level task in the work task list is "to complete the structural inspection of Gate 3." Based on this, the system generates a second layer of intelligent control: {Strategic objective: "to complete a comprehensive survey of Level I risk cracks in all visible areas of Gate A," execution strategy: "After completing a detailed scan of the current point, automatically plan an optimal path to sequentially visit all discovered suspected crack points for confirmation and modeling"}. This provides the unmanned vessel with a clear direction for its next action after completing the current task, ensuring that its work always revolves around the core mission.

[0098] Therefore, the intelligent control system of the unmanned vessel (UAV) is determined based on the first level of intelligent control content, the second level of intelligent control content, and the current battery level of the UAV. Multiple operational contents are identified based on the recognition of the UAV's intelligent control system. Based on these multiple operational contents, their corresponding operational dimensions, and the current dynamic behavior of the target object's characteristics, the multi-dimensional operational contents of the UAV relative to the target object's characteristics are determined. This approach integrates the overall consideration of multiple operational contents, their corresponding operational dimensions, and the current dynamic behavior of the target object's characteristics, ensuring the accuracy of the multi-dimensional operational contents of the UAV relative to the target object's characteristics. Simultaneously, it achieves an overall consideration of the project content, corresponding priorities, and the UAV's task schedule for multiple target cruise projects, improving the accuracy of the UAV's intelligent control system and ensuring precise interaction between the UAV and the target object's characteristics.

[0099] At this point, the system comprehensively considers the instructions output by S152, strategic objectives, and the most critical physical constraint—current battery power. The decision logic performs a survivability check: if the battery power is lower than the preset "safe return threshold," safety takes precedence over everything, and the system will ignore all tasks and directly instantiate the decision as "emergency return." If the battery power is sufficient, the system will treat the instructions as the "current action to be executed" and the strategic objectives as the "subsequent planning objectives," package the two into a complete action plan, and "instantiate" it into an "intelligent control system" that includes current tasks, subsequent planning, and resource status.

[0100] The system decomposes the instantiated "intelligent control system" into an ordered, atomized sequence of work content; it assigns rich work dimensions to each work content, such as space, time, sensors, accuracy, and energy consumption, and defines specific parameters for "how to do it"; at the same time, the system combines the current dynamic behavior (static or dynamic) of the target object's characteristics to inject corresponding control strategies into the instructions; the system generates a series of "multi-dimensional work content" containing all execution details, and these instructions are directly issued to the unmanned vessel's underlying control system to drive it to complete high-precision autonomous operations.

[0101] Specifically, the system receives the output from S152 and detects that the current battery level of the unmanned vessel is 85%. Since 85% is much higher than the 20% safe return threshold, the survivability check is passed. The system enters the fusion decision-making stage, packaging the current action "fine scan the crack" with the subsequent strategy "complete all crack surveys". The system instantiates the final intelligent control system: {System: "Execute fine scan and continue survey", Current task: "Fine scan Crack_PierA_01", Subsequent planning: "Automatically plan and visit the next crack point", Resource status: "Sufficient battery"}.

[0102] The system decomposes the previous step's structure, resulting in two atomized task functions: "Move to the optimal scanning position" and "Execute 3D laser scanning." Considering the target crack is static, the system injects detailed execution parameters into these two task functions. For "Move to the optimal scanning position," the generated multi-dimensional task function is: {Execution: "Path planning and navigation", Target: "Scan position_P1", Dimension: {Spatial: "5 meters from the crack, directly facing", Time: "Arrival within 60 seconds", Sensor: "RTK-GPS / IMU", Accuracy: "Centimeter-level positioning", Energy consumption: "Economy mode"}}, Target state: "Static"}; For "Execute 3D laser scanning", the generated instructions are: {Execute: "Laser scan", Target: "Crack_PierA_01", Dimension: {Spatial: "Maintain 5-meter distance", Time: "Continue for 30 seconds", Sensor: "LiDAR_X1", Sampling rate: "100kHz", Accuracy: "Sub-millimeter point cloud"}, Target state: "Static"}. These finally generated "multi-dimensional work content" containing all execution details are directly sent to the unmanned ship's underlying control system to drive it to complete high-precision autonomous operations.

[0103] Please see Figure 7 , Figure 7 This is a schematic diagram of the structural composition of the intelligent control system for the unmanned surface vessel based on vision detection according to an embodiment of the present invention; the intelligent control system for the unmanned surface vessel based on vision detection includes: The cruise dynamic map module 21 is used to perform visual detection of the surrounding environment by the camera configured on the unmanned vessel when the unmanned vessel is in the water surface cruise state, so as to collect multiple surrounding environment images and determine the corresponding cruise dynamic map based on the multiple surrounding environment images and the overall shape of the unmanned vessel. The cruise warning information module 22 is used to determine multiple object regions based on the image recognition of the cruise dynamic map, determine the corresponding object features according to the regional position, corresponding regional shape and image recognition model of each object region, and determine each cruise warning information based on the relative position of the object features and the unmanned vessel and the corresponding feature information. The optimal cruise route module 23 is used to determine the optimal cruise route of the unmanned vessel based on the feature position of the target object feature, the cruise path of the unmanned vessel, and the cruise warning information of the multiple non-target object features if the multiple object features are target object features and multiple non-target object features. The target cruise event module 24 is used for the unmanned vessel to cruise along the optimal cruise route and to determine the target cruise event of the unmanned vessel based on the dynamic information of the unmanned vessel's attitude parameter combination, the corresponding current position and the characteristics of the target object. The intelligent control module 25 is used to identify multiple target cruise items based on the recognition of the target cruise event, determine the intelligent control system of the unmanned vessel according to the project content, corresponding priority and the unmanned vessel's work task table of the multiple target cruise items, and determine the multi-dimensional work content of the unmanned vessel relative to the characteristics of the target object.

[0104] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. An intelligent control method for unmanned surface vessels based on vision detection, characterized in that, include: When the unmanned vessel is cruising on the water, the camera installed on the unmanned vessel performs visual detection of the surrounding environment to collect multiple images of the surrounding environment. Based on the multiple images of the surrounding environment and the overall shape of the unmanned vessel, the corresponding cruise dynamic map is determined. Based on the image recognition of the cruise dynamic map, multiple object regions are identified. According to the regional location, corresponding regional shape and image recognition model of each object region, the corresponding object features are determined. Based on the relative position of the object features and the unmanned vessel and the corresponding feature information, each cruise warning information is determined. The cruise warning information not only includes the warning type and risk level, but also specifies the target ID, key parameters and specific recommended actions. If multiple object features are respectively target object features and multiple non-target object features, the optimal cruise route of the unmanned vessel is determined based on the feature position of the target object features, the cruise path of the unmanned vessel, and the cruise warning information of the multiple non-target object features; the optimal cruise route includes a complete path with detailed waypoints and speed commands. The unmanned vessel cruises along the optimal cruise route, and the target cruise event of the unmanned vessel is determined based on the dynamic information of the unmanned vessel's attitude parameter combination, the corresponding current position and the characteristics of the target object. Based on the identification of the target cruise event, multiple target cruise projects are determined. According to the project content, corresponding priority and unmanned vessel's work task list of multiple target cruise projects, the intelligent control system of the unmanned vessel is determined, and the multi-dimensional work content of the unmanned vessel relative to the characteristics of the target object is determined.

2. The intelligent control method for unmanned vessels based on vision detection according to claim 1, characterized in that, When the unmanned surface vessel (USV) is cruising on the water, the cameras mounted on the USV perform visual detection of the surrounding environment to collect multiple images of the surrounding environment. Based on these multiple images and the overall shape of the USV, a corresponding dynamic cruising map is determined, including: The unmanned vessel floats on the water surface and cruises relative to the water surface. Based on the detection of the unmanned vessel, the corresponding cruise information is determined. Based on the identification of the cruise information, multiple key detection areas are determined. Based on the regional location of each key detection area and the camera configured on the unmanned vessel, the visual detection mode of the camera is determined, and the camera is triggered to perform visual detection of the surrounding environment along the visual detection mode. Multiple surrounding environment images are determined based on the visual detection of the surrounding environment by the camera, and the corresponding image content is marked. The corresponding cruise dynamic map is determined based on the multiple surrounding environment images, the corresponding image content, and the overall shape of the unmanned vessel.

3. The intelligent control method for unmanned vessels based on vision detection according to claim 1, characterized in that, The image recognition based on the cruise dynamic map determines multiple object regions. Based on the region location, corresponding region shape, and image recognition model of each object region, corresponding object features are determined. Based on the relative position of these object features with the unmanned vessel and corresponding feature information, various cruise warning messages are determined, including: Image recognition is performed on the cruise animation, and multiple regions are identified during the recognition process. Based on the image recognition of each region, the corresponding object regions are identified, thus identifying multiple object regions. Based on the detection of each object region, the region position and corresponding region shape of each object region are determined. A model database is established based on the detection of the unmanned vessel. The corresponding image recognition model is determined by matching the model database with the cruise dynamic map. The corresponding object features are determined based on the regional position of each object area, the corresponding regional shape, and the image recognition model. At the same time, the relative position of the object feature and the unmanned vessel is marked. Each cruise warning message is determined based on the relative position, the feature information of the object feature, and the corresponding feature shape.

4. The intelligent control method for unmanned vessels based on vision detection according to claim 1, characterized in that, If the multiple object features are respectively target object features and multiple non-target object features, the optimal cruise route of the unmanned vessel is determined based on the feature position of the target object features, the cruise path of the unmanned vessel, and the cruise warning information of the multiple non-target object features, including: Multiple object features are identified, and target object features and multiple non-target object features are marked during the identification process. The target object features serve as the final interaction object of the unmanned vessel. Based on the early warning detection of each non-target object feature, the corresponding cruise early warning information is determined, thereby marking the cruise early warning information of multiple non-target object features.

5. The intelligent control method for unmanned vessels based on vision detection according to claim 4, characterized in that, If the multiple object features are respectively target object features and multiple non-target object features, determining the optimal cruise route of the unmanned vessel based on the feature position of the target object features, the cruise path of the unmanned vessel, and the cruise warning information of the multiple non-target object features also includes: The unmanned surface vessel (USV) is collected, and a first set of cruise routes is determined based on the USV's cruise route and the feature location of the target object. A second set of cruise routes is determined based on the USV's cruise route and cruise warning information of multiple non-target object features. Each priority cruise route is determined based on the matching of the first group of cruise routes and the second group of cruise routes, and the route cruise coefficient of each priority cruise route is marked to determine the optimal cruise route for the unmanned vessel.

6. The intelligent control method for unmanned surface vessels based on vision detection according to claim 1, characterized in that, The unmanned surface vessel (USV) cruises along an optimal route. Based on the USV's attitude parameter combinations, its current position, and dynamic information about the target object's features, the target cruise event is determined, including: The optimal cruise route is input into the route analysis module of the unmanned vessel, and multiple inspection nodes are determined by the calculation of the route analysis module. Based on the node positions of multiple cruise nodes, the corresponding node areas, and the current position of the unmanned vessel, the unmanned vessel is triggered to carry out targeted cruise operations relative to the characteristics of the target object.

7. The intelligent control method for unmanned surface vessels based on vision detection according to claim 6, characterized in that, The unmanned surface vessel (USV) cruises along an optimal route. The target cruise event of the USV is determined based on the USV's attitude parameter combination, its current position, and dynamic information about the target object's features. The method also includes: In this targeted patrol operation, multiple attitude parameters of the unmanned vessel are collected. Based on these multiple attitude parameters, the current position of the unmanned vessel, and the water surface condition, the combination of attitude parameters of the unmanned vessel is determined. At the same time, dynamic information of the target object features is collected. Based on the combination of attitude parameters of the unmanned vessel and the dynamic information of the target object features, the first layer of target patrol content is determined. The second layer of target cruise content is determined based on the current position of the unmanned vessel and the dynamic information of the target object characteristics. The target cruise event of the unmanned vessel is determined based on the first layer of target cruise content and the second layer of target cruise content.

8. The intelligent control method for unmanned surface vessels based on vision detection according to claim 1, characterized in that, The process involves identifying multiple target cruise items based on the target cruise event, determining the intelligent control system of the unmanned vessel (UAV) based on the item content, corresponding priority, and the UAV's task schedule, and defining the multi-dimensional operational content of the UAV relative to the target object's characteristics, including: The target cruise event is dynamically identified, and multiple project nodes are determined during the identification process. The corresponding target cruise project is determined based on the tracing of each project node, thus identifying multiple target cruise projects.

9. The intelligent control method for unmanned surface vessels based on vision detection according to claim 8, characterized in that, The process of identifying multiple target cruise items based on the target cruise event, determining the intelligent control system of the unmanned vessel based on the item content, corresponding priority, and unmanned vessel's work task list of the multiple target cruise items, and determining the multi-dimensional work content of the unmanned vessel relative to the characteristics of the target object, also includes: The priority of each target cruise project is determined by comparing multiple target cruise projects. At the same time, the work task list of the unmanned vessel is collected. The first level of intelligent control content is determined based on the priority of multiple target cruise projects and the work task list of the unmanned vessel. The second level of intelligent control content is determined based on the project content of multiple target cruise projects and the work task list of the unmanned vessel. The intelligent control system of the unmanned vessel is determined based on the first level of intelligent control content, the second level of intelligent control content, and the current battery level of the unmanned vessel. Multiple working contents are determined based on the identification of the intelligent control system of the unmanned vessel. Based on the multiple working contents, the corresponding working dimensions, and the current dynamic behavior of the target object characteristics, the multi-dimensional working contents of the unmanned vessel relative to the target object characteristics are determined.

10. An intelligent control system for an unmanned surface vessel based on vision detection, characterized in that, The intelligent control system for the vision-based unmanned vessel is applied to the intelligent control method for the vision-based unmanned vessel as described in any one of claims 1-9.

Images