An Automatic Docking Method and System for Large Marine LNG Discharge Arms Based on Visual SLAM
By constructing a local cooperative localization map using visual SLAM and inferring the flange pose using the ship's natural structural features, the docking accuracy problem caused by flange obstruction in existing technologies has been solved, enabling high-precision automatic docking in complex sea conditions and reducing modification costs.
Patent Information
- Application Number
- CN202611098777.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-23
- Publication Date
- 2026-11-10
AI Technical Summary
Existing automated docking technology for large marine LNG unloading arms cannot continuously obtain reliable relative positions and postures after the flange is blocked, and requires modification of the ship or installation of external targets, which makes it impossible to achieve high-precision docking in complex sea conditions.
A vision-based SLAM approach is adopted, which constructs a local cooperative positioning map through non-contact sensing sensors and inertial measurement units at the end of the loading arm. The relative pose is calculated when the flange is obscured by the ship's natural structural features. The closed-loop control is achieved by combining inertial and visual data for multi-source fusion estimation.
It can still achieve high-precision docking even when the flange is blocked, reducing the cost of ship modification, and is suitable for various LNG ship types, while maintaining stable system operation in complex sea conditions.
Smart Images

Figure CN123064918A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the fields of automation of ship-to-shore fluid transfer equipment at LNG receiving terminal terminals, machine vision, visual SLAM, and control technology of large hydraulic unloading arms. In particular, it is an automatic docking method and system for large marine LNG unloading arms based on visual SLAM. Background Technology
[0002] Marine-mounted large LNG unloading arms are key equipment connecting terminal cryogenic pipelines to LNG ship manifold flanges. They typically consist of a column, inner arm, outer arm, cryogenic rotary joint, emergency disconnect device, quick-connect device, and hydraulic drive system. The medium they transport is cryogenic and flammable liquefied natural gas, and the operating area is located at the ship-shore interface, requiring extremely high levels of sealing and safety. Furthermore, berthed vessels continuously experience six degrees of freedom micro-motions (sway, pitch, heave, roll, pitch, and bow) due to wind, waves, currents, tides, and the elasticity of mooring lines, significantly complicating the final docking operation.
[0003] Currently, the docking operation of marine LNG unloading arms is still mainly carried out manually and remotely. Operators judge the relative position of the unloading arm end and the target flange by visual observation or video monitoring, and control the hydraulic joints slowly to approach and complete the docking connection in a jogging manner. Due to the long arm length and high moment of inertia of large unloading arms, and the nonlinear characteristics of the hydraulic actuator system such as dead zone, hysteresis and low-speed crawling, relying solely on open-loop control of the joints cannot guarantee the millimeter-level accuracy required for the end docking. Manual docking is not only inefficient, but also requires operators to be close to the ship's manifold area, facing multiple safety risks such as cryogenic medium leakage, flammable gas accumulation, collisions, and misoperation. Existing automatic docking technologies can be divided into three technical routes, but all of them have fundamental limitations.
[0004] The first category is the direct flange identification and phased positioning method. This technology uses point lasers, line lasers, infrared cameras, laser 3D vision cameras, monocular or binocular cameras, and ultrasonic sensors to identify the target flange and switches between coarse and fine positioning processes based on the working distance. This method can provide effective positioning information when the flange is visible, but its core technology always relies on continuous direct observation of the flange itself. Once the end of the unloading arm, the quick-connect device, or the guide structure enters in front of the flange, the flange end face, inner and outer circles, and bolt holes will be obstructed, the visual link will be interrupted, and the end docking will be forced to degenerate into low-speed blind docking or short-term outward push. Traditional blind docking is based on the premise that the ship's motion changes little in a short period of time. When wind, waves, currents, or changes in cable tension are significant, this premise cannot be established, thus severely limiting the sea state range that automatic docking can be applied to.
[0005] The second category involves dual-camera, multi-sensor, or fixed-target-assisted methods. This technology deploys multiple sets of visual sensors at different distances, or sets up target boards, reflectors, QR codes, coded markers, or actively emitting targets around the ship's manifold, allowing the system to continue guidance even after the flange is out of sight, thanks to the assistance of nearby targets. While this approach can alleviate the obstruction problem to some extent, it requires cooperation from the ship's side to install, maintain, or deploy additional targets, involving engineering challenges such as cross-ship type adaptation, adjustments to loading and unloading procedures, explosion-proof certification, and the division of ship-shore responsibilities.
[0006] The third category is solutions based on ship-side sensors or communication assistance. This technology involves installing inertial sensors, wireless positioning devices, or dedicated tracking devices on the ship's side to directly acquire the ship's motion status. Theoretically, this type of solution can obtain ship motion information, but it requires modifications to the ship or temporary installation of equipment, relies on ship-to-shore communication links and equipment compatibility, and is not conducive to rapid deployment for different ship owners, different ship types, and different berthing batches.
[0007] In summary, the core issue of automatic docking of large marine LNG unloading arms is not simply improving the accuracy of flange image recognition, but rather ensuring that the system can still obtain continuous and reliable relative pose even when the flange is obscured by the end mechanism, and complete a safe and compliant connection under continuous six-degree-of-freedom micro-motion conditions, without modifying the ship or relying on external targets. Current technologies have not fully utilized the naturally occurring structural features around the ship's manifold area, which, along with the flange, belong to the rigid hull, nor have they introduced a "local visual SLAM map in a moving hull reference frame" into the docking control process of large LNG unloading arms. Summary of the Invention
[0008] The purpose of this invention is to address the deficiencies or shortcomings of the existing technology by providing an automated docking method for large marine LNG unloading arms based on visual SLAM. This method does not treat the ship's manifold flange merely as a target that needs to remain continuously visible, but rather incorporates it into a local visual SLAM map of the ship's manifold area for processing. Even if the flange is obscured by the end of the unloading arm during its approach, the system can still infer the flange's position from the unobstructed hull structure around it, thus providing continuous relative pose information for the automated docking of the unloading arm.
[0009] The technical solution for achieving the objective of this invention is as follows: On one hand, a method for automatic docking of a large marine LNG unloading arm based on visual SLAM is provided, the method comprising:
[0010] Step 1: After the loading arm enters the ship's pipeline interface area, the initial relative pose of the target interface with respect to the loading arm is obtained using a non-contact sensing sensor mounted on the loading arm, and the ship's local reference frame following the ship is determined accordingly; the loading arm is also equipped with an inertial measurement unit.
[0011] Step 2: Around the target interface, detect and extract various types of natural geometric features that are rigidly connected to the hull as map references, construct a local cooperative positioning map with the ship as a rigid body reference and following the ship's motion, and lock the fixed relative spatial constraint relationship of the target interface in the local cooperative positioning map.
[0012] Step 3: During the docking guidance process, the motion measurement data of the loading arm, the inertial measurement data of the loading arm end, and the local cooperative positioning map are fused and estimated in a spatiotemporal manner. The pose change of the ship's local reference frame relative to the loading arm end or the dock base is calculated in real time. Based on the fixed relative spatial constraint relationship, the relative pose deviation of the loading arm end relative to the target interface is calculated in real time.
[0013] Step 4: When the end of the loading arm or the docking mechanism it carries obstructs the target interface, causing the non-contact sensing sensor to be unable to directly observe the complete features of the target interface, the remaining rigid natural geometric features of the hull that are not obstructed in the local cooperative positioning map are continuously extracted and matched, the real-time pose of the local reference system of the ship is estimated, and the real-time pose of the obstructed target interface is calculated in reverse according to the fixed relative spatial constraint relationship.
[0014] Step 5: Output the calculated relative pose deviation to the loading arm control system to drive the loading arm end to perform centering, approximation and physical locking, thereby realizing closed-loop drive control.
[0015] Furthermore, the initial relative pose of the target interface relative to the end of the loading arm in step 1 is obtained by fitting and solving one or more geometric topological constraints in the target interface's end face circular contour, internal aperture contour, fastening hole array distribution, end face outer edge, and manually assisted designated area, in order to establish the initial pose anchor point of the target interface in the local cooperative positioning map.
[0016] Furthermore, the construction of the local collaborative positioning map in step 2 includes: performing dynamic and non-rigid interference screening on the acquired images or 3D point clouds, retaining only stable environmental features that have a rigid connection with the hull, and eliminating time-varying interference features that do not meet rigid body consistency.
[0017] The time-varying interference features include at least sea surface reflection, sky, moving personnel, swinging of cables and hoses, low-temperature medium vapor, drastic changes in light and shadow, and temporary mobile equipment;
[0018] The local cooperative positioning map records the three-dimensional coordinates of each feature point in the ship's local reference frame, local feature descriptors, multi-angle observation weights, and measurement uncertainties.
[0019] Furthermore, in step 3, a local joint estimation framework is adopted to match the current environmental features collected by the non-contact sensing sensor with the local cooperative positioning map, and to establish a joint optimization objective function that includes visual reprojection residuals, inertial state pre-integration residuals, loading arm joint kinematic residuals, spatial position constraint residuals when the interface is visible, and ship motion continuity residuals based on physical laws, so as to solve the motion state of the loading arm end and the ship local reference frame at the current moment.
[0020] Furthermore, in step 4, when the proportion of directly observable geometric features of the target interface is lower than a preset threshold, or when the overlap between the projection of the docking mechanism at the end of the loading arm in the sensor's field of view and the projection of the target interface is greater than a preset ratio, it is determined that the target interface has entered an obstruction state.
[0021] Once in the obstruction state, the non-contact sensing sensor continuously tracks the unobstructed deck, bulkhead, pipeline support, or railing structure around the target interface to maintain continuous measurement of the six degrees of freedom motion of the ship's local reference frame, and derives the pose of the target interface in real time through the fixed relative spatial constraint relationship.
[0022] Furthermore, in step 2, the local collaborative positioning map is built entirely based on the ship's naturally existing structural texture, without the need to install any active or passive artificial markers, targets, encoders, or wireless communication positioning units on the ship's side.
[0023] Furthermore, in step 2, during the construction of the local cooperative localization map, the local reference frame of the ship is used as the local coordinate system by adopting a graph optimization-based SLAM algorithm. By performing sliding window keyframe optimization on historical observation frames, a factor graph containing spatial point poses and camera poses is constructed. Gravity direction constraints and inertial prior residuals are added when updating map points to suppress long-term drift.
[0024] Furthermore, in step 3, the motion measurement data of the loading and unloading arm includes angle sensor data, length sensor data and joint speed feedback of each active joint. By establishing a forward kinematic model of the fluid multi-link mechanism of the loading and unloading arm, the predicted joint pose of the loading and unloading arm end relative to the dock base is calculated.
[0025] Furthermore, in step 5, the closed-loop drive control adopts a composite control law that combines feedforward and feedback. The feedback control term is calculated based on real-time deviation using PID or sliding mode control, while the feedforward control term is compensated based on the multi-degree-of-freedom ship servo components predicted in the future short period according to the ship's historical motion sequence, so as to offset the motion lag of the ship under wave surge in advance.
[0026] On the other hand, an automated docking system for a large marine LNG unloading arm based on visual SLAM is provided, the system comprising:
[0027] A non-contact sensing module, mounted at the end of the loading arm, is used to collect structural features and image point cloud data of the target interface and the surrounding ship environment;
[0028] An inertial measurement module, mounted on the end of the loading and unloading arm, is used to collect high-frequency triaxial angular velocity and triaxial acceleration data;
[0029] The joint state sensing module is distributed in each joint of the loading arm and is used to acquire real-time geometric deformation and pose measurement data of the loading arm.
[0030] A multi-source fusion processor is used to execute the automatic docking method for large marine LNG unloading arms based on visual SLAM, construct a cooperative motion map, calculate the relative pose deviation, and continuously calculate the interface pose in the non-line-of-sight state of the target interface.
[0031] The control drive controller is used to receive the pose deviation command output by the multi-source fusion processor and drive the actuators of each joint of the loading and unloading arm in a closed loop to complete fully automatic docking and locking.
[0032] On the other hand, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the automatic docking method for large marine LNG unloading arms based on visual SLAM.
[0033] On the other hand, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the visual SLAM-based automatic docking method for large marine LNG unloading arms.
[0034] Compared with the prior art, the significant advantages of this invention are:
[0035] (1) The non-contact sensing sensors and inertial measurement units required by this invention are all configured on the loading and unloading arm side; the local collaborative positioning map is constructed entirely based on the natural structural texture and geometric contour of the ship, without the need to install any active or passive artificial markers, targets, encoders or wireless communication positioning units on the ship side, which can greatly reduce the modification costs and application access threshold of ports and ship owners, and is applicable to various LNG ship types and temporary berthing operations.
[0036] (2) In the final stage when the loading arm approaches the target interface, the flange is easily obscured by the docking mechanism itself. Traditional vision servo cannot continue to directly observe the target after the flange is obscured by the quick-connect device. This invention uses the unobscured rigid natural geometric features of the hull (such as decks, bulkheads, railings, etc.) in the local collaborative map to continue to estimate the pose, and reversely calculates the pose of the target interface through fixed relative spatial constraints, ensuring high accuracy of relative pose estimation and continuity of closed-loop control in non-line-of-sight states.
[0037] (3) When constructing a local collaborative positioning map, the algorithm can dynamically filter and eliminate time-varying dynamic interferences that do not meet the rigid body consistency requirement, such as sea surface reflection, sky, moving personnel, swinging cables and hoses, and steam generated by low-temperature media. This processing ensures the stable operation of the system under complex and changeable weather, lighting conditions and harsh working conditions at sea.
[0038] (4) The system performs spatiotemporal multi-source fusion estimation of non-contact feature reprojection residuals, inertial state pre-integration residuals, loading and unloading arm joint kinematic residuals, and ship motion continuity residuals. The high-frequency and high-dynamic characteristics of inertial data complement the low-frequency and drift-free characteristics of visual features, effectively suppressing the measurement divergence and drift problem of a single sensor in the complex undulating environment of a port.
[0039] (5) The control system predicts the multi-degree-of-freedom ship servo components in the future short period based on the ship's historical motion sequence and uses them as feedforward compensation. This composite control law combining feedforward and feedback can effectively offset the mechanical response lag of the loading arm during wave fluctuations and hull undulations, ensuring a smoother and safer docking process.
[0040] (6) This invention does not regard the ship manifold area as a static environment under the dock coordinate system, but makes an explicit estimate of the six degrees of freedom motion of the ship's local map relative to the dock, which is more in line with the actual state of LNG ships under the action of wind, waves and current.
[0041] (7) The system continuously outputs the relative position between the end of the unloading arm and the ship's manifold flange. The hysteresis, dead zone and long arm flexibility error of the hydraulic system can be corrected by subsequent control links, which reduces the requirements for the absolute positioning accuracy of the large hydraulic unloading arm body.
[0042] (8) The present invention can output relative pose, tracking status and map quality information, and can be connected to existing hydraulic servo control, speed limit control or compliant control modules without modifying the ship side equipment.
[0043] The present invention will now be described in further detail with reference to the accompanying drawings. Attached Figure Description
[0044] Figure 1This is an overall flowchart of the automatic docking method for large marine LNG unloading arms based on visual SLAM according to one embodiment of the present invention.
[0045] Figure 2 This is a schematic diagram showing the coordinate system relationship between the end of the unloading arm, the vision camera, the inertial measurement unit, and the ship's manifold flange in one embodiment.
[0046] Figure 3 This is a schematic diagram of the construction of a local visual SLAM map of the ship's manifold flange and the surrounding hull-attached natural structures in one embodiment.
[0047] Figure 4 This is a schematic diagram of a visual SLAM joint estimation model in a moving hull reference frame in one embodiment. Detailed Implementation
[0048] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0049] It should be noted that if the embodiments of the present invention involve directional indicators (such as up, down, left, right, front, back, etc.), the directional indicators are only used to explain the relative positional relationship and movement of the components in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indicators will also change accordingly.
[0050] Furthermore, if the embodiments of this invention involve descriptions such as "first" or "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of those features. Additionally, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by this invention.
[0051] In one embodiment, a method for automated docking of a large marine LNG unloading arm based on visual SLAM is provided, the method comprising:
[0052] Step 1: After the loading arm enters the ship's pipeline interface area, the initial relative pose of the target interface with respect to the loading arm is obtained using a non-contact sensing sensor mounted on the loading arm, and the ship's local reference frame following the ship is determined accordingly; the loading arm is also equipped with an inertial measurement unit.
[0053] Step 2: Around the target interface, detect and extract various types of natural geometric features that are rigidly connected to the hull as map references, construct a local cooperative positioning map with the ship as a rigid body reference and following the ship's motion, and lock the fixed relative spatial constraint relationship of the target interface in the local cooperative positioning map.
[0054] Step 3: During the docking guidance process, the motion measurement data of the loading arm, the inertial measurement data of the loading arm end, and the local cooperative positioning map are fused and estimated in a spatiotemporal manner. The pose change of the ship's local reference frame relative to the loading arm end or the dock base is calculated in real time. Based on the fixed relative spatial constraint relationship, the relative pose deviation of the loading arm end relative to the target interface is calculated in real time.
[0055] Step 4: When the end of the loading arm or the docking mechanism it carries obstructs the target interface, causing the non-contact sensing sensor to be unable to directly observe the complete features of the target interface, the remaining rigid natural geometric features of the hull that are not obstructed in the local cooperative positioning map are continuously extracted and matched, the real-time pose of the local reference system of the ship is estimated, and the real-time pose of the obstructed target interface is calculated in reverse according to the fixed relative spatial constraint relationship.
[0056] Step 5: Output the calculated relative pose deviation to the loading arm control system to drive the loading arm end to perform centering, approximation and physical locking, thereby realizing closed-loop drive control.
[0057] Furthermore, in one embodiment, the initial relative pose of the target interface relative to the end of the loading arm in step 1 is obtained by fitting and solving one or more geometric topological constraints in the target interface's end face circular contour, internal aperture contour, fastening hole array distribution, end face outer edge, and manually assisted designated area, in order to establish the initial pose anchor point of the target interface in the local cooperative positioning map.
[0058] Furthermore, in one embodiment, the construction of the local cooperative positioning map in step 2 includes: performing dynamic and non-rigid interference screening on the acquired images or 3D point clouds, retaining only stable environmental features that have a rigid connection with the hull, and eliminating time-varying interference features that do not meet rigid body consistency.
[0059] The time-varying interference features include at least sea surface reflection, sky, moving personnel, swinging of cables and hoses, low-temperature medium vapor, drastic changes in light and shadow, and temporary mobile equipment;
[0060] The local cooperative positioning map records the three-dimensional coordinates of each feature point in the ship's local reference frame, local feature descriptors, multi-angle observation weights, and measurement uncertainties.
[0061] Furthermore, in one embodiment, step 3 employs a local joint estimation framework to match the current environmental features collected by the non-contact sensing sensor with the local cooperative positioning map, and establishes a joint optimization objective function that includes visual reprojection residuals, inertial state pre-integration residuals, loading arm joint kinematic residuals, spatial position constraint residuals when the interface is visible, and ship motion continuity residuals based on physical laws, to solve the motion state of the loading arm end and the ship's local reference frame at the current moment.
[0062] Furthermore, in one embodiment, in step 4, when the proportion of directly observable geometric features of the target interface is lower than a preset threshold, or when the overlap between the projection of the docking mechanism at the end of the loading arm in the sensor's field of view and the projection of the target interface is greater than a preset ratio, it is determined that the target interface has entered an obstruction state.
[0063] Once in the obstruction state, the non-contact sensing sensor continuously tracks the unobstructed deck, bulkhead, pipeline support, or railing structure around the target interface to maintain continuous measurement of the six degrees of freedom motion of the ship's local reference frame, and derives the pose of the target interface in real time through the fixed relative spatial constraint relationship.
[0064] Furthermore, in one embodiment, in step 2, the local cooperative positioning map is built entirely based on the ship's naturally existing structural texture, without the need to install any active or passive artificial markers, targets, encoders, or wireless communication positioning units on the ship's side.
[0065] Furthermore, in step 2, during the construction of the local cooperative localization map, the local reference frame of the ship is used as the local coordinate system by adopting a graph optimization-based SLAM algorithm. By performing sliding window keyframe optimization on historical observation frames, a factor graph containing spatial point poses and camera poses is constructed. Gravity direction constraints and inertial prior residuals are added when updating map points to suppress long-term drift.
[0066] Furthermore, in one embodiment, in step 3, the motion measurement data of the loading arm includes angle sensor data, length sensor data and joint speed feedback of each active joint. By establishing a forward kinematic model of the fluid multi-link mechanism of the loading arm, the predicted joint pose of the loading arm end relative to the dock base is calculated.
[0067] Furthermore, in one embodiment, in step 5, the closed-loop drive control adopts a composite control law that combines feedforward and feedback. The feedback control term is calculated based on real-time deviation using PID or sliding mode control, while the feedforward control term is compensated based on the multi-degree-of-freedom ship follow-up components predicted in the future short period according to the ship's historical motion sequence, so as to offset the motion lag of the ship under wave surge in advance.
[0068] Preferably, in one embodiment, the method includes the following steps:
[0069] S1, after the unloading arm enters the ship's manifold area, under the condition that the ship's manifold flange can be observed, the initial pose of the ship's manifold flange relative to the unloading arm is obtained, and the ship's local map coordinate system following the ship is determined; the unloading arm is equipped with a quick connection device or a guide connection device, and is equipped with a vision camera and an inertial measurement unit.
[0070] Here, the initial pose of the ship's manifold flange is obtained through one or more of the following geometric features or methods: flange outer circle, inner circle, bolt hole array, sealing surface edge, or manually confirmed position; the initial pose is used to determine the initial position of the ship's manifold flange in the local visual SLAM map, so as to establish a fixed transformation matrix between the ship's manifold flange coordinate system and the ship's local map coordinate system.
[0071] Here, the visual camera and the inertial measurement unit are only arranged on the side of the unloading arm. It is not required to install any QR codes, reflectors, coded targets, active light-emitting targets, auxiliary inertial measurement units or wireless positioning devices on the flanges, decks or bulkheads of the LNG ship, so as to achieve zero-modification berthing and docking at the ship end.
[0072] S2, around the ship's manifold flange, select a natural structure that is rigidly connected to the hull as a map reference; extract visual features from the natural structure and perform stable structure screening to construct a local visual SLAM map with the ship as a rigid body reference, and record the fixed relative position of the ship's manifold flange in the local visual SLAM map.
[0073] Here, the natural structures rigidly connected to the hull include one or more of the following: deck, bulkhead, manifold support, valve assembly, pipeline, railing, hatch coaming, drip tray, or fixed support near the flange.
[0074] Here, the stable structure selection when constructing the local visual SLAM map includes one or more of the following methods:
[0075] Based on continuous frame geometric consistency screening: if the relative position of a certain visual feature in multiple frames of images does not satisfy the same rigid body transformation constraint, it is judged as an unstable feature and is removed.
[0076] Based on semantic segmentation filtering: identify and mask or reduce the feature weights of areas where the sea surface, sky, people, cables, hose swinging, low-temperature steam, reflective water surface, temporary tools or mobile equipment are located;
[0077] Based on observation direction filtering: prioritize retaining lateral structural features that can still be observed from the approach path of the unloading arm end, and reduce the weight of features located directly in front of the flange that are easily obscured;
[0078] Uncertainty-based screening: For map points whose reprojection residuals are greater than a preset threshold, whose parallax is insufficient, whose textures are repetitive, or whose depth estimation is unstable, their covariance is increased or they are removed from the local visual SLAM map.
[0079] Here, in the local visual SLAM map, each visual map point stores its three-dimensional coordinates, descriptor, observation direction, scale level, and uncertainty in the local map coordinate system of the ship.
[0080] S3, during the docking guidance process, the motion data of the unloading arm end, the inertial measurement data and the local visual SLAM map are combined for joint estimation, and the pose change of the ship's local map coordinate system relative to the unloading arm end or the dock base is solved in real time, and the real-time relative pose deviation of the unloading arm end relative to the ship's manifold flange is calculated.
[0081] Here, a visual SLAM tracking module is used to match the fixed natural structural features of the ship's hull in the current image with the local visual SLAM map, and pose estimation is performed by combining inertial measurement unit data and unloading arm joint motion data; the unloading arm joint motion data is provided by the unloading arm encoder, hydraulic cylinder displacement sensor or angle sensor, and the pose estimation is used to describe the continuous movement of the ship's manifold area relative to the end of the unloading arm.
[0082] Here, the joint estimation adopts a visual-inertial-kinematic joint optimization model in the moving hull reference frame. Within the sliding window, the state variable X to be estimated includes at least:
[0083]
[0084] In the formula, For the first Coordinate system of the unloading arm end Relative to the dock base coordinate system The position, For the first Ship local coordinate system at any time Relative to the dock base coordinate system The position, The velocity or motion state of the ship in its local coordinate system. For inertial measurement unit bias, This is a fixed transformation from the ship's local coordinate system to the flange coordinate system. For the first Map points of the ship's natural hull structure.
[0085] Here, the objective function of the joint optimization model includes one or more of the following residual terms:
[0086] Visual reprojection residual: formed by current image feature observations and the local visual SLAM map points;
[0087] Inertial pre-integration residual: used to constrain the high-frequency motion of the camera or the end of the unloading arm at adjacent time points;
[0088] Kinematic residuals of the unloading arm joint: calculated based on joint angles, hydraulic cylinder displacements, or encoder data;
[0089] Flange position constraint residual: used to constrain the fixed transformation when the ship's manifold flange is visible. ;
[0090] Ship motion continuity residual: used to constrain the six-degree-of-freedom continuous motion of a berthed ship over a short period of time.
[0091] S4. When the end of the unloading arm or its docking mechanism obstructs the ship's manifold flange, the current pose of the ship's local map coordinate system is estimated using the unobstructed hull fixed natural structural features in the local visual SLAM map, and the current pose of the ship's manifold flange is calculated based on the fixed relative position relationship.
[0092] Here, in S4, the ship's manifold flange is determined to be in a blocked state by one or more of the following methods:
[0093] The visible outline of the ship's manifold flange is reduced and falls below a set proportion threshold.
[0094] The number of effective visual observation points of the ship's manifold flange is lower than a preset threshold.
[0095] The projection area of the end of the unloading arm or the quick-connect device on it coincides with the projection area of the flange.
[0096] Here, in S4, after entering the occlusion state, the local visual SLAM map continues to be tracked by the visual tracking module through the unoccluded side railings, valve groups, supports, bulkhead edges, or drip tray boundaries. The system solves the current pose of the ship's local map coordinate system through multi-point perspective projection, visual inertial optimization, or local bundle adjustment, and, combined with the fixed relative position relationship of the flange in the map, back-calculates the current pose of the occluded flange to maintain closed-loop estimation of relative pose during the occlusion period.
[0097] S5, output the relative pose deviation obtained from S3 or S4 to the unloading arm control system, and the unloading arm control system drives the end of the unloading arm to complete centering, approach and connection.
[0098] In one embodiment, a vision SLAM-based automated docking system for large marine LNG unloading arms is provided, the system comprising:
[0099] A non-contact sensing module, mounted at the end of the loading arm, is used to collect structural features and image point cloud data of the target interface and the surrounding ship environment;
[0100] An inertial measurement module, mounted on the end of the loading and unloading arm, is used to collect high-frequency triaxial angular velocity and triaxial acceleration data;
[0101] The joint state sensing module is distributed in each joint of the loading arm and is used to acquire real-time geometric deformation and pose measurement data of the loading arm.
[0102] A multi-source fusion processor is used to execute the automatic docking method for large marine LNG unloading arms based on visual SLAM, construct a cooperative motion map, calculate the relative pose deviation, and continuously calculate the interface pose in the non-line-of-sight state of the target interface.
[0103] The control drive controller is used to receive the pose deviation command output by the multi-source fusion processor and drive the actuators of each joint of the loading and unloading arm in a closed loop to complete fully automatic docking and locking.
[0104] Specific limitations regarding the vision-SLAM-based automated docking system for large marine LNG unloading arms can be found in the above description of the limitations on the vision-SLAM-based automated docking method for large marine LNG unloading arms, and will not be repeated here. Each module in the aforementioned vision-SLAM-based automated docking system for large marine LNG unloading arms can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.
[0105] In one embodiment, a vision SLAM-based automated docking device for a large marine LNG unloading arm is provided, the device comprising:
[0106] The first module is used to: after the loading arm enters the pipeline interface area of the ship, use a non-contact sensing sensor mounted on the loading arm to obtain the initial relative pose of the target interface with respect to the loading arm, and determine the ship local reference frame that follows the ship; the loading arm is also equipped with an inertial measurement unit.
[0107] The second module is used to: detect and extract various types of natural geometric features that are rigidly connected to the hull around the target interface as map references, construct a local cooperative positioning map with the ship as a rigid body reference and following the ship's motion, and lock the fixed relative spatial constraint relationship of the target interface in the local cooperative positioning map.
[0108] The third module is used to: during the docking guidance process, perform spatiotemporal multi-source fusion estimation of the loading arm motion measurement data, the loading arm end inertial measurement data, and the local cooperative positioning map; calculate the pose change of the ship's local reference frame relative to the loading arm end or the dock base in real time; and calculate the relative pose deviation of the loading arm end relative to the target interface in real time based on the fixed relative spatial constraint relationship.
[0109] The fourth module is used to: when the end of the loading arm or the docking mechanism it carries obstructs the target interface, causing the non-contact sensing sensor to be unable to directly observe the complete features of the target interface, continuously extract and match the remaining rigid natural geometric features of the hull that are not obstructed in the local cooperative positioning map, continue to estimate the real-time pose of the local reference system of the ship, and inversely calculate the real-time pose of the obstructed target interface according to the fixed relative spatial constraint relationship;
[0110] The fifth module is used to output the calculated relative pose deviation to the loading arm control system, drive the loading arm end to perform centering, approximation and physical locking, and realize closed-loop drive control.
[0111] Specific limitations regarding the automatic docking device for large marine LNG unloading arms based on vision SLAM can be found in the limitations of the automatic docking method for large marine LNG unloading arms based on vision SLAM mentioned above, and will not be repeated here. Each module in the aforementioned automatic docking device for large marine LNG unloading arms based on vision SLAM can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.
[0112] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements:
[0113] Step 1: After the loading arm enters the ship's pipeline interface area, the initial relative pose of the target interface with respect to the loading arm is obtained using a non-contact sensing sensor mounted on the loading arm, and the ship's local reference frame following the ship is determined accordingly; the loading arm is also equipped with an inertial measurement unit.
[0114] Step 2: Around the target interface, detect and extract various types of natural geometric features that are rigidly connected to the hull as map references, construct a local cooperative positioning map with the ship as a rigid body reference and following the ship's motion, and lock the fixed relative spatial constraint relationship of the target interface in the local cooperative positioning map.
[0115] Step 3: During the docking guidance process, the motion measurement data of the loading arm, the inertial measurement data of the loading arm end, and the local cooperative positioning map are fused and estimated in a spatiotemporal manner. The pose change of the ship's local reference frame relative to the loading arm end or the dock base is calculated in real time. Based on the fixed relative spatial constraint relationship, the relative pose deviation of the loading arm end relative to the target interface is calculated in real time.
[0116] Step 4: When the end of the loading arm or the docking mechanism it carries obstructs the target interface, causing the non-contact sensing sensor to be unable to directly observe the complete features of the target interface, the remaining rigid natural geometric features of the hull that are not obstructed in the local cooperative positioning map are continuously extracted and matched, the real-time pose of the local reference system of the ship is estimated, and the real-time pose of the obstructed target interface is calculated in reverse according to the fixed relative spatial constraint relationship.
[0117] Step 5: Output the calculated relative pose deviation to the loading arm control system to drive the loading arm end to perform centering, approximation and physical locking, thereby realizing closed-loop drive control.
[0118] For specific limitations on each step, please refer to the limitations on the automatic docking method for large marine LNG unloading arms based on vision SLAM mentioned above, which will not be repeated here.
[0119] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program being implemented when executed by a processor:
[0120] Step 1: After the loading arm enters the ship's pipeline interface area, the initial relative pose of the target interface with respect to the loading arm is obtained using a non-contact sensing sensor mounted on the loading arm, and the ship's local reference frame following the ship is determined accordingly; the loading arm is also equipped with an inertial measurement unit.
[0121] Step 2: Around the target interface, detect and extract various types of natural geometric features that are rigidly connected to the hull as map references, construct a local cooperative positioning map with the ship as a rigid body reference and following the ship's motion, and lock the fixed relative spatial constraint relationship of the target interface in the local cooperative positioning map.
[0122] Step 3: During the docking guidance process, the motion measurement data of the loading arm, the inertial measurement data of the loading arm end, and the local cooperative positioning map are fused and estimated in a spatiotemporal manner. The pose change of the ship's local reference frame relative to the loading arm end or the dock base is calculated in real time. Based on the fixed relative spatial constraint relationship, the relative pose deviation of the loading arm end relative to the target interface is calculated in real time.
[0123] Step 4: When the end of the loading arm or the docking mechanism it carries obstructs the target interface, causing the non-contact sensing sensor to be unable to directly observe the complete features of the target interface, the remaining rigid natural geometric features of the hull that are not obstructed in the local cooperative positioning map are continuously extracted and matched, the real-time pose of the local reference system of the ship is estimated, and the real-time pose of the obstructed target interface is calculated in reverse according to the fixed relative spatial constraint relationship.
[0124] Step 5: Output the calculated relative pose deviation to the loading arm control system to drive the loading arm end to perform centering, approximation and physical locking, thereby realizing closed-loop drive control.
[0125] For specific limitations on each step, please refer to the limitations on the automatic docking method for large marine LNG unloading arms based on vision SLAM mentioned above, which will not be repeated here.
[0126] As a specific example, in one embodiment, the invention will be further described with reference to the accompanying drawings.
[0127] This embodiment provides an automatic docking method for horizontal loading arms based on multi-sensor fusion. Figure 1 The diagram shown is the overall flowchart of this method, which includes the following specific implementation steps:
[0128] 1. System Composition
[0129] The large LNG unloading arm for ships in this embodiment is installed at an LNG receiving terminal terminal and includes a column, inner arm, outer arm, three-dimensional joint, cryogenic rotary joint, emergency disengagement device, quick-connect device, hydraulic drive mechanism, joint angle measurement unit, and control system. An explosion-proof vision camera and inertial measurement unit are installed at the end of the unloading arm near the quick-connect device. Existing joint encoders and hydraulic cylinder displacement sensors on the unloading arm are used to provide kinematic constraints for the end effector in the terminal base coordinate system.
[0130] like Figure 2 As shown, for ease of explanation, the following coordinate system is defined:
[0131] Wharf base coordinate system 1. Fixed to the unloading arm column or wharf foundation;
[0132] Coordinate system at the end of the unloading arm 2. Fixed to a quick-connect device or a guide connection device;
[0133] Camera coordinate system 3. Fixed to the vision camera;
[0134] Inertial Measurement Unit Coordinate System 4. Fixed to the inertial measurement unit;
[0135] Ship manifold flange coordinate system 5. Fix to the end face of the target manifold flange.
[0136] Ship local coordinate system 6. A local rigid structure fixed to the ship's manifold area;
[0137] in, As the ship undergoes six degrees of freedom of motion relative to the dock, Compared to Keep fixed. Compared to The change is driven by the hydraulic joint of the unloading arm. This invention provides a real-time solution. Compared to Positional deviation.
[0138] 2. Obtaining the initial flange position
[0139] After the unloading arm moves from its parking or standby position to the vicinity of the ship's manifold area, the vision camera acquires images of the manifold area. The system can use conventional image recognition methods to determine the flange candidate area, or it can use geometric detection methods based on flange circular or elliptical projections, bolt hole arrays, or sealing surface edges to identify the flange.
[0140] After obtaining the candidate regions, the system performs a fine-fit on the flange end face. For the visible outer and inner circles, robust ellipse fitting is used to obtain the end face projection; for the visible bolt hole array, the flange attitude is corrected using the hole position circumferential distribution constraint; for the partially obscured sealing surface, RANSAC or least squares optimization can be used to recover the flange end face circle. Combining camera intrinsic parameters, standard flange dimensions, and camera-end hand-eye calibration matrix, the coordinate system of the ship manifold flange is calculated. Relative to camera coordinate system The initial pose.
[0141] The flange identification described above is only used to determine the initial position of the flange in the local visual SLAM map. The local map around the flange at the initial moment can be defined as the ship's local coordinate system. Then record the flange coordinate system. Compared to Fixed transformation During the docking process, In the ship's local coordinate system The internal structure remains unchanged.
[0142] 3. Construction of maps showing the natural features of the ship's fixed hull
[0143] like Figure 3 As shown, after initial capture of the flange, the system extracts natural visual features from the image area within a predetermined range around the flange. Preferred features include manifold support edges, valve assembly outlines, fixed pipeline edges, railing intersections, bulkhead corners, hatch coaming edges, drip tray boundaries, fixed support textures, deck anti-slip textures, and fixed nameplates or structural corners near the flange.
[0144] To avoid including non-rigid structures in the map, the system performs static structure filtering. Specifically, it uses one or more of the following methods:
[0145] (1) Screening based on geometric consistency of consecutive frames. If the relative position of a visual feature in multiple frames does not satisfy the same rigid body transformation, it is judged as an unstable feature and removed.
[0146] (2) Based on semantic segmentation filtering. Reduce or block the weight of areas such as sea surface, sky, personnel, cables, hoses, low temperature steam, reflective water surface, temporary tools and mobile equipment.
[0147] (3) Screening based on observation direction. Prioritize retaining lateral structural features that can still be observed from the approach path of the unloading arm end, and reduce the weight of features located directly in front of the flange that are easily obscured by the quick-connect device.
[0148] (4) Uncertainty-based screening. Set a larger covariance or delete map points with large reprojection residuals, insufficient parallax, repetitive textures, or unstable depth estimation.
[0149] Each visual map point in the map stores its local coordinates within the ship's local coordinate system. The system stores the three-dimensional coordinates, descriptors, observation directions, scale levels, and uncertainties. It also preserves the flange coordinate system. With ship local coordinate system The relative positional relationships between them are thus established. This forms a local visual SLAM map with reference to the ship's natural structure.
[0150] 4. Visual SLAM Joint Estimation in Moving Hull Reference Frame
[0151] Traditional SLAM typically assumes the environment map is stationary relative to the ground, while this invention observes a moored ship, whose local map is constantly moving relative to the dock pier. Therefore, this invention employs a joint estimation model under a moving ship reference frame. Figure 4 As shown,
[0152] Within the sliding window, the state to be estimated can be represented as:
[0153]
[0154] in, For the first Coordinate system of the unloading arm end Relative to the dock base coordinate system The position, For the first Ship local coordinate system at any time Relative to the dock base coordinate system The position, The velocity or motion state of the ship in its local coordinate system. For inertial measurement unit bias, This is a fixed transformation from the ship's local coordinate system to the flange coordinate system. For the first Map points of the ship's natural hull structure.
[0155] The optimization objectives include visual reprojection residuals, inertial pre-integration residuals, unloading boom joint kinematic residuals, flange position constraint residuals, and ship motion continuity residuals.
[0156]
[0157] Among them, visual reprojection residual Formed from feature observations in the current image and local visual SLAM map points; inertial pre-integration residuals Constraining high-frequency motion of the camera or end effector at adjacent time points; kinematic residuals of the unloading arm joints. Calculated from joint angle, hydraulic cylinder displacement, or encoder; flange position constraint residual. Constraints when flange is visible Ship motion continuity residuals Constraining the continuous changes in the six degrees of freedom of motion of a berthed vessel over a short period of time.
[0158] Based on the coordinate transformation relationship, the position of the ship's manifold flange relative to the end of the unloading arm can be obtained by the following formula:
[0159]
[0160] in, For the first Coordinate system of the unloading arm end Relative to the dock base coordinate system The position, For the first Ship local coordinate system at any time Relative to the dock base coordinate system The position, This is a fixed transformation from the ship's local coordinate system to the flange coordinate system.
[0161] control system The position deviation, attitude deviation, relative velocity, and uncertainty obtained from the decomposition are used as docking guidance inputs.
[0162] 5. Closed-loop pose maintenance during the flange blocking phase
[0163] As the unloading arm approaches the ship's manifold flange, quick-connect devices, guide devices, or end pipe sections may obstruct the flange face in the camera's field of view. The system can determine whether it has entered a flange obstruction state based on information such as the visible profile ratio of the flange, the number of effective observation points, and the projection position of the quick-connect device.
[0164] Once the system enters the obscured state, it no longer requires the flange face to be visible, but instead continues to track the unobstructed natural structural features of the hull surrounding the flange. For example, even after the quick-connect device obscures the flange front, railings, valve assemblies, supports, bulkhead edges, or drip tray boundaries located on the upper side can still be seen by the camera. Since these structures are all fixed to the same rigid hull as the flange, their... Coordinates and Fixed Transformations Together, they determine the current position of the obstructed flange.
[0165] When multiple map points are visible, the system solves the problem using PnP, visual-inertial optimization, or local BA. , and by Estimation If some map points are lost due to occlusion or reflection, the system adjusts the weights based on the uncertainty of the map points and the observation direction; if the flange becomes visible again, the direct observation of the flange is added back to the positioning calculation to correct for drift during the occlusion period.
[0166] In this way, the docking of the unloading arm end no longer relies on the open-loop assumption that "the flange does not change much in a short period of time", but maintains the closed-loop estimation of the relative position and orientation of the flange under continuous observation of the ship's fixed natural structure.
[0167] 6. Interface control
[0168] After the unloading arm tip enters the guide capture zone or the quick-connect device pre-connection zone, the visual SLAM module continues to output the relative pose deviation between the unloading arm tip and the ship's manifold flange. This output can be used by the unloading arm control system to control the tip to complete alignment, approach, and connection.
[0169] Specifically, the visual SLAM module provides the control system with the following information: current relative pose deviation, number of visible map points, and tracking status. Based on this information, the control system can choose to continue advancing, decelerate, maintain position, or re-search for the target.
[0170] In this embodiment, the visual SLAM module is only responsible for providing reliable relative pose and map tracking status, and does not limit the specific control law used at the end of the unloading arm. The unloading arm control system can use position servo, speed limiting, impedance control, or other existing control methods.
[0171] This interface design enables the visual SLAM positioning method of this invention to be connected to different models of large LNG unloading arms, without relying on a specific controller.
[0172] 7. Implementation Results
[0173] After adopting the method of this invention, the ship manifold flange is no longer just a single-frame visual detection target, but becomes a fixed-position object in the hull-attached visual SLAM map. Even if the flange is obscured by the quick-connect device during the end approach phase, the system can still continuously calculate its pose through the surrounding unobstructed hull structure, so that the visual positioning information of the large LNG unloading arm end docking is maintained.
[0174] Compared to solutions that require the installation of targets or tracking devices on the ship's side, this invention does not change the ship's side equipment and operational procedures, and the deployment boundaries are clear. Compared to solutions that only use monocular, binocular, laser, or ultrasonic direct ranging, this invention can continue to estimate targets when the flange is not visible, and uses the natural structure of the ship's manifold area as a continuous positioning reference.
[0175] This invention can be used for the automatic docking of LNG liquid phase unloading arms, gas phase return arms, and other large ship-to-shore loading and unloading arms for cryogenic or hazardous media. It can also be integrated with existing unloading arm electrical control systems, quick connection devices, emergency disengagement systems, and dock safety interlocking systems.
[0176] This invention eliminates the need for any additional equipment to be installed on the ship's end. It uses visual SLAM for docking with moving targets and for continuous positioning after obstruction, effectively reducing blind docking at the end and improving the stability and safety of ship-to-shore docking.
[0177] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention without departing from its spirit and scope should be included within the protection scope of the present invention.
Claims
1. An automatic docking method for a large marine LNG unloading arm based on visual SLAM, characterized in that, The method includes: Step 1: After the loading arm enters the ship's pipeline interface area, the initial relative pose of the target interface with respect to the loading arm is obtained using a non-contact sensing sensor mounted on the loading arm, and the ship's local reference frame following the ship is determined accordingly; the loading arm is also equipped with an inertial measurement unit. Step 2: Around the target interface, detect and extract various types of natural geometric features that are rigidly connected to the hull as map references, construct a local cooperative positioning map with the ship as a rigid body reference and following the ship's motion, and lock the fixed relative spatial constraint relationship of the target interface in the local cooperative positioning map. Step 3: During the docking guidance process, the motion measurement data of the loading arm, the inertial measurement data of the loading arm end, and the local cooperative positioning map are fused and estimated in a spatiotemporal manner. The pose change of the ship's local reference frame relative to the loading arm end or the dock base is calculated in real time. Based on the fixed relative spatial constraint relationship, the relative pose deviation of the loading arm end relative to the target interface is calculated in real time. Step 4: When the end of the loading arm or the docking mechanism it carries obstructs the target interface, causing the non-contact sensing sensor to be unable to directly observe the complete features of the target interface, the remaining rigid natural geometric features of the hull that are not obstructed in the local cooperative positioning map are continuously extracted and matched, the real-time pose of the local reference system of the ship is estimated, and the real-time pose of the obstructed target interface is calculated in reverse according to the fixed relative spatial constraint relationship. Step 5: Output the calculated relative pose deviation to the loading arm control system to drive the loading arm end to perform centering, approximation and physical locking, thereby realizing closed-loop drive control.
2. The automatic docking method for large marine LNG unloading arms based on visual SLAM according to claim 1, characterized in that, The initial relative pose of the target interface with respect to the end of the loading arm in step 1 is obtained by fitting and solving one or more geometric topological constraints in the target interface's end face circular contour, internal aperture contour, fastening hole array distribution, end face outer edge, and manually assisted designated area, in order to establish the initial pose anchor point of the target interface in the local cooperative positioning map.
3. The automatic docking method for large marine LNG unloading arms based on visual SLAM according to claim 1, characterized in that, Step 2 in constructing the local collaborative positioning map includes: performing dynamic and non-rigid interference filtering on the acquired images or 3D point clouds, retaining only stable environmental features that have a rigid connection with the hull, and removing time-varying interference features that do not meet rigid body consistency. The time-varying interference features include at least sea surface reflection, sky, moving personnel, swinging of cables and hoses, low-temperature medium vapor, drastic changes in light and shadow, and temporary mobile equipment; The local cooperative positioning map records the three-dimensional coordinates of each feature point in the ship's local reference frame, local feature descriptors, multi-angle observation weights, and measurement uncertainties.
4. The automatic docking method for large marine LNG unloading arms based on visual SLAM according to claim 1, characterized in that, Step 3 employs a local joint estimation framework to match the current environmental features collected by the non-contact sensing sensor with the local cooperative localization map. A joint optimization objective function is established, which includes visual reprojection residuals, inertial state pre-integration residuals, loading arm joint kinematic residuals, spatial position constraint residuals when the interface is visible, and ship motion continuity residuals based on physical laws. The motion state of the loading arm end and the ship's local reference frame at the current moment is calculated.
5. The automatic docking method for large marine LNG unloading arms based on visual SLAM according to claim 1, characterized in that, In step 4, when the proportion of directly observable geometric features of the target interface is lower than a preset threshold, or when the overlap between the projection of the docking mechanism at the end of the loading arm in the sensor field of view and the projection of the target interface is greater than a preset ratio, it is determined that the target interface has entered an obstruction state. Once in the obstruction state, the non-contact sensing sensor continuously tracks the unobstructed deck, bulkhead, pipeline support, or railing structure around the target interface to maintain continuous measurement of the six degrees of freedom motion of the ship's local reference frame, and derives the pose of the target interface in real time through the fixed relative spatial constraint relationship.
6. The automatic docking method for large marine LNG unloading arms based on visual SLAM according to claim 1, characterized in that, In step 2, the local collaborative positioning map is built entirely based on the ship's naturally existing structural texture, without the need to install any active or passive artificial markers, targets, encoders, or wireless communication positioning units on the ship's side. In step 2, during the construction of the local cooperative localization map, the local reference frame of the ship is used as the local coordinate system by adopting a graph optimization-based SLAM algorithm. By performing sliding window keyframe optimization on historical observation frames, a factor graph containing spatial point poses and camera poses is constructed. Gravity direction constraints and inertial prior residuals are added when updating map points to suppress long-term drift.
7. The automatic docking method for large marine LNG unloading arms based on visual SLAM according to claim 1, characterized in that, In step 3, the motion measurement data of the loading and unloading arm includes angle sensor data, length sensor data and joint speed feedback of each active joint. By establishing a forward kinematic model of the fluid multi-link mechanism of the loading and unloading arm, the predicted joint pose of the loading and unloading arm end relative to the dock base is calculated.
8. The automatic docking method for large marine LNG unloading arms based on visual SLAM according to claim 1, characterized in that, In step 5, the closed-loop drive control adopts a composite control law that combines feedforward and feedback. The feedback control term is calculated based on real-time deviation using PID or sliding mode control, while the feedforward control term is compensated based on the multi-degree-of-freedom ship follow-up components predicted in the future short period according to the ship's historical motion sequence, so as to offset the motion lag of the ship under wave surge in advance.
9. An automated docking system for a large marine LNG unloading arm based on vision SLAM, characterized in that, The system includes: A non-contact sensing module, mounted at the end of the loading arm, is used to collect structural features and image point cloud data of the target interface and the surrounding ship environment; An inertial measurement module, mounted on the end of the loading and unloading arm, is used to collect high-frequency triaxial angular velocity and triaxial acceleration data; The joint state sensing module is distributed in each joint of the loading and unloading arm and is used to acquire real-time geometric deformation and pose measurement data of the loading and unloading arm. A multi-source fusion processor is used to perform the method as described in any one of claims 1 to 8, construct a cooperative motion map, calculate relative pose deviations, and continuously calculate the interface pose in a non-line-of-sight state of the target interface. The control drive controller is used to receive the pose deviation command output by the multi-source fusion processor and drive the actuators of each joint of the loading and unloading arm in a closed loop to complete fully automatic docking and locking.
10. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method according to any one of claims 1 to 8.
