Visual image-guided float-over installation automation system and method of implementing the same

Abstract

The application provides a visual image guided floatation installation automation system and an implementation method thereof. The method comprises the following steps: acquiring a plurality of original RGB image frames and a disturbance state vector to generate an enhanced image frame; acquiring current frame sparse point cloud data; calculating a linear difference of a space posture of floatation in a current frame and a previous frame to generate a floatation predicted posture, and constructing a control compensation amount according to a difference between the floatation predicted posture and a target docking posture; acquiring a response state of each propulsion dimension, combining the disturbance state vector and the compensation control amount to generate a final control instruction for driving a floatation propulsion system; and combining the space posture of the floatation in the current frame, the final control instruction and the target docking posture to determine whether the floatation has completed stable docking of the target posture. The application not only guarantees real-time adaptation of the system to a complex environment, but also improves response accuracy of a dynamic process, so that stable, efficient and intelligent control of the whole floatation installation process is realized.

Description

Technical Field

[0001] This invention belongs to the field of visual imaging, and in particular relates to a visual image-guided automated floating installation system and its implementation method. Background Technology

[0002] In marine engineering construction, float-over installation, as a key technology for the efficient, economical, and suitable docking of ultra-large offshore structures, has been widely applied in the placement of deep-water platforms, modular decks, and oil and gas processing facilities. This technology significantly reduces reliance on large lifting equipment by controlling the berthing and precise docking of large structures under buoyancy. However, traditional float-over installation processes heavily rely on human experience, especially during berthing and positioning, requiring on-site personnel to rely on visual observation and manual control to adjust attitude and guide the path. This reliance on human judgment faces significant challenges in complex sea conditions. On the one hand, surface reflections, sudden changes in lighting, tidal variations, and the float's own sway can severely interfere with the accuracy and responsiveness of human vision; on the other hand, float-over structures are typically enormous and slow to respond, and even slight errors can lead to docking deviations or even collisions, directly impacting project safety and efficiency.

[0003] To improve the automation of the float-over installation process, some studies in recent years have attempted to introduce image recognition-based auxiliary systems. These systems use cameras to capture on-site video streams and combine them with traditional image processing algorithms (such as edge detection and color matching) to identify and locate the docking area. However, such systems generally have the following limitations: First, they are mostly developed based on ideal working conditions and lack adaptability to dynamic environments (such as wind and wave disturbances, water surface glare, and low visibility). The image guidance effect is prone to failure in the actual marine engineering environment. Second, existing systems mostly use two-dimensional image information for target recognition and path planning, making it difficult to accurately recover the three-dimensional spatial relationship between the float-over and the target platform, especially subtle attitude information such as tilt, deflection, and gap, resulting in coarse and unstable docking actions. Third, in terms of control response mechanisms, mainstream methods still rely on the current state error for passive adjustments, which cannot cope with the control delay caused by the response lag of the float-over structure itself, thus affecting docking accuracy and system stability. It is evident that existing technologies have not yet formed a complete, reliable, and intelligent vision-guided automation solution for complex operation scenarios such as floating installation, which involves large structures, dynamic environments, and high precision requirements. Key shortcomings remain, such as easy failure of perception, insufficient spatial understanding capabilities, and slow control response, which urgently require systematic technological breakthroughs. Summary of the Invention

[0004] The purpose of this invention is to propose a visual image-guided floating installation automation system and its implementation method, which solves the core bottleneck of the current system's fragmented functional modules and the inability to unify scheduling strategies and distribution paths, and significantly improves the actual command and dispatch efficiency and reliability of the system in highly dynamic scenarios such as urban emergency response and government coordination.

[0005] To achieve the above objectives, a first aspect of the present invention provides a visual image-guided automated floating installation system, the system comprising:

[0006] The image enhancement module is used to acquire multiple original RGB image frames and perturbation state vectors, and dynamically adjust the original RGB image frames according to the perturbation state vectors to generate enhanced image frames; wherein the perturbation state vectors include the current ambient light intensity, camera angle perturbation, and platform acceleration perturbation intensity;

[0007] The 3D modeling module is used to acquire sparse point cloud data of the current frame; extract structural feature points of the enhanced image frame and back-project them into 3D spatial points; optimize the registration error function controlled by perturbation based on the 3D spatial points to generate the spatial attitude of the floating in the current frame; generate the spatial position of the 3D spatial points in the registered world coordinate system to form a new set of position points; register the new set of position points with the sparse point cloud data of the current frame to construct a joint point set; and generate a 3D structural model and target docking attitude based on the joint point set.

[0008] The path prediction and compensation module is used to calculate the linear difference between the spatial attitude of the floating and the previous frame to generate the floating predicted attitude, and to construct the control compensation amount based on the difference between the floating predicted attitude and the target docking attitude.

[0009] The control execution module is used to acquire the response status of each propulsion dimension, and combine the disturbance state vector and the compensation control quantity to generate the final control command to drive the buoyancy propulsion system.

[0010] The docking judgment and feedback module is used to determine whether the floating platform has completed a stable docking with the target attitude by combining the spatial attitude of the floating platform in the current frame, the final control command, and the target docking attitude. If it has not completed the docking, it feeds back the corresponding linear difference and the corresponding disturbance state vector, and returns it to the path prediction and compensation module to form a control closed loop. If it has completed the docking, the current round of work is completed.

[0011] In another aspect of the present invention, a method for implementing a visual image-guided automated floating installation system is provided, comprising the following steps:

[0012] S1. Acquire multiple original RGB image frames and perturbation state vectors, and dynamically adjust the original RGB image frames according to the perturbation state vectors to generate enhanced image frames; wherein the perturbation state vectors include the current ambient light intensity, camera angle perturbation, and platform acceleration perturbation intensity;

[0013] S2. Obtain the sparse point cloud data of the current frame; extract the structural feature points of the enhanced image frame and back-project them into three-dimensional spatial points; optimize the registration error function controlled by perturbation based on the three-dimensional spatial points to generate the floating spatial attitude of the current frame; generate the spatial position of the three-dimensional spatial points in the registered world coordinate system to form a new set of position points; register the new set of position points with the sparse point cloud data to construct a joint point set; generate a three-dimensional structural model and target docking attitude based on the joint point set.

[0014] S3. Calculate the linear difference between the spatial attitude of the floating and the previous frame to generate the floating prediction attitude, and construct the control compensation amount based on the difference between the floating prediction attitude and the target docking attitude.

[0015] S4. Obtain the response status of each propulsion dimension, combine the disturbance state vector and the compensation control quantity, and generate the final control command to drive the buoyancy propulsion system.

[0016] S5. Based on the current frame's floating spatial attitude, final control command, and target docking attitude, determine whether the floating has completed stable docking with the target attitude; if not, feed back the corresponding linear difference and the corresponding disturbance state vector, and return to S3 to form a control closed loop; if completed, the current round of work is completed.

[0017] Furthermore, the process involves acquiring multiple raw RGB image frames and perturbation state vectors, and obtaining environmental parameters through a light sensor, an IMU module, and an accelerometer; wherein,

[0018] The step of dynamically adjusting the original RGB image frame according to the perturbation state vector to generate an enhanced image frame specifically includes:

[0019] Based on the current ambient light intensity, the original RGB image frame is dynamically brightened to generate an intermediate image frame with adjusted brightness.

[0020] The intermediate image frame after brightness adjustment is subjected to image enhancement processing based on camera angle perturbation and platform acceleration perturbation intensity to generate an enhanced image frame.

[0021] Furthermore, the process of optimizing the registration error function based on the three-dimensional spatial points through perturbation control to generate the spatial attitude of the current frame's floating position specifically includes:

[0022] Structural feature points are extracted from the enhanced image frame, and points with significant edges and angle abrupt changes are identified as target anchor points using multi-scale Canny edge detection and Harris corner detection methods; the anchor points are back-projected into target three-dimensional space points using camera intrinsic and extrinsic parameters;

[0023] Based on the target three-dimensional spatial point, search for its nearest neighbor in the current frame sparse point cloud data to form an initial point pair set;

[0024] A registration error function is constructed based on the initial set of point pairs. ,as follows:

[0025] ;

[0026] Where i is point i, and M is the total number of points. As weight, For rotation matrix, For the target three-dimensional space point, It is a translation vector. As the nearest neighbor, This is the attitude regularization coefficient, used to control the suppression of continuity offset between the current attitude and the attitude of the previous frame. The default value is 0.5. The rotation matrix of the previous frame's floating element. The squared Frobenius norm between the current and previous frame rotation matrices measures the amount of rotation change. This is the translation vector of the previous frame's floating position. The weight is the squared Euclidean distance between the current position and the previous frame position; where, the weights are... Calculated using the following formula:

[0027] ;

[0028] in, It is a natural exponential function. To enhance image frames, Enhance the frame of the image At point The squared magnitude of the image gradient at a given point represents the edge strength of the structure at that point, and is calculated using the Sobel operator. This is the image gradient normalization factor. As a perturbation-driven enhancement factor, This is the adjustment coefficient for the disturbance-driven enhancement factor, with a default value of 0.5. For camera angle perturbation, The intensity of the platform acceleration disturbance;

[0029] Optimize the registration error function Obtain the rotation matrix of the buoyancy. With translation vector This allows us to construct the spatial attitude of the floating object in the current frame. ;in, The position in three-dimensional coordinates. The attitude angle;

[0030] Specifically, generating the three-dimensional structural model and target docking posture based on the joint point set includes:

[0031] A three-dimensional structural model is generated by reconstructing a mesh using the Poisson surface reconstruction method and a point set.

[0032] Based on the aforementioned three-dimensional structural model, the spatial attitude of the floating mechanism in the current frame is determined. The simulation was performed to generate the target docking attitude.

[0033] Furthermore, S2 also includes:

[0034] In the floating attitude prediction, a disturbance suppression coefficient is introduced to control the prediction amplitude. When the environmental disturbance is most severe, the prediction amplitude is automatically reduced to the minimum, ensuring the stability and reliability of the predicted attitude.

[0035] In the control compensation quantity, a control gain factor is also introduced to reflect the control response intensity that the system should take under different disturbance levels. The control response intensity is the highest when the vibration is the strongest, so as to quickly correct the floating attitude deviation.

[0036] Furthermore, the predicted buoyancy attitude is a six-dimensional vector; wherein,

[0037] The process of acquiring the response states of each propulsion dimension, combining the disturbance state vector and the compensation control quantity, and generating the final control command to drive the overflight propulsion system specifically includes:

[0038] Based on the predicted attitude, the perturbation state vector and compensation control quantity are used to generate an execution adjustment factor to control different channels (x, y, z, ...). , , The allocation of weights in actual implementation;

[0039] The execution adjustment factor is applied to the compensation control quantity to generate the final control command.

[0040] Furthermore, the final control command is expressed as:

[0041] ;

[0042] in, For the final control command, To implement the adjustment factor, To compensate for the control quantity, The value is a small-amplitude stabilization factor, empirically taken as: ;

[0043] in, It is the actual control command scaled according to the execution channel status; It is a soft constraint elastic repair item. When the execution adjustment factor of a certain channel is very small, it still allows a correction component with consistent sign and extremely small amplitude to be retained, so as to avoid control zero mutation leading to floating lag.

[0044] Furthermore, the step of determining whether the floating platform has completed stable docking with the target attitude by combining the spatial attitude of the floating platform in the current frame, the final control command, and the target docking attitude specifically includes:

[0045] The current attitude error vector is determined based on the spatial attitude of the floating vessel and the target docking attitude of the current frame.

[0046] A disturbance-sensitive tolerance function is constructed based on the current attitude error vector, and the docking is determined based on the disturbance-sensitive tolerance function; wherein the minimum value of the disturbance-sensitive tolerance function indicates that the target is closest and the system is most stable;

[0047] If the docking is not completed, the residual information at the time of the incomplete docking will be fed back to S3 to form a control closed loop.

[0048] Furthermore, the determination of whether docking is complete based on the disturbance sensitivity tolerance function specifically involves:

[0049] If the disturbance sensitivity tolerance function is less than 1.0 and continuously satisfies If a continuous control cycle is completed, the floating docking is considered successful, and a status flag is set. Otherwise set to It also feeds back the residual information when the docking is not completed to S3.

[0050] Furthermore, the residual information includes:

[0051] The difference between the current attitude error vector and the target docking attitude represents the area where control was not achieved;

[0052] Current final control command;

[0053] The current perturbation state vector.

[0054] The beneficial technical effects of the present invention are at least as follows:

[0055] This invention proposes a visual image-guided automated floating docking system for complex sea conditions. Through the systematic integration of image acquisition, 3D spatial modeling, and intelligent control mechanisms, it constructs an integrated solution with environmental adaptability, spatial perception, and feedforward decision-making capabilities. Firstly, this invention introduces an environment-driven image adaptation mechanism in the perception stage, enabling the system to automatically adjust image acquisition parameters based on interference factors such as real-time lighting and water surface reflection. Enhancement processing further improves feature clarity, ensuring the stability and accuracy of perception from the outset. Secondly, by constructing a spatial structure understanding module that integrates image and sparse point cloud information, this invention significantly improves the ability to recover the 3D relative position and attitude between the floating platform and the target structure, providing a spatial decision-making basis for precise docking. Furthermore, in terms of control strategy, this invention no longer relies on traditional passive error feedback but introduces a trend prediction method based on state sequences. This method combines the rigid body motion characteristics of the floating platform to predict future error trends and outputs feedforward control commands, achieving proactive compensation and adjustment of the floating path and attitude. The aforementioned key modules operate collaboratively in a closed-loop logic, which not only ensures the system's real-time adaptation to complex environments but also improves the response accuracy to dynamic processes, thereby achieving stable, efficient, and intelligent control over the entire floating installation process. Attached Figure Description

[0056] The present invention will be further described with reference to the accompanying drawings, but the embodiments in the drawings do not constitute any limitation on the present invention. For those skilled in the art, other drawings can be obtained based on the following drawings without creative effort.

[0057] Figure 1 This is a flowchart illustrating a method for implementing a visual image-guided automated floating installation system according to an embodiment of the present invention.

[0058] Figure 2 This is a framework diagram of a visual image-guided automated floating installation system according to an embodiment of the present invention. Detailed Implementation

[0059] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0060] like Figure 1 As shown in the figure, an embodiment of the present invention provides a method for implementing a visual image-guided automated floating installation system, comprising:

[0061] S1. Acquire multiple original RGB image frames and perturbation state vectors, and dynamically adjust the original RGB image frames according to the perturbation state vectors to generate enhanced image frames; wherein the perturbation state vectors include the current ambient light intensity, camera angle perturbation, and platform acceleration perturbation intensity.

[0062] Specifically, this step is responsible for generating an image frame with clear structure, balanced illumination, and suitable for feature extraction. It outputs the current environmental disturbance state vector. This module is used for subsequent modeling and control tasks. It addresses the dynamic visual disturbances encountered during the floating installation process (such as sea surface reflection, platform vibration, and lighting changes), introducing a disturbance-driven enhancement strategy. This strategy utilizes two channels: image exposure adjustment and structural texture enhancement, and employs regularization to prevent over-enhancement of unstructured areas. The entire system runs on an edge-embedded platform, with processing latency controlled within 50ms.

[0063] Further, obtain the following data:

[0064] Original RGB image frame (640×480 pixels), captured by the camera module, in pixel value (uint8).

[0065] : Current ambient light intensity, in lx (lux), obtained through TSL2591 light sensor; Camera angle disturbance (unit: °), calculated by the IMU six-axis module (mainly taking the pitch value); Platform acceleration disturbance intensity (unit: m / s²), obtained from acceleration sensors.

[0066] Furthermore, dynamic brightness adjustment and exposure control factor are performed first. Used to balance brightness:

[0067] ;

[0068] in, Target illumination reference value, set to 1000 lx; A tiny constant, set to 1 lx to prevent division by zero; Dimensionless exposure control factor; : Original image frame, in pixels; : The intermediate image frame after brightness adjustment, in pixels; : Pixel value cropping function, which limits the result to Within the range.

[0069] Next, structural enhancement is performed, introducing perturbation-driven enhancement terms and structural saliency regularization terms:

[0070] ;

[0071] in, The final enhanced image frame, showing the RGB values ​​of each pixel; Image values ​​after applying the Laplacian filter, used to highlight edges, in pixels; : Disturbance-driven enhancement factor (dimensionless), dynamically calculated based on real-time sensor output; : Structural regularization weight coefficient, dimensionless, empirically set to 0.2; : Structural saliency function, representing pixel Normalized variance of pixels within a 7x7 window, expressed in pixel value².

[0072] in, The calculation method is as follows:

[0073] ;

[0074] in, :image China and Israel Centered Pixel window; : The variance of pixel values ​​within the window. Total number of windows, used for normalization.

[0075] Final output: Enhanced image frames, data type uint8RGB, size 640×480, with clear structure and enhanced contrast; : Perturbation state vector, reserved for weight adjustment or image confidence processing in the modeling stage of the next module.

[0076] Understandably, this step achieves dynamic adaptive image enhancement with a minimal number of variables and low computational complexity, taking into account typical interference sources in floating installation scenarios. By driving the image enhancement coefficients with external perturbation variables, structural regions of the image are enhanced while smooth regions are preserved, thus improving support for subsequent structure recognition modules.

[0077] S2. Obtain the sparse point cloud data of the current frame; extract the structural feature points of the enhanced image frame and back-project them into three-dimensional spatial points; optimize the registration error function controlled by perturbation based on the three-dimensional spatial points to generate the floating spatial attitude of the current frame; generate the spatial position of the three-dimensional spatial points in the registered world coordinate system to form a new set of position points; register the new set of position points with the sparse point cloud data to construct a joint point set; generate a three-dimensional structural model and target docking attitude based on the joint point set.

[0078] Specifically, this step is based on the enhanced image output by S1. and perturbation state vector Combined with the current frame sparse point cloud data Construct a three-dimensional model of the floating structure relative to the installation platform. And estimate the current spatial attitude of the buoy. In floating installation scenarios, point cloud data is sparse, heavily occluded, and the image structure is easily disturbed. This step proposes an image structure-guided sparse point cloud modeling method, which adjusts the participation of image features by perturbation state and fuses the image and point cloud to generate high-confidence structural modeling results.

[0079] Further, obtain the input:

[0080] Enhanced image frames, output by S1;

[0081] : Disturbance state vector, from environmental sensors;

[0082] : Sparse point cloud data of the current frame acquired by the structured light module;

[0083] Camera intrinsic parameter matrix;

[0084] : The extrinsic parameter matrix between the camera and the structured light module.

[0085] Furthermore, firstly, from the image Extracting structural feature points We identified points with significant edges and abrupt angle changes as anchor points using multi-scale Canny edge detection and Harris corner detection methods. Then, we used camera intrinsics... and external references Backproject these points into three-dimensional space points ;

[0086] Furthermore, for each In the current frame of sparse point cloud data Search for its nearest neighbor. This forms the initial set of point pairs. The registration error function for disturbance control is constructed as follows:

[0087] ;

[0088] Among them, weight Calculated using the following formula:

[0089] ;

[0090] Optimize the objective function to obtain the rotation matrix of the buoyancy. With translation vector This allows us to construct the spatial attitude of the floating object in the current frame. ;in, The position is represented by three-dimensional coordinates (in meters). The attitude angle is expressed in degrees.

[0091] Furthermore, the key integration process for the structural modeling part is as follows:

[0092] First, the 3D projections of all the points in the 3D space are registered and transformed to form a new set of location points:

[0093] , where is the spatial position of a three-dimensional point in the registered world coordinate system.

[0094] Then, the current frame sparse point cloud data Pairs of points that are similar to these image structural points It was reserved as a high-confidence reference point.

[0095] The two types of points are then merged in the following way:

[0096] Constructing a joint point set ,in ;

[0097] right Density estimation is performed on all points in the region if there are only image projection points. But the surroundings If the point cloud is sparse, mark the point as an "image completion point";

[0098] The point set was reconstructed using the PoissonSurface Reconstruction method. Reconstruct the mesh and generate a 3D structural model. ;

[0099] Adding normal vector information (estimated through image gradient direction) to image points enhances reconstruction quality;

[0100] The reconstruction process is implemented using the PCL library and libigl, running on an embedded platform (such as Jetson AGX), with a single frame processing time of approximately 120ms.

[0101] Understandably, the key to this structural fusion mechanism lies in: the original sparse point cloud data It was acquired using structured light, resulting in uneven density, and the image guide points... Obtained through high-confidence image structure points, providing structural completion capabilities. Through joint reconstruction, It has a more complete structural outline and stronger edge consistency, making it suitable for subsequent path planning and obstacle avoidance.

[0102] Final output: The three-dimensional structural model is reconstructed by fusing image structural anchor points with point clouds, and the format is a point cloud mesh structure. The spatial attitude of the current frame's floating motion is represented as: , used in the subsequent path prediction module;

[0103] Understandably, this step proposes a complete image-point cloud fusion 3D modeling workflow: first, extract image structural features; then, project and register them to construct high-confidence point pairs; control the point pair weights through perturbation states; and finally, use a point cloud-image joint reconstruction strategy to generate a structural model. The final output is the floating posture. The innovations in the fusion process are: ① using image completion points to enhance sparse point clouds; ② dynamically adjusting the confidence of point pairs using perturbation states; and ③ using pose regularization to control the continuity of estimation.

[0104] S3. Calculate the linear difference between the spatial attitude of the floating and the previous frame to generate the floating prediction attitude, and construct the control compensation amount based on the difference between the floating prediction attitude and the target docking attitude.

[0105] Specifically, this step involves obtaining the floating attitude output by S2. With three-dimensional structural model Based on this, combined with the state of environmental disturbance This enables the prediction and generation of control compensation amounts for the attitude in the next frame of floating, and ultimately outputs the predicted attitude. and control compensation instructions Considering the dynamic disturbances and platform response delays that exist during the operation of the offshore floating system, this step designs a linear trend-driven, disturbance-adaptive path prediction and control compensation method, focusing on controlling the number of variables and structural simplicity to ensure the feasibility of real-time deployment on the edge platform.

[0106] Get input: The current frame's floating posture is derived from the previous step; The previous frame's floating posture is cached by the system. Target docking attitude, by Automatically extract buoyancy posture get; : Disturbance state, where For ambient lighting, For the camera pitch disturbance angle. This represents the platform vibration intensity.

[0107] Furthermore, the pose of the next frame is first estimated based on the linear difference between the poses of the previous two frames, and a perturbation suppression coefficient is introduced to control the prediction amplitude:

[0108] ;

[0109] in, . This is a perturbation adjustment factor used to adjust the prediction step size; its default value is [value missing]. Between ranges. The system is in a state of low disturbance (e.g., , When the value is close to 1, it allows for a larger prediction range; however, when the disturbance is severe (such as...), the value is close to 1, allowing for a larger prediction range; while ... or The value is automatically adjusted down to around 0.6 to suppress potential over-predictive jumps. (This variable...) , and All are six-dimensional vectors, with units consistent with S2, without any format or physical quantity conversion, and the predictions are directly called by the control module.

[0110] Next, based on the difference between the current prediction result and the target docking attitude, the control compensation quantity is constructed:

[0111] ;

[0112] in, This control gain factor reflects the strength of the control response the system should take under different disturbance levels, with a default value of [missing information]. Dynamic adjustment within a certain range: When the platform vibration is small (e.g.) When the platform vibrates significantly (e.g., when the platform vibration is significant), the control gain is kept at 0.6 to avoid excessive control convergence; )hour, Increasing it to 0.8 allows the system to respond to errors more quickly and correct position or attitude deviations. It is a six-dimensional control vector with the same attitude dimension, used as the control command executed by the next module, and can be specifically mapped to the input of thruster thrust, attitude control surface deflection or other control devices.

[0113] S4. Obtain the response status of each propulsion dimension, combine the disturbance state vector and the compensation control quantity, and generate the final control command to drive the float propulsion system.

[0114] Specifically, this step involves receiving the floating attitude prediction output from S3. With control compensation amount Based on this, generate low-level control commands that can directly drive the propulsion system. And combined with the actual execution status feedback of the thruster Compared with the current disturbance state This paper proposes a dynamic weighted control allocation mechanism that couples disturbance constraints with execution state to dynamically adjust control commands and impose physical feasibility constraints. Floating-over installation operations are characterized by distributed control of multiple thrusters, strong disturbances, and slow response. Traditional direct mapping mechanisms for control commands often lead to attitude overshoot or command inconsistencies, resulting in system oscillations or inaccurate docking.

[0115] Furthermore, to obtain Response status of each advancement dimension Indicates the first The current workload ratio (current thrust / maximum thrust) of the propulsion channel is periodically uploaded by the internal controller of the thruster;

[0116] Furthermore, this step first constructs the execution regulation factor. Control different channels (x, y, z, , , The weighting in actual execution takes into account both the disturbance state and the current thruster load pressure. Therefore, the following formula is introduced:

[0117] ;

[0118] in, For the first The control channel's execution adjustment factor. This indicates that the platform is vibrating. For the camera pitch disturbance angle. This represents the thruster load ratio for the current channel. The three weighting factors are set as follows: , , This design fully reflects the three core influencing factors of the control and allocation process:

[0119] The larger the value, the more violent the platform shakes; in this case, the aggressiveness of global control should be suppressed.

[0120] The larger the value, the higher the risk of image perception bias, and the more necessary it is to limit perception-related control variables (such as pitch).

[0121] The closer it is to 1, the closer the current thruster load is to its limit. Control command injection into its channels should be reduced to prevent execution overflow.

[0122] Furthermore, regulatory factors Applied to compensating control quantity Generate final control commands:

[0123] ;

[0124] The formula consists of two terms:

[0125] Key control items: It is the actual control command scaled according to the execution channel status;

[0126] Innovation item: It is a soft-constraint elastic repair term, when a certain channel's Even when the risk or load in that direction is too high, a correction component with a consistent sign and a very small magnitude is still allowed to be retained to avoid control "zero mutation" that could lead to float lag. The value is a small-amplitude stabilization factor, empirically taken as: .

[0127] This design embodies the innovation of this step: instead of simply reducing the proportion at the execution end, it introduces a disturbance-oriented soft suppression logic, which enables the system to maintain a low-amplitude but correct-direction control trend in high-risk dimensions, thereby ensuring the continuity of the control path and the compliance of the system response.

[0128] Execute instructions The data will be sent to the float-over propulsion control bus, where thrust inversion and attitude decoupling operations (not part of this step) will be performed in the lower-level module to drive the six-channel attitude adjustment device to complete the corresponding actions. (Unit: ...) All dimensions are consistent, and weighted adjustments have been made according to the system disturbance state and the thruster load state, which are available for use by the propulsion execution unit.

[0129] S5. Based on the current frame's floating spatial attitude, final control command, and target docking attitude, determine whether the floating has completed stable docking with the target attitude; if not, feed back the corresponding linear difference and the corresponding disturbance state vector, and return to S3 to form a control closed loop; if completed, the current round of work is completed.

[0130] Specifically, this step is used based on the actual attitude of the current buoyancy. Control execution instructions and expected docking posture The system determines whether the float has achieved stable docking with the target attitude and provides feedback on residual attitude errors and disturbance states. This feedback is used for the next round of command updates in the path prediction module, thereby achieving closed-loop system control. This step specifically addresses issues such as misjudging docking success due to strong disturbances and premature termination due to command non-convergence during offshore float docking. It proposes a tolerance judgment mechanism with disturbance weight adjustment to ensure that docking judgment has both geometric proximity and dynamic stability, preventing docking failure due to erroneous termination.

[0131] Get Target attitude, by Extracted from the model;

[0132] Furthermore, in the first part, it determines whether the float has reached the target docking state. The current attitude error vector is defined as:

[0133] ;

[0134] This quantity represents the attitude residual between the buoyancy and the target, with dimensions and Consistent. The estimated attitude is used here, rather than the predicted attitude, to ensure that the judgment is based on actual results rather than predicted trends.

[0135] To determine whether docking is complete, a disturbance-sensitive tolerance function is constructed. :

[0136] ;

[0137] in, These are the basic static tolerances for each dimension of the attitude, set to 0.05m for the position dimension and 1.5° for the attitude dimension. For platform acceleration disturbance, The pitch angle disturbances all originate from ; It is the perturbation amplification factor, and its empirical value is [value missing]. This is used to appropriately increase the tolerance when the disturbance is large; the smaller the value of this function, the closer it is to the target and the more stable it is.

[0138] It should be noted that the judgment rule is: if Continuous satisfaction One continuous control cycle ( If the floating docking is successful, a status flag is set. Otherwise set to This leads to a closed-loop feedback loop.

[0139] The second part feeds back the residual information from incomplete docking to the path prediction module, forming a control closed loop. The feedback package is defined as follows:

[0140] ;

[0141] The feedback package includes:

[0142] residual The difference between the current attitude and the target attitude represents the area where control has not been achieved.

[0143] : Executed control commands, which are used by the path prediction module to dynamically adjust the strategy;

[0144] Current disturbance intensity information is used to estimate the reliability of prediction strategies or the degree of aggressiveness in control.

[0145] Furthermore, all feedback data will be sent back to the path prediction and compensation module (S3) in structured vector form for updating. The generation strategy enables a stable, continuous, and self-correcting closed-loop buoyancy control system.

[0146] like Figure 2 As shown, in another embodiment of the present invention, a visual image-guided automated floating installation system is provided, the system comprising:

[0147] The image enhancement module 301 is used to acquire multiple original RGB image frames and perturbation state vectors, and dynamically adjust the original RGB image frames according to the perturbation state vectors to generate enhanced image frames; wherein the perturbation state vectors include the current ambient light intensity, camera angle perturbation, and platform acceleration perturbation intensity;

[0148] The 3D modeling module 302 is used to acquire the sparse point cloud data of the current frame; extract the structural feature points of the enhanced image frame and back-project them into 3D spatial points; optimize the registration error function controlled by perturbation based on the 3D spatial points to generate the floating spatial attitude of the current frame; generate the spatial position of the 3D spatial points in the registered world coordinate system to form a new set of position points; register the new set of position points with the sparse point cloud data of the current frame to construct a joint point set; and generate a 3D structural model and target docking attitude based on the joint point set.

[0149] The path prediction and compensation module 303 is used to calculate the linear difference between the spatial attitude of the floating and the previous frame to generate the floating prediction attitude, and to construct the control compensation amount based on the difference between the floating prediction attitude and the target docking attitude.

[0150] The control execution module 304 is used to acquire the response status of each propulsion dimension, and combine the disturbance state vector and the compensation control quantity to generate the final control command for driving the float propulsion system.

[0151] The docking judgment and feedback module 305 is used to determine whether the floating has completed a stable docking with the target attitude by combining the spatial attitude of the floating in the current frame, the final control command and the target docking attitude; if it has not been completed, it feeds back the corresponding linear difference and the corresponding disturbance state vector and returns to the path prediction and compensation module to form a control closed loop; if it is completed, the current round of work is completed.

[0152] In addition, for technical details not described in detail in this embodiment, please refer to the parameter operation method provided in any embodiment of the present invention, which will not be repeated here.

[0153] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0154] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0155] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory / random access memory, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0156] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A method for implementing a visual image-guided automated floating installation system, characterized in that, Includes the following steps: S1. Acquire multiple original RGB image frames and perturbation state vectors, and dynamically adjust the original RGB image frames according to the perturbation state vectors to generate enhanced image frames; wherein the perturbation state vectors include the current ambient light intensity, camera angle perturbation, and platform acceleration perturbation intensity; S2. Obtain the sparse point cloud data of the current frame; The structural feature points of the enhanced image frame are extracted and back-projected into three-dimensional spatial points. Based on the three-dimensional spatial points, the registration error function controlled by perturbation is optimized to generate the spatial attitude of the current frame floating. The spatial positions of the three-dimensional spatial points in the registered world coordinate system are generated to form a new set of position points. The new set of position points is registered with sparse point cloud data to construct a joint point set. A three-dimensional structural model and target docking posture are generated based on the joint point set. S3. Calculate the linear difference between the spatial attitude of the floating and the previous frame to generate the floating prediction attitude, and construct the control compensation amount based on the difference between the floating prediction attitude and the target docking attitude. S4. Obtain the response status of each propulsion dimension, combine the disturbance state vector and the compensation control quantity, and generate the final control command to drive the buoyancy propulsion system. S5. Combining the current frame's floating attitude, final control command, and target docking attitude, determine whether the floating has completed stable docking with the target attitude; if not, feed back the corresponding linear difference and the corresponding disturbance state vector, and return to S3 to form a control closed loop. If completed, this round of work is finished; The process of optimizing the registration error function based on the three-dimensional spatial points through perturbation control to generate the spatial attitude of the current frame's floating position specifically includes: Structural feature points are extracted from the enhanced image frame, and points with significant edges and angle abrupt changes are identified as target anchor points using multi-scale Canny edge detection and Harris corner detection methods. The target anchor points are then back-projected into target three-dimensional space points using camera intrinsic and extrinsic parameters. Based on the target three-dimensional spatial point, search for its nearest neighbor in the current frame sparse point cloud data to form an initial point pair set; A registration error function is constructed based on the initial set of point pairs. ,as follows: ； Where i is point i, and M is the total number of points. As weight, For rotation matrix, For the target three-dimensional space point, It is a translation vector. As the nearest neighbor, This is the attitude regularization coefficient, used to control the suppression of continuity offset between the current attitude and the attitude of the previous frame. The default value is 0.

5. The rotation matrix of the previous frame's floating element. The squared Frobenius norm between the current and previous frame rotation matrices measures the amount of rotation change. This is the translation vector of the previous frame's floating position. The weight is the squared Euclidean distance between the current position and the previous frame position; where, the weights are... Calculated using the following formula: ； in, It is a natural exponential function. To enhance image frames, Enhance the frame of the image At point The squared magnitude of the image gradient at a given point represents the edge strength of the structure at that point, and is calculated using the Sobel operator. This is the image gradient normalization factor. As a perturbation-driven enhancement factor, This is the adjustment coefficient for the disturbance-driven enhancement factor, with a default value of 0.

5. For camera angle perturbation, The intensity of the platform acceleration disturbance; Optimize the registration error function Obtain the rotation matrix of the buoyancy. With translation vector This allows us to construct the spatial attitude of the floating object in the current frame. ;in, The position in three-dimensional coordinates. The attitude angle; Specifically, generating the three-dimensional structural model and target docking posture based on the joint point set includes: A three-dimensional structural model is generated by reconstructing a mesh using the Poisson surface reconstruction method and a point set. Based on the aforementioned three-dimensional structural model, the spatial attitude of the floating mechanism in the current frame is determined. The simulation was performed to generate the target docking attitude.

2. The method for implementing a visual image-guided floating installation automation system according to claim 1, characterized in that, The process involves acquiring multiple raw RGB image frames and perturbation state vectors, and obtaining environmental parameters through a light sensor, IMU module, and accelerometer; wherein... The step of dynamically adjusting the original RGB image frame according to the perturbation state vector to generate an enhanced image frame specifically includes: Based on the current ambient light intensity, the original RGB image frame is dynamically brightened to generate an intermediate image frame with adjusted brightness. The intermediate image frame after brightness adjustment is subjected to image enhancement processing based on camera angle perturbation and platform acceleration perturbation intensity to generate an enhanced image frame.

3. The method for implementing a visual image-guided floating installation automation system according to claim 1, characterized in that, The S2 also includes: In the floating attitude prediction, a disturbance suppression coefficient is introduced to control the prediction amplitude. When the environmental disturbance is most severe, the prediction amplitude is automatically reduced to the minimum, ensuring the stability and reliability of the predicted attitude. In the control compensation quantity, a control gain factor is also introduced to reflect the control response intensity that the system should take under different disturbance levels. The control response intensity is the highest when the vibration is the strongest, so as to quickly correct the floating attitude deviation.

4. The method for implementing a visual image-guided floating installation automation system according to claim 1, characterized in that, The predicted attitude of the buoyancy is a six-dimensional vector; where... The process of acquiring the response states of each propulsion dimension, combining the disturbance state vector and the compensation control quantity, and generating the final control command to drive the overflight propulsion system specifically includes: Based on the predicted attitude, the perturbation state vector and compensation control quantity are used to generate an execution adjustment factor to control different channels (x, y, z, ...). , , The allocation of weights in actual implementation; The execution adjustment factor is applied to the compensation control quantity to generate the final control command.

5. The method for implementing a visual image-guided floating installation automation system according to claim 4, characterized in that, The final control command is expressed as follows: ； in, For the final control command, To implement the adjustment factor, To compensate for the control quantity, The value is a small-amplitude stabilization factor, empirically taken as: ; in, It is the actual control command scaled according to the execution channel status; It is a soft constraint elastic repair item. When the execution adjustment factor of a certain channel is very small, it still allows a correction component with consistent sign and extremely small amplitude to be retained, so as to avoid control zero mutation leading to floating lag.

6. The method for implementing a visual image-guided automated floating installation system according to claim 1, characterized in that, The step of determining whether the floating platform has completed stable docking with the target attitude by combining the spatial attitude of the floating platform in the current frame, the final control command, and the target docking attitude specifically includes: The current attitude error vector is determined based on the spatial attitude of the floating vessel and the target docking attitude of the current frame. A disturbance-sensitive tolerance function is constructed based on the current attitude error vector, and the docking is determined based on the disturbance-sensitive tolerance function; wherein the minimum value of the disturbance-sensitive tolerance function indicates that the target is closest and the system is most stable; If the docking is not completed, the residual information at the time of the incomplete docking will be fed back to S3 to form a control closed loop.

7. The method for implementing a visual image-guided floating installation automation system according to claim 6, characterized in that, The determination of whether docking is complete based on the disturbance-sensitive tolerance function is as follows: If the disturbance sensitivity tolerance function is less than 1.0 and continuously satisfies If a continuous control cycle is completed, the floating docking is considered successful, and a status flag is set. Otherwise set to It also feeds back the residual information when the docking is not completed to S3.

8. The method for implementing a visual image-guided floating installation automation system according to claim 6, characterized in that, The residual information includes: The difference between the current attitude error vector and the target docking attitude represents the area where control was not achieved; Current final control command; The current perturbation state vector.

9. A visual image-guided floating installation automation system that performs the implementation method as described in claim 1, characterized in that, The system includes: The image enhancement module is used to acquire multiple original RGB image frames and perturbation state vectors, and dynamically adjust the original RGB image frames according to the perturbation state vectors to generate enhanced image frames; wherein the perturbation state vectors include the current ambient light intensity, camera angle perturbation, and platform acceleration perturbation intensity; The 3D modeling module is used to acquire sparse point cloud data of the current frame; extract structural feature points of the enhanced image frame and back-project them into 3D spatial points; optimize the registration error function controlled by perturbation based on the 3D spatial points to generate the spatial attitude of the floating in the current frame; generate the spatial position of the 3D spatial points in the registered world coordinate system to form a new set of position points; register the new set of position points with the sparse point cloud data of the current frame to construct a joint point set; and generate a 3D structural model and target docking attitude based on the joint point set. The path prediction and compensation module is used to calculate the linear difference between the spatial attitude of the floating and the previous frame to generate the floating predicted attitude, and to construct the control compensation amount based on the difference between the floating predicted attitude and the target docking attitude. The control execution module is used to acquire the response status of each propulsion dimension, and combine the disturbance state vector and the compensation control quantity to generate the final control command to drive the buoyancy propulsion system. The docking judgment and feedback module is used to determine whether the floating platform has completed a stable docking with the target attitude by combining the spatial attitude of the floating platform in the current frame, the final control command, and the target docking attitude. If it has not completed the docking, it feeds back the corresponding linear difference and the corresponding disturbance state vector, and returns it to the path prediction and compensation module to form a control closed loop. If it has completed the docking, the current round of work is completed.

Images