Dynamic positioning floating ship entering control method and system based on image recognition

Abstract

The invention discloses a dynamic positioning floating ship entering control method and system based on image recognition, and aims at solving the problems that manual operation is poor in precision, low in efficiency, insufficient in safety and the like. According to the method, a camera is arranged on a jacket to collect images, image recognition is performed after preprocessing, image features of the jacket and a floating mounting ship are extracted, and the relative position and posture of the ship relative to the jacket are estimated according to the image features. And the system generates a control instruction by using a dynamic positioning control algorithm based on the position attitude information, automatically controls a ship propeller, and realizes accurate entering of the floating mounting ship to the preset position of the jacket. And the system continuously monitors and adjusts a control instruction in real time to form closed-loop control, so that the ship is ensured to safely and accurately complete a floating ship entering task. The positioning precision, the operation efficiency and the safety of floating ship entering are effectively improved, and automatic control is achieved.

Description

Technical Field

[0001] This application relates to the technical field of ship control, and in particular, to a dynamic positioning and floating installation ship control method and system based on image recognition. Background Art

[0002] In the field of offshore engineering, the installation of the upper module of a jacket platform usually adopts the dynamic positioning and floating installation technology. This technology uses the thrusters of the dynamic positioning ship itself to accurately control the position and heading of the ship, so as to realize the transportation and installation of the upper module. During the floating installation ship entry process, the dynamic positioning ship needs to drive into the inside of the jacket and maintain a very small gap with the fenders installed on the jacket to achieve precise docking between the upper platform and the jacket.

[0003] However, affected by factors such as wind, waves, and currents in the marine environment, the ship is prone to movement, resulting in collisions between the dynamic positioning ship and the fenders when driving into the jacket. Traditionally, the position information of the dynamic positioning ship mainly relies on the Global Navigation Satellite System (GNSS), but its accuracy is difficult to meet the requirements of floating installation ship entry in harsh marine environments. Therefore, the existing technology usually adopts manual control, that is, the captain manually controls the translation of the ship through the dynamic positioning joystick, and at the same time maintains the ship's heading unchanged through the automatic control system to control the collision load as much as possible. This method highly depends on the captain's experience and visual judgment, and has the following deficiencies: First, the positioning accuracy is limited and is easily affected by factors such as fatigue, judgment errors, visual range, sea conditions, and lighting; second, the operation efficiency is low, requiring repeated adjustments and taking a long time; third, it depends on the captain's experience, and the reliability and consistency fluctuate.

[0004] At present, there is still a lack of a method for realizing fully automatic ship entry for floating installation based on image recognition technology in the industry. Therefore, there is an urgent need for a dynamic positioning and floating installation ship control method that can improve the positioning accuracy, operation efficiency, and safety, so as to reduce manual operations, expand the operation window, and realize a more automated and precise floating installation ship entry task.

[0005] It should be noted that the information disclosed in the above background art section is only used to enhance the understanding of the background of this application, and therefore may include information that does not constitute the prior art known to those of ordinary skill in the art.

[0006] Content of the Application

[0007] In view of the problems in the existing dynamic positioning and floating installation ship control method that rely on manual operations, such as limited positioning accuracy, low operation efficiency, and reliability affected by the captain's experience, the present invention provides a dynamic positioning and floating installation ship control method based on image recognition, aiming to improve the positioning accuracy, operation efficiency, and safety of floating installation ship entry, and realize automatic control of the floating installation ship entry process.

[0008] An embodiment of the present application provides a dynamic positioning floating installation ship approaching control method based on image recognition, including the following steps:

[0009] Install at least one camera on the jacket, with the camera's view facing the floating installation ship, for real-time acquisition of image data containing the jacket and the floating installation ship;

[0010] Preprocess the acquired image data, including at least performing image distortion correction and image enhancement processing to improve the image quality;

[0011] Perform image recognition processing on the preprocessed image to identify the jacket and the floating installation ship in the image, extract the image features of the jacket and the floating installation ship, and obtain the relative position and attitude information of the floating installation ship relative to the jacket based on the image features. The image features include the contours and key structure points of the jacket and the floating installation ship;

[0012] Based on the estimated relative position and attitude information of the floating installation ship relative to the jacket, and the preset target position and attitude, use the dynamic positioning control algorithm to generate control instructions, which are used to control the movement of the floating installation ship;

[0013] Send the generated control instructions to the dynamic positioning system of the floating installation ship, control the operation of the ship's thrusters, and drive the floating installation ship to move according to the control instructions to achieve automatic approaching control of the floating installation ship to the predetermined position of the jacket;

[0014] Continuously monitor and update the relative position and attitude information of the floating installation ship relative to the jacket using the image recognition method, and adjust the control instructions in real time for closed-loop control to ensure that the floating installation ship accurately approaches the jacket to complete the floating installation ship approaching task.

[0015] In some alternative embodiments, performing image recognition processing on the preprocessed image to identify the jacket and the floating installation ship in the image, extract the image features of the jacket and the floating installation ship, and obtain the relative position and attitude information of the floating installation ship relative to the jacket includes the following steps:

[0016] Perform image registration on the images from at least one camera to determine the same feature points corresponding to the jacket and the floating installation ship in different images;

[0017] Based on the image registration results and the camera calibration parameters, calculate the three-dimensional spatial position information of the jacket and the floating installation ship, so as to estimate the relative position and attitude information of the floating installation ship relative to the jacket.

[0018] In some alternative embodiments, the image recognition processing uses a deep learning-based object detection algorithm, including the following steps:

[0019] Build and train a convolutional neural network model, which includes a convolutional layer, a pooling layer, and a fully connected layer;

[0020] Input the preprocessed image into the convolutional neural network model;

[0021] Through the forward propagation calculation of the convolutional neural network model, extract the features of the image layer by layer, and generate detection results including target detection boxes and class confidence levels at the output layer of the model;

[0022] Based on the detection results, determine the positions of the jacket and the floating installation vessel in the image, and extract the contour and key structural point features of the jacket and the floating installation vessel according to the target detection boxes.

[0023] In some alternative embodiments, extracting the image features of the jacket and the floating installation vessel includes using the SIFT algorithm or the SURF algorithm to extract the local invariant features of the images of the jacket and the floating installation vessel, specifically including the following steps:

[0024] In the images of the jacket and the floating installation vessel, detect significant feature points respectively, and the feature points are local extreme points or edge points in the images;

[0025] For each detected feature point, calculate the gradient direction histogram of its local image region to generate a feature descriptor, and the feature descriptor is a vector representing the local characteristics of the image around the feature point;

[0026] Between images from different perspectives, perform feature matching based on the feature descriptors, find the matching point pairs with the closest distances between the descriptors, and obtain the corresponding relationship of the feature points between the images.

[0027] In some alternative embodiments, the dynamic positioning control algorithm is a model predictive control algorithm, and the model predictive control algorithm includes the following steps:

[0028] Establish a dynamic model of the floating installation vessel;

[0029] Based on the dynamic model of the floating installation vessel, predict the motion trajectory of the floating installation vessel within the prediction time domain according to the rolling time domain strategy, and generate the optimal control instruction within the control period.

[0030] In some alternative embodiments, the image preprocessing further includes the step of adaptively adjusting the camera exposure time and the image contrast according to the environmental light intensity, specifically including:

[0031] Detect the environmental light intensity, and when the detected environmental light intensity decreases, increase the camera exposure time;

[0032] When the detected image contrast decreases, increase the image contrast enhancement coefficient.

[0033] In some alternative embodiments, it further includes a sensor data fusion step. In the position and attitude estimation step, data from sensors is fused, and the sensors include one or more of lidar, infrared thermal imager, GPS, and IMU.

[0034] In some alternative embodiments, in the sensor data fusion step, the Kalman filtering algorithm or the extended Kalman filtering algorithm is adopted to fuse the position and attitude estimation results of image recognition with other sensor data.

[0035] The embodiment of the present invention further provides a dynamic positioning floating pontoon ship docking control system based on image recognition for implementing the above-mentioned dynamic positioning floating pontoon ship docking control method based on image recognition, including:

[0036] An image acquisition module, arranged on the jacket, includes at least one camera, and the camera's view is directed towards the floating pontoon installation ship for real-time acquisition of image data including the jacket and the floating pontoon installation ship.

[0037] An image processing module for preprocessing the acquired image data, including at least image distortion correction and image enhancement processing to improve the image quality.

[0038] A position and attitude estimation module for performing image recognition processing on the preprocessed image, identifying the jacket and the floating pontoon installation ship in the image, extracting the image features of the jacket and the floating pontoon installation ship, and obtaining the relative position and attitude information of the floating pontoon installation ship relative to the jacket based on the image features. The image features include the contours and key structure points of the jacket and the floating pontoon installation ship.

[0039] A control instruction generation module for generating control instructions based on the estimated relative position and attitude information of the floating pontoon installation ship relative to the jacket and the preset target position and attitude, and the control instructions are used to control the movement of the floating pontoon installation ship.

[0040] A dynamic positioning control module for receiving the control instructions and controlling the dynamic positioning system of the floating pontoon installation ship to drive the floating pontoon installation ship to move according to the control instructions, realizing the automatic docking control of the floating pontoon installation ship to the predetermined position of the jacket.

[0041] A feedback adjustment module for continuously monitoring and updating the relative position and attitude information of the floating pontoon installation ship relative to the jacket using the image recognition method, and adjusting the control instructions in real time for closed-loop control to ensure that the floating pontoon installation ship accurately approaches the jacket to complete the floating pontoon docking task.

[0042] In some alternative embodiments, it further includes a fault diagnosis and redundancy module for real-time monitoring the working states of each module of the system and performing fault diagnosis.

[0043] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and do not limit the present disclosure.

[0044] A dynamic positioning floating jacking ship control method and system based on image recognition according to the present application have the following beneficial effects:

[0045] The image recognition technology of the present application can obtain information such as the position and attitude of the target object by detecting, recognizing, and extracting features of the target object in the image. The dynamic positioning system can achieve precise position holding and motion control of the ship on the water surface by controlling the thrust magnitude and direction of the ship's thrusters. Applying the image recognition technology to the dynamic positioning floating jacking ship control, the position and attitude information of the ship relative to the jacket obtained through image recognition provides precise control parameters for the dynamic positioning system, thereby driving the ship to move along a predetermined trajectory and realizing automatic ship jacking. Through this technology, the errors and limitations of manual operation can be avoided, the positioning accuracy can be improved, and the operation efficiency and safety can be enhanced. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Other features, objects, and advantages of the present application will become more apparent by reading the detailed description of the non-limiting embodiments with reference to the following drawings.

[0047] Figure 1 is a flowchart of a dynamic positioning floating jacking ship control method based on image recognition according to an embodiment of the present application;

[0048] Figure 2 is a schematic diagram of the multi-camera arrangement of a dynamic positioning floating jacking ship control method according to an embodiment of the present application;

[0049] Figure 3 is a schematic diagram of the structure of a dynamic positioning floating jacking ship control system based on image recognition according to an embodiment of the present application;

[0050] Figure 4 is a flowchart of a dynamic positioning floating jacking ship control method based on image recognition according to another embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0051] Example embodiments will now be described more fully with reference to the accompanying drawings. However, the example embodiments can be implemented in various forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be more complete and will fully convey the concept of the example embodiments to those skilled in the art. The described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments.

[0052] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus repeated descriptions thereof will be omitted. Some of the block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0053] The flowcharts shown in the accompanying drawings are only exemplary illustrations and do not necessarily include all steps. For example, some steps can be further decomposed, while some steps can be combined or partially combined. Therefore, the actual execution order may be changed according to the actual situation.

[0054] The present invention provides a dynamic positioning floating installation vessel approach control method based on image recognition. By setting cameras on the jacket to collect images, performing image preprocessing and recognition, and extracting the image features of the jacket and the floating installation vessel, the relative position and attitude information of the floating installation vessel relative to the jacket can be obtained, and control instructions are generated to control the movement of the vessel. This method can reduce the dependence on manual operation, reduce errors caused by human factors, improve the positioning accuracy and operation efficiency of the floating installation vessel approach to a certain extent, and improve the operation safety in complex marine environments. Through closed-loop control, it can ensure that the floating installation vessel accurately approaches the jacket, thereby completing the floating installation vessel approach task.

[0055] As Figure 1 shown, the present invention provides a dynamic positioning floating installation vessel approach control method based on image recognition, including the following steps:

[0056] S100. A technical solution of setting at least one camera on the jacket, with the camera's viewing angle facing the floating installation vessel, for real-time collecting image data including the jacket and the floating installation vessel. Among them, the jacket is an important part of an offshore oil and gas platform, used to support the upper platform structure; the floating installation vessel is a special vessel used for transporting and installing offshore platform modules. The camera is used to capture the image information of the jacket and the floating installation vessel in the embodiments of the present invention and is the basis for realizing image recognition and position and attitude estimation. The camera is set on the jacket, which can avoid the interference of vessel movement on image collection and improve the clarity and stability of the image data. The number of cameras can be adjusted according to actual needs. For example, one camera can be set at each of the four corner points of the jacket to achieve omnidirectional image coverage. Specifically, as Figure 2As shown in the figure, cameras are respectively installed on the jacket platforms on both sides of the approach route of the ship, namely camera A1, camera A2, camera A3, camera B1, camera B2, and camera B3. Among them, camera A1, camera A2, and camera A3 are arranged at approximately equal intervals along the length direction of the jacket platform and are located on one side of the width direction of the jacket platform structure. Camera B1, camera B2, and camera B3 are also arranged at approximately equal intervals along the length direction of the jacket platform and are located on the other side of the width direction of the jacket platform structure, opposite to camera A1, camera A2, and camera A3. By installing three groups of cameras on both sides of the jacket platform respectively, it is possible to collect image information in a relatively large range in front of the jacket platform, ensuring that the floating installation vessel is always within the field of view of the cameras during the approach process, avoiding the situation of target loss, and improving the robustness of the system. The camera's viewing angle is directed towards the floating installation vessel, and the installation angle of the camera can be adjusted so that the optical axis of each camera points to the area where the floating installation vessel is about to enter the jacket platform, ensuring that the captured images can clearly contain the two key targets, namely the jacket platform and the floating installation vessel. In this embodiment, the camera preferably uses an industrial-grade CCD camera with high resolution to ensure the clarity and quality of the images, providing high-quality original data for subsequent image recognition algorithms. The camera can also be a global shutter camera to avoid image smear caused by insufficient image acquisition speed during the ship's movement, further improving the image quality. In addition, to adapt to the complex lighting conditions in the marine environment, the camera can also be a wide dynamic range (WDR) camera to ensure that both bright and dark areas of the image are clearly visible in scenarios where strong light and weak light coexist, avoiding overexposure or underexposure situations.

[0057] This technical solution provides high-quality original data for subsequent image processing and position and attitude estimation by installing cameras on the jacket platform, which is a key link in realizing the automatic control of dynamic positioning floating installation vessel approaching the ship. Through this technical solution, the relative position information between the ship and the jacket platform can be accurately captured, providing reliable data support for the execution of subsequent control algorithms, thereby improving the accuracy and safety of the floating installation vessel approaching the ship.

[0058] S200. A technical solution for preprocessing the collected image data, including at least image distortion correction and image enhancement processing to improve the image quality. Among them, image preprocessing is to preliminarily process the original image collected by the camera to provide quality guarantee for subsequent image recognition. Image distortion correction refers to correcting the image deformation caused by factors such as the camera optical system or imaging angle. For example, the Zhang Zhengyou calibration method or other calibration methods can be used to obtain the internal and external parameters of the camera, and the distortion model is used to correct the image. Commonly used distortion models include radial distortion model, tangential distortion model, etc. Image enhancement processing refers to adjusting the contrast, brightness, sharpness, etc. of the image to improve the clarity and recognizability of the image. For example, methods such as histogram equalization, CLAHE (Contrast Limited Adaptive Histogram Equalization) can be used for image enhancement.

[0059] The purpose of image preprocessing is to eliminate image noise, reduce image distortion, improve the image quality, and thus improve the accuracy and robustness of image recognition. Through image distortion correction, the positioning error caused by the camera's own factors can be reduced; through image enhancement processing, the recognizability of the image under different lighting conditions can be improved, thus expanding the operation window.

[0060] S300. Perform image recognition processing on the preprocessed image, recognize the jacket and the floatover installation vessel in the image, extract the image features of the jacket and the floatover installation vessel, and obtain the relative position and attitude information of the floatover installation vessel relative to the jacket based on the image features. The image features include the contours and key structure points of the jacket and the floatover installation vessel. Among them, image recognition processing refers to using image recognition algorithms to analyze the preprocessed image, recognize the target objects in the image, and extract their feature information. The jacket and the floatover installation vessel are the key target objects in the present invention, and their contours and key structure points are important features for position and attitude estimation. Commonly used image recognition algorithms include deep learning-based object detection algorithms such as YOLO, SSD, Faster R-CNN, etc., and traditional feature extraction algorithms such as SIFT, SURF, HOG, etc. For example, the YOLO algorithm can be used to recognize the jacket and the floatover installation vessel in the image and extract their key structure points such as contours and corner points. Then, based on the extracted image features, methods such as triangulation method, PnP algorithm, etc. can be used to estimate the relative position and attitude information of the floatover installation vessel relative to the jacket, including relative distance, azimuth angle, heading angle, pitch angle, roll angle, etc.

[0061] Through image recognition and feature extraction, the relative position and attitude information of the ship and the jacket can be accurately obtained, providing an accurate data basis for the generation of subsequent control commands.

[0062] In the technical solution of S400, based on the relative position and attitude information of the jack-up installation vessel relative to the jacket and the preset target position and attitude, a control instruction is generated using a dynamic positioning control algorithm. The control instruction is used to control the movement of the jack-up installation vessel. Among them, the relative position and attitude information is the result of image recognition processing, reflecting the actual state of the jack-up installation vessel relative to the jacket; the preset target position and attitude are set according to the requirements of the jack-up installation task. For example, it can be the position and attitude after the jack-up installation vessel completely enters the jacket. The dynamic positioning control algorithm is a control algorithm designed based on the ship dynamics model, used to calculate the propulsion force and direction required for the ship to reach the target state from the current state. Commonly used dynamic positioning control algorithms include PID control, model predictive control (MPC), fuzzy control, etc. For example, the model predictive control algorithm can be adopted. According to the dynamics model of the jack-up installation vessel, the movement trajectory of the ship in the future period is predicted, and a series of optimal control instructions are generated to enable the ship to accurately reach the target position and attitude. The control instruction usually includes information such as the rotation speed and angle of the thruster, used to control the propulsion system of the ship.

[0063] Converting the position and attitude information obtained by image recognition into instructions for controlling the movement of the ship through the dynamic positioning control algorithm is the key link to realizing automatic ship entry control. Through this technical solution, the movement trajectory of the ship can be dynamically adjusted according to the actual situation, improving the control accuracy and robustness.

[0064] In the technical solution of S500, the generated control instruction is sent to the dynamic positioning system of the jack-up installation vessel to control the operation of the thrusters of the ship and drive the jack-up installation vessel to move according to the control instruction, realizing the automatic ship entry control of the jack-up installation vessel to the predetermined position of the jacket. Among them, the dynamic positioning system is a propulsion control system installed on the jack-up installation vessel, used to automatically control the position and attitude of the ship according to external instructions. The thruster is the actuator of the dynamic positioning system. By adjusting the rotation speed and angle of the thruster, different propulsion forces and torques can be generated, thereby controlling the movement of the ship. Commonly used dynamic positioning systems include the dynamic positioning system based on PID control, the dynamic positioning system based on model predictive control, etc. For example, the control instruction generated by the model predictive control algorithm can be sent to the dynamic positioning system, and the dynamic positioning system adjusts the rotation speed and angle of the thruster according to the instruction, thereby driving the jack-up installation vessel to move along the predetermined trajectory and finally reach the predetermined position of the jacket.

[0065] Executing the control instruction through the dynamic positioning system and converting the output of the control algorithm into the actual movement of the ship is the key link to realizing automatic ship entry control. Through this technical solution, the movement trajectory of the ship can be accurately controlled, automatic ship entry can be realized, manual intervention can be reduced, and the operation efficiency and safety can be improved.

[0066] The S600 continuously monitors and updates the relative position and attitude information of the floating installation vessel with respect to the jacket using image recognition methods, and adjusts the control instructions in real time for closed-loop control to ensure that the floating installation vessel accurately approaches the jacket to complete the task of floating the vessel onto the jacket. Among them, continuous monitoring means that the system continuously collects image data, performs image processing and recognition to obtain the latest vessel position and attitude information. The image recognition method is the same as the previous step and is used to update the relative position and attitude information of the vessel with respect to the jacket in real time. Closed-loop control means that the actual vessel position and attitude information are fed back into the control system, compared with the target position and attitude, and the control instructions are adjusted according to the deviation to achieve precise vessel motion control. For example, a PID controller or a model predictive controller can be used to achieve closed-loop control.

[0067] Through closed-loop control, external disturbances and system errors can be effectively eliminated, and the control accuracy and robustness can be improved. Through continuous monitoring and closed-loop control, it can be ensured that the floating installation vessel accurately approaches the jacket in a complex marine environment and completes the task of floating the vessel onto the jacket. Furthermore, the influence of factors such as wind, waves and currents on the vessel motion can be effectively suppressed, and the stability and reliability of the control system can be improved.

[0068] In some embodiments, the preprocessed image is subjected to image recognition processing to identify the jacket and the floating installation vessel in the image, extract the image features of the jacket and the floating installation vessel, and obtain the relative position and attitude information of the floating installation vessel with respect to the jacket based on the image features, including the following steps:

[0069] Perform image registration on the images from at least one camera to determine the same feature points corresponding to the jacket and the floating installation vessel in different images;

[0070] Based on the image registration results and the camera calibration parameters, calculate the three-dimensional spatial position information of the jacket and the floating installation vessel, so as to estimate the relative position and attitude information of the floating installation vessel with respect to the jacket.

[0071] Among them, image registration refers to aligning images from different cameras or different perspectives, establishing the corresponding relationship between images, so as to achieve multi-view image fusion and 3D reconstruction. In the embodiments of the present invention, the purpose of image registration is to match the same feature points corresponding to the jacket and the floating installation vessel in the images from different cameras, providing a basis for subsequent calculation of 3D position information. The methods of image registration include feature-based registration methods and region-based registration methods. The feature-based registration method first extracts feature points in the image, such as corner points, edges, etc., and then matches them according to the descriptors of the feature points. Commonly used feature descriptors include SIFT, SURF, etc. The region-based registration method directly compares the pixel values or gray values of the images, and finds the best alignment method through optimization algorithms. Commonly used optimization algorithms include mutual information method, least squares method, etc. The camera calibration parameters refer to the internal and external parameters of the camera. The internal parameters include focal length, principal point coordinates, distortion coefficients, etc., and the external parameters include the position and attitude of the camera. Camera calibration can use Zhang Zhengyou calibration method or other calibration methods.

[0072] Through image registration and camera calibration, the 3D spatial position information of the jacket and the floating installation vessel can be accurately calculated, thereby improving the accuracy of position and attitude estimation.

[0073] In some embodiments, the image recognition process adopts an object detection algorithm based on deep learning, including the following steps:

[0074] Construct and train a convolutional neural network model, and the convolutional neural network model includes a convolutional layer, a pooling layer and a fully connected layer;

[0075] Input the preprocessed image into the convolutional neural network model;

[0076] Through the forward propagation calculation of the convolutional neural network model, extract the features of the image layer by layer, and generate detection results including object detection boxes and class confidence levels at the output layer of the model;

[0077] Based on the detection results, determine the positions of the jacket and the floating installation vessel in the image, and extract the outlines and key structural point features of the jacket and the floating installation vessel according to the object detection boxes.

[0078] Among them, deep learning is a machine learning method that can automatically learn the features in images and achieve high-precision object detection by constructing a multi-layer neural network model. The convolutional neural network (CNN) is a commonly used deep learning model, especially suitable for image processing tasks. The convolutional layer is the core component of the CNN, which extracts the local features of the image through convolutional operations; the pooling layer is used to reduce the dimension of the feature map, reduce the computational amount, and improve the robustness of the model; the fully connected layer is used to map the extracted features to the output layer to generate the object detection results. Commonly used deep learning-based object detection algorithms include YOLO, SSD, Faster R-CNN, etc. For example, the YOLOv5 algorithm can be adopted to construct a convolutional neural network model containing multiple convolutional layers, pooling layers, and fully connected layers, and train it on a large amount of image data so that the model can accurately identify the jacket and floating installation vessel in the image. After training, the preprocessed image is input into the trained convolutional neural network model. The model extracts the features of the image layer by layer through forward propagation calculation and generates the detection results including the object detection box and class confidence at the output layer. Based on the detection results, the positions of the jacket and floating installation vessel in the image can be determined, and the contours and key structural point features of the jacket and floating installation vessel, such as corner points, edges, etc., can be extracted according to the object detection box.

[0079] Through the deep learning-based object detection algorithm, the features in the image can be automatically learned, and the accuracy and robustness of object detection can be improved, thus providing a more accurate data basis for subsequent position and attitude estimation.

[0080] In some embodiments, extracting the image features of the jacket and floating installation vessel includes using the SIFT algorithm or SURF algorithm to extract the local invariant features of the jacket and floating installation vessel images, specifically including the following steps:

[0081] In the images of the jacket and floating installation vessel, significant feature points are respectively detected, and the feature points are local extreme points or edge points in the image;

[0082] For each detected feature point, calculate the gradient direction histogram of its local image region to generate a feature descriptor, and the feature descriptor is a vector representing the local characteristics of the image around the feature point;

[0083] Between images from different perspectives, feature matching is performed based on the feature descriptors to find the pair of matching points with the closest distance between the descriptors, and the corresponding relationship of the feature points between the images is obtained.

[0084] Among them, the SIFT (Scale-Invariant Feature Transform) algorithm and the SURF (Speeded Up Robust Features) algorithm are commonly used image feature extraction algorithms, which can extract local invariant features in images, that is, they are robust to changes in the scale, rotation, illumination, etc. of the images. Local invariant features refer to the feature points in the images that are significant and not easily affected by external factors. For example, corner points, edge points, etc. Feature descriptors are vectors used to describe the image information around the feature points. Commonly used feature descriptors include Histogram of Oriented Gradients (HOG), etc. In the embodiments of the present invention, using the SIFT algorithm or the SURF algorithm to extract the local invariant features of the jacket and the floating installation vessel images can effectively improve the accuracy and robustness of image recognition, especially in the case of large illumination changes and many interference factors such as water surface reflection in the marine environment. By extracting local invariant features and performing feature matching, the corresponding relationship between images from different perspectives can be established, providing a reliable data basis for subsequent 3D reconstruction and position and attitude estimation.

[0085] By extracting the local invariant features of the image, the robustness and accuracy of image recognition can be improved, so as to achieve more accurate position and attitude estimation in a complex marine environment.

[0086] In some embodiments, the dynamic positioning control algorithm is a model predictive control algorithm, and the model predictive control algorithm includes the following steps:

[0087] Establish the dynamic model of the floating installation vessel;

[0088] Based on the dynamic model of the floating installation vessel, predict the motion trajectory of the floating installation vessel within the prediction time domain according to the rolling time domain strategy, and generate the optimal control instruction within the control period.

[0089] Among them, Model Predictive Control (MPC) is an advanced control algorithm. Its core idea is to predict the system behavior in the future for a period of time based on the system model, and solve the optimal control strategy through an optimization algorithm. The dynamic model is a mathematical model that describes the motion law of the floating installation vessel, including parameters such as the mass, inertia, and damping of the ship, as well as the influence of the external environment, such as wind, waves, and currents. The dynamic model can be established using theoretical modeling methods or system identification methods. The rolling horizon strategy means that in each control cycle, based on the current state information, predict the behavior in the future for a period of time, solve the optimal control strategy, and then only apply the first control instruction to the system, and repeat the above process in the next control cycle. This strategy can effectively reduce the computational load and improve the real-time performance of the control system. The prediction horizon refers to the time range for which the model predictive control algorithm predicts the future system behavior, and the control cycle refers to the time interval for which the control system executes the control instruction. The optimal control instruction refers to the control instruction that optimizes the system performance index under certain constraint conditions. For example, a dynamic model including state variables such as the ship's position, speed, and heading angle can be established, and the quadratic programming algorithm can be used to solve the optimal control instruction, so that the ship can accurately track the predetermined trajectory and maintain a good attitude. By adopting the model predictive control algorithm, the dynamic model information of the ship can be fully utilized to achieve precise control of the ship's motion, and improve the robustness and anti-interference ability of the control system.

[0090] In some embodiments, the image preprocessing further includes the step of adaptively adjusting the camera exposure time and image contrast according to the ambient light intensity, specifically including:

[0091] Real-time detect the ambient light intensity, and when the detected ambient light intensity decreases, increase the camera exposure time;

[0092] When the detected image contrast decreases, increase the image contrast enhancement coefficient.

[0093] Among them, the step of further enhancing the quality of the image during image preprocessing by adaptively adjusting the camera exposure time and image contrast according to the ambient light intensity means that, in order to further improve the quality of the image and enable the subsequent image recognition algorithm to work more stably and reliably, when preprocessing the image data collected by the camera, the system will automatically adjust the camera exposure time and image contrast according to the current ambient light intensity. In the marine environment, the lighting conditions change significantly with factors such as time and weather. For example, day and night, sunny and cloudy days, and water surface reflections can all cause large fluctuations in the brightness and contrast of the image, which is not conducive to the stable operation of the image recognition algorithm. Therefore, by adding the step of adaptively adjusting the camera exposure time and image contrast according to the ambient light intensity, the problem of image quality degradation caused by light changes can be effectively solved, providing higher-quality image data guarantee for subsequent image recognition, position and attitude estimation, and dynamic positioning control. The ambient light intensity refers to the brightness level of the environment where the camera is located, which directly affects the intensity of the light received by the camera's photosensitive element and thus the brightness of the image. Real-time detection of the ambient light intensity means that the system can continuously monitor the current ambient light intensity level in order to make adjustments in a timely manner according to light changes. In the embodiments of the present invention, the ambient light intensity can be detected in various ways. For example, a dedicated light intensity sensor can be installed near the camera to directly measure the ambient light intensity value; or, the ambient light intensity can also be indirectly evaluated by analyzing the average brightness value of the image frames collected by the camera in real time. For example, when using the method of analyzing the average brightness value of the image frames, if the average gray value of the current image frame is lower than a preset threshold, it can be determined that the ambient light intensity has decreased.

[0094] By means of the step of adaptively adjusting the camera exposure time and image contrast according to the ambient light intensity, the embodiments of the present invention can make the image preprocessing process adaptive, dynamically adjust image parameters according to different lighting conditions, effectively improve the image quality in various lighting environments, thus ensuring the stability and reliability of the subsequent image recognition algorithm, and ultimately enhancing the overall performance of the dynamic positioning floating dock ship entry control system.

[0095] In some embodiments, the dynamic positioning floating installation ship control method based on image recognition further includes a sensor data fusion step. In the position and attitude estimation step, data from the sensors is fused. The sensors include one or more of lidar, infrared thermal imager, GPS, and IMU. In the embodiments of the present invention, in order to further improve the accuracy and robustness of the position and attitude estimation, after obtaining the preliminary position and attitude information of the ship relative to the jacket based on image recognition, the system will also fuse the perception data from a variety of other types of sensors to correct and optimize the preliminary estimation results, so as to obtain more accurate and reliable position and attitude information. In the marine environment, a single image recognition method may be interfered by various factors such as light, weather, and sea conditions, resulting in limitations in the accuracy and robustness of the estimation results. By introducing a variety of types of sensors and fusing their data, the advantages of different sensors can be fully utilized to complement each other, overcome the limitations of a single sensor, and thus improve the performance of the entire perception system. Lidar (LiDAR) is an active remote sensing technology. By emitting laser beams and receiving the returned signals, it can accurately measure the distance between an object and the sensor, thereby obtaining the three-dimensional point cloud data of the scene. Fusing lidar data with camera image data can effectively make up for the lack of obtaining depth information of the image and improve the accuracy and robustness of depth estimation. Especially in the case of poor lighting conditions or weak image texture information, the advantages of lidar are more obvious. The infrared thermal imager can sense the thermal radiation of an object. Even under low visibility conditions such as at night and in haze, it can clearly image and obtain the outline and thermal characteristics of the target. Fusing infrared thermal imager data with visible light image data can improve the target recognition ability of the system under various complex lighting and weather conditions and ensure reliable operation all-weather and all-day. GPS (Global Positioning System) can provide the global positioning information of the floating installation ship. Although its positioning accuracy is relatively low, in the open sea environment, it can provide a global rough reference for position and attitude estimation to assist the system in initialization and global positioning. IMU (Inertial Measurement Unit) can measure the motion information such as the angular velocity and acceleration of the floating installation ship. Through integral operation, the attitude information of the ship can be obtained, such as roll angle, pitch angle, and yaw angle. Fusing IMU data with the position and attitude estimation results of image recognition can improve the accuracy of attitude estimation and dynamic response and can make up for the problem of tracking loss that may occur in image recognition in the case of rapid movement or occlusion.

[0096] Furthermore, the specific method of sensor data fusion can be implemented using multiple algorithms. For example, the Kalman filter algorithm or the extended Kalman filter algorithm can be used. The Kalman filter is an optimal estimation theory that can effectively fuse data from different sensors and give the optimal state estimation result. In sensor data fusion, the position and attitude estimation results based on image recognition and the measurement data from sensors such as lidar, infrared thermal imager, GPS, and IMU can be used as the input of the Kalman filter. By designing appropriate system models and observation models, the Kalman filter algorithm is used for data fusion to obtain the optimal position and attitude estimation results after fusion. In addition, other data fusion algorithms such as particle filter, information fusion, and deep learning fusion can also be used, and the appropriate fusion algorithm can be selected according to specific application requirements and sensor characteristics.

[0097] It should be noted that the above-mentioned lidar, infrared thermal imager, GPS, and IMU are only examples of several commonly used sensor types that can be fused, and are not limitations on the present invention. In actual applications, one or more of these sensors can be selected for fusion according to specific scenario requirements and cost considerations, or other types of sensors such as millimeter-wave radar and ultrasonic sensors can be used. As long as they can provide useful supplementary information for position and attitude estimation, they all belong to the protection scope of the embodiments of the present invention.

[0098] Through the sensor data fusion step, the present invention can effectively fuse information from multiple sensors, make full use of the complementary advantages of different sensors, overcome the limitations of a single sensor, significantly improve the accuracy, robustness, and environmental adaptability of position and attitude estimation, and provide a more solid data basis for the accuracy and reliability of subsequent dynamic positioning control.

[0099] In some other embodiments, the sensor data fusion step can be performed after image preprocessing and before image recognition processing. For example, the lidar point cloud data can be pre-fused with the camera image to generate a fused depth image, and then the depth image is input into the image recognition algorithm for processing. The algorithm for data fusion can also select a deep learning fusion network to automatically learn the optimal fusion strategy using neural networks. These alternative embodiments all belong to the protection scope of the present invention.

[0100] As Figure 3 shown, the embodiments of the present invention also provide a dynamic positioning floating installation ship control system based on image recognition for implementing the above-mentioned dynamic positioning floating installation ship control method based on image recognition. The system includes:

[0101] An image acquisition module M100, arranged on the jacket, includes at least one camera, and the camera is oriented towards the floating installation ship, and is used for real-time acquisition of image data including the jacket and the floating installation ship;

[0102] The image processing module M200 is used to preprocess the image data collected by the image acquisition module M100, including at least performing image distortion correction and image enhancement processing to improve the image quality;

[0103] The position and attitude estimation module M300 is used to perform image recognition processing on the image preprocessed by the image processing module M200, recognize the jacket and the floating installation vessel in the image, extract the image features of the jacket and the floating installation vessel 2, and obtain the relative position and attitude information of the floating installation vessel relative to the jacket based on the image features. The image features include the outlines and key structure points of the jacket and the floating installation vessel;

[0104] The control instruction generation module M400 is used to generate control instructions based on the relative position and attitude information of the floating installation vessel relative to the jacket estimated by the position and attitude estimation module M300 and the preset target position and attitude, and use a dynamic positioning control algorithm. The control instructions are used to control the movement of the floating installation vessel;

[0105] The dynamic positioning control module M500 is used to receive the control instructions generated by the control instruction generation module M400, control the dynamic positioning system of the floating installation vessel, and drive the floating installation vessel to move according to the control instructions to realize the automatic approach control of the floating installation vessel to the predetermined position of the jacket;

[0106] The feedback adjustment module M600 is used to continuously monitor and update the relative position and attitude information of the floating installation vessel relative to the jacket using an image recognition method, and adjust the control instructions generated by the control instruction generation module M400 in real time for closed-loop control to ensure that the floating installation vessel accurately approaches the jacket to complete the floating approach task.

[0107] In some embodiments, the above-mentioned dynamic positioning floating approach control system based on image recognition further includes a fault diagnosis and redundancy module, which is used to monitor the working states of all modules of the system in real time and perform fault diagnosis. The module can continuously monitor the operating states of each module such as the image acquisition module M100, the image processing module M200, the position and attitude estimation module M300, the control instruction generation module M400, and the dynamic positioning control module M500. For example, it can monitor the input and output data, running time, resource occupancy rate, and whether abnormal errors occur in each module. Performing fault diagnosis indicates that the fault diagnosis and redundancy module can judge whether a fault occurs in the system based on the monitored module operating state information, and further analyze the type, location, and cause of the fault.

[0108] Through the fault diagnosis and redundancy module, the embodiments of the present invention can realize real-time monitoring and fault diagnosis of the operation status of the control system itself, timely detect potential system faults, and take corresponding treatment measures. For example, an alarm is issued to prompt the operator for manual intervention, or the redundant backup system is automatically started for switching, thereby effectively improving the reliability and safety of the control system, reducing the risk of system failure, and ensuring the smooth progress of the floating-in operation.

[0109] In some embodiments, the fault diagnosis and redundancy module may further have a redundancy switching function. For example, when it is detected that a certain camera fails, the system can automatically switch to the backup camera for image acquisition; when it is detected that a certain computing module fails, the system can automatically switch to the backup computing module for computing tasks, realizing redundant backup and seamless switching of hardware or software, and further improving the fault tolerance and reliability of the system. The fault diagnosis and redundancy module can also interact with the operator, providing friendly fault information prompts and treatment suggestions to assist the operator in fault troubleshooting and maintenance. These alternative embodiments all fall within the protection scope of the present invention.

[0110] The power positioning floating-in ship control system based on image recognition provided by the present invention has clear division of labor and collaborative work among its various modules, realizing automated and intelligent control of the floating-in ship process, effectively solving the problems of poor accuracy, low efficiency, and insufficient safety caused by manual operation in the prior art, and significantly improving the intelligent level and engineering application value of the floating-in ship operation.

[0111] As Figure 4 shown, it is a power positioning floating-in ship control method provided by another embodiment of the present invention, including the following steps:

[0112] Image acquisition and preprocessing.

[0113] Multiple cameras on the jacket continuously collect images of the surrounding sea area and transmit the collected image data to the image processing module M200 in real time. In the embodiments of the present invention, the image acquisition module M100 adopts Figure 2In the shown camera arrangement form, cameras A1, A2, A3, B1, B2, and B3 are respectively installed on the jacket, a total of six cameras, to achieve full coverage of the ship entry area. After the image processing module M200 receives the image data, it first performs de-distortion processing on the image to eliminate the influence of camera lens distortion on the geometric accuracy of the image. Specifically, camera calibration methods such as Zhang Zhengyou calibration method can be used to pre-calibrate the internal parameters and distortion parameters of the camera, and then the collected image is corrected for anti-distortion using the distortion model to obtain the corrected image. Next, the image processing module M200 performs image enhancement processing on the de-distorted image to improve the quality and contrast of the image, facilitating subsequent image recognition algorithm processing. In the embodiment of the present invention, histogram equalization algorithm can be used for image enhancement processing, and this algorithm can effectively enhance the contrast of the image, making the detailed information of the image clearer. After being processed by the image acquisition and preprocessing steps, the original image data is converted into an image with higher quality and less distortion, laying a good foundation for the subsequent target recognition and feature extraction steps.

[0114] Target recognition and feature extraction.

[0115] The image processing module M200 transmits the preprocessed image data to the position and attitude estimation module M300. The position and attitude estimation module M300 first performs object detection on the image to identify the jacket and the floating installation vessel contained in the image. In this embodiment, the object detection algorithm preferably adopts the YOLO (You Only Look Once) algorithm, which is an object detection algorithm based on deep learning and has the advantages of fast detection speed and high accuracy, and can meet the requirements of real-time performance and accuracy in this embodiment of the present invention. Through the YOLO algorithm, the position and attitude estimation module M300 can quickly and accurately identify the jacket and the floating installation vessel in the image and give their position information in the image. Then, the position and attitude estimation module M300 extracts key feature points from the image regions of the identified jacket and floating installation vessel for subsequent image registration and position and attitude estimation. In this embodiment, the feature point extraction algorithm preferably adopts the SIFT (Scale-Invariant Feature Transform) algorithm, which is a local feature descriptor and has the advantages of scale invariance, rotation invariance, illumination invariance, etc., and can effectively cope with the complex changes in the marine environment. Through the SIFT algorithm, the position and attitude estimation module M300 can extract significant feature points such as corners and edges in the images of the jacket and the floating installation vessel and generate a 128-dimensional feature descriptor for each feature point. In order to achieve the alignment of multi-view images, the position and attitude estimation module M300 also needs to perform image registration processing. In this embodiment, the image registration adopts a feature point-based registration method. First, the SIFT algorithm is used to extract feature points in images of different views, and then feature matching is performed based on the feature descriptors to find feature point pairs corresponding to the same physical position in different images and establish the corresponding relationship between the images. Based on the established corresponding relationship of feature points, the position and attitude estimation module M300 can adopt a robust estimation algorithm such as the RANSAC algorithm to estimate the transformation matrix between images of different views, such as the rotation matrix and the translation vector, so as to achieve accurate image registration, unify the images from different cameras into the same coordinate system, and prepare for subsequent depth estimation and three-dimensional reconstruction. After being processed by the object recognition and feature extraction steps, the system obtains accurate object detection results and robust image feature point matching relationships, providing key data support for the subsequent depth estimation and three-dimensional reconstruction steps.

[0116] Depth Estimation and Three-Dimensional Reconstruction.

[0117] After the position and attitude estimation module M300 completes image registration and feature extraction, it enters the depth estimation and 3D reconstruction steps. In this embodiment, stereo vision method is adopted for depth estimation. A multi-view vision system composed of multiple cameras installed on the jacket is used to calculate the depth information of each pixel point in the image through the principle of triangulation. Specifically, the position and attitude estimation module M300 first uses the image registration result obtained in the previous step to establish the matching relationship between the corresponding feature points in the images of different viewpoints. Then, based on the known internal and external parameters of the cameras (the internal parameters are obtained in advance during the camera calibration stage, and the external parameters are the installation position and attitude of the cameras on the jacket), the 3D space point coordinates corresponding to each matching point pair are calculated using the triangulation formula. By performing depth estimation on a large number of feature points in the image, the position and attitude estimation module M300 can construct a sparse 3D point cloud model, which can reflect the approximate 3D structure of the jacket and the floating installation vessel in the scene. To obtain a more refined 3D model, the position and attitude estimation module M300 can also adopt a dense stereo matching algorithm, such as the SGM (Semi-Global Matching) algorithm, to perform depth estimation on each pixel point in the image, thereby constructing a dense 3D depth image. Based on the constructed 3D point cloud model or depth image, the position and attitude estimation module M300 can further calculate the relative distance, azimuth angle, and attitude information of the floating installation vessel relative to the jacket. In this embodiment, the relative distance can be defined as the 3D distance between the center point of the bow of the floating installation vessel and the center point of the jacket; the azimuth angle can be defined as the angle between the bow direction of the floating installation vessel and the X-axis of the jacket coordinate system; the attitude information can include the roll angle, pitch angle, and yaw angle of the floating installation vessel, etc. After depth estimation and 3D reconstruction processing, the position and attitude estimation module M300 obtains the accurate 3D relative position and attitude information of the floating installation vessel relative to the jacket, providing key input data for the subsequent data fusion and position estimation steps and the control instruction generation step.

[0118] Data fusion and position estimation.

[0119] After the position and attitude estimation module M300 completes the depth estimation and 3D reconstruction steps, it enters the data fusion and position estimation steps. In this embodiment, in order to further improve the accuracy and robustness of position and attitude estimation, the position and attitude estimation module M300 fuses the relative position and attitude information of the ship obtained based on image recognition in the steps with the data from the floating installation vessel's own IMU (Inertial Measurement Unit) and GPS (Global Positioning System). Specifically, the position and attitude estimation module M300 first reads the real-time angular velocity and acceleration data from the IMU sensor of the floating installation vessel, and reads the position and velocity information from the GPS receiver. Then, the position and attitude estimation module M300 uses the Kalman filter algorithm to fuse the data from image recognition, IMU, and GPS. In this embodiment, the Kalman filter can be designed as an Extended Kalman Filter (EKF) to handle the non-linear ship motion model and sensor observation model. In the Kalman filter, the position and attitude estimation results based on image recognition, the angular velocity and acceleration measured by the IMU, and the position and velocity information measured by the GPS are all used as observables and input into the filter; at the same time, combined with the pre-established dynamic model of the floating installation vessel, the Kalman filter can recursively estimate the optimal position, attitude, and velocity states of the floating installation vessel. In order to further filter out the high-frequency motions caused by environmental factors such as sea waves and wind currents, the position and attitude estimation module M300 can also perform low-pass filtering on the output results of the Kalman filter, such as using a Butterworth filter or a moving average filter, to filter out the high-frequency noise components in the position and attitude estimation results and only retain the low-frequency motion information, so as to obtain a more stable and reliable low-frequency motion estimation result of the ship. After being processed by the data fusion and position estimation steps, the position and attitude estimation module M300 finally outputs the low-frequency motion information of the floating installation vessel that fuses multi-sensor information and filters out high-frequency noise, providing high-quality and high-robustness position and attitude estimation data for the subsequent control instruction generation step.

[0120] Control instruction generation.

[0121] The control instruction generation module M400 receives the low-frequency motion information of the vessel output by the position and attitude estimation module M300, and generates control instructions for controlling the motion of the floating installation vessel based on this information. In this embodiment, the control instruction generation module M400 adopts the model predictive control (MPC) algorithm, which is an advanced control strategy that can effectively handle multi-variable, strongly coupled, and constrained control problems, and is very suitable for complex scenarios such as floating installation vessel control. Specifically, the control instruction generation module M400 first establishes a dynamic model of the floating installation vessel, which describes the relationship between various moments acting on the vessel in the marine environment, such as propeller thrust, hydrodynamic force, wind-wave-current interference force, etc., and the motion state of the vessel, such as position, attitude, speed, etc. Then, based on the dynamic model, the control instruction generation module M400 predicts the motion trajectory of the floating installation vessel over a period of time in the future. On the basis of the prediction, the control instruction generation module M400 constructs a receding horizon optimization problem, the goal of which is to make the actual motion trajectory of the floating installation vessel as close as possible to the desired trajectory and finally accurately reach the predetermined target position and attitude of the jacket under the premise of satisfying various constraint conditions, such as propeller thrust limit, heading change rate limit, etc. To solve the receding horizon optimization problem, the control instruction generation module M400 can adopt efficient numerical optimization methods such as the sequential quadratic programming (SQP) algorithm to calculate the optimal control sequence over a period of time in the future. Considering the real-time requirement of calculation, the control instruction generation module M400 usually only takes the first control quantity of the optimal control sequence as the control instruction output for the current control period, and sends this control instruction to the dynamic positioning control module M500 for controlling the dynamic positioning system of the floating installation vessel. In the next control period, the control instruction generation module M400 will repeat the above prediction, optimization, and control instruction generation steps based on the latest position and attitude estimation values to achieve receding horizon control and ensure the closed-loop performance and robustness of the control system. After being processed by the control instruction generation step, the system generates control instructions that can drive the floating installation vessel to move precisely, providing a control signal for the subsequent dynamic positioning control execution step.

[0122] Dynamic positioning control execution.

[0123] After receiving the control instructions generated by the control instruction generation module M400, the dynamic positioning control module M500 converts them into specific control operations for the dynamic positioning system of the floating installation vessel, driving the floating installation vessel to move according to the requirements of the control instructions. In this embodiment, the dynamic positioning system of the floating installation vessel generally includes subsystems such as main thrusters, side thrusters, rudders, and ballast systems. The dynamic positioning control module M500 needs to reasonably allocate the control amounts of each subsystem according to the content of the control instructions, coordinate the work of each thruster, rudder, and ballast system, so as to generate the desired thrust, torque, and attitude adjustment torque, and drive the floating installation vessel to generate the desired movement. Specifically, for the main thrusters and side thrusters, the dynamic positioning control module M500 can calculate the rotational speed and thrust direction of each thruster according to the control instructions, and control the operation of the thrusters through the thruster driver. For the rudder, the dynamic positioning control module M500 can calculate the rudder angle according to the control instructions, and control the deflection angle of the rudder through the steering gear, thereby controlling the heading of the ship. For the ballast system, the dynamic positioning control module M500 can adjust the ballast water volume of the ship by controlling the opening degrees of the ballast pumps and ballast valves, thereby adjusting the draft, trim, and heel attitudes of the ship. In order to achieve precise control of the floating installation vessel, the dynamic positioning control module M500 also needs to consider the dynamic characteristics of the ship and the influence of environmental disturbances, and adopt advanced control algorithms, such as vector thrust allocation algorithms, anti-interference control algorithms, etc., to improve the performance and robustness of the control system. After being processed by the dynamic positioning control execution step, the floating installation vessel starts to move according to the requirements of the control instructions under the drive of the dynamic positioning system, gradually adjusts its own position and attitude, and approaches the predetermined target position of the jacket 1.

[0124] Real-time feedback and adjustment.

[0125] The feedback adjustment module M600 plays a crucial closed-loop feedback role in the entire control system, ensuring that the system can operate continuously, stably, and precisely. The feedback adjustment module M600 continuously monitors the relative position and attitude information of the floating installation vessel relative to the jacket output by the position and attitude estimation module M300, and feeds back these real-time position and attitude information to the control instruction generation module M400. Based on the latest position and attitude feedback information, the control instruction generation module re-performs prediction, optimization, and control instruction generation, and sends them to the dynamic positioning control module M500 for execution. Through this real-time feedback adjustment mechanism, the system can form a closed-loop control loop, continuously correct the movement trajectory of the ship, eliminate the influence of various errors and disturbances, ensure that the floating installation vessel always moves precisely along the desired trajectory, and finally reach the predetermined target position of the jacket safely and smoothly, completing the floating installation task.

[0126] Through the collaborative work of the above steps, the embodiment of the present invention provides a complete dynamic positioning floating installation ship control method based on image recognition. This method uses the camera 211 installed on the jacket to obtain image information, and through technologies such as image processing, target recognition, depth estimation, and data fusion, accurately estimates the relative position and attitude of the floating installation ship relative to the jacket, and generates control instructions based on the model predictive control algorithm to drive the dynamic positioning system of the floating installation ship to automatically and accurately complete the floating installation ship task, effectively solving the problems of poor accuracy, low efficiency, and insufficient safety caused by manual operation in the prior art, and significantly improving the intelligent level and engineering application value of the floating installation ship operation.

[0127] In some other alternative embodiments, the control system may further include a human-machine interaction interface, which is used to display information such as the relative position, attitude, and movement trajectory of the ship and the jacket, and allows the operator to perform operations such as parameter setting, mode switching, and manual intervention, further improving the usability and flexibility of the system. Each module in the control system can be implemented in the form of a hardware circuit, a software program, or a combination of software and hardware, and the appropriate implementation method can be selected according to actual needs. These alternative implementation manners all fall within the protection scope of the present invention.

[0128] By adopting the above specific implementation scheme, the present invention can truly realize the full-automatic control of the dynamic positioning floating installation ship process without manual intervention, significantly reducing the labor intensity and skill requirements of the operator, improving the operation efficiency and safety, reducing the operation cost, and being able to work stably and reliably in complex marine environments such as high sea states and low visibility, with broad application prospects and great engineering application value.

[0129] The specific embodiment of the present invention, the dynamic positioning floating installation ship control method and system based on image recognition, uses the high-definition images of multiple cameras on the jacket to achieve centimeter-level high-precision relative position estimation, with strong environmental adaptability and not affected by ship movement. Infrared or image enhancement technology is used to improve the operation ability at night and in bad weather, reduce the manual operation intensity and the risk of misjudgment, and improve the operation safety. Automatic control reduces the dependence on the ship captain, reduces the labor cost and training investment, and precise control can extend the equipment life, with significant economic benefits.

[0130] The above content is a further detailed description of the present application in combination with specific preferred implementation manners, and it cannot be determined that the specific implementation of the present application is only limited to these descriptions. For those of ordinary skill in the technical field to which the present application belongs, without departing from the concept of the present application, several simple deductions or substitutions can be made, and all should be regarded as belonging to the protection scope of the present application.

Claims

1. A dynamic positioning floating jacking ship control method based on image recognition, characterized in that Including the following steps: Set at least one camera on the jacket, with the viewing angle of the camera facing the floatover installation vessel, for real-time acquisition of image data containing the jacket and the floatover installation vessel; Preprocess the acquired image data, including at least image distortion correction and image enhancement processing to improve the image quality; Perform image recognition processing on the preprocessed image, identify the jacket and the floatover installation vessel in the image, extract the image features of the jacket and the floatover installation vessel, and obtain the relative position and attitude information of the floatover installation vessel relative to the jacket based on the image features. The image features include the contours and key structure points of the jacket and the floatover installation vessel; Based on the estimated relative position and attitude information of the floatover installation vessel relative to the jacket, and the preset target position and attitude, generate control instructions using a dynamic positioning control algorithm. The control instructions are used to control the movement of the floatover installation vessel; Send the generated control instructions to the dynamic positioning system of the floatover installation vessel, control the operation of the ship's thrusters, and drive the floatover installation vessel to move according to the control instructions to achieve automatic ship approach control of the floatover installation vessel to the predetermined position of the jacket; Continuously monitor and update the relative position and attitude information of the floatover installation vessel relative to the jacket using the image recognition method, and adjust the control instructions in real time for closed-loop control to ensure that the floatover installation vessel accurately approaches the jacket to complete the floatover ship task.

2. The dynamic positioning floating installation ship control method based on image recognition according to claim 1, wherein The performing image recognition processing on the preprocessed image, identifying the jacket and the floatover installation vessel in the image, extracting the image features of the jacket and the floatover installation vessel, and obtaining the relative position and attitude information of the floatover installation vessel relative to the jacket includes the following steps: Perform image registration on the images from at least one camera to determine the same feature points corresponding to the jacket and the floatover installation vessel in different images; Based on the image registration results and the camera calibration parameters, calculate the three-dimensional spatial position information of the jacket and the floatover installation vessel, thereby estimating the relative position and attitude information of the floatover installation vessel relative to the jacket.

3. The dynamic positioning floating installation ship-in control method based on image recognition according to claim 1, wherein The image recognition processing uses a deep learning-based object detection algorithm, including the following steps: Construct and train a convolutional neural network model, which includes convolutional layers, pooling layers, and fully connected layers; Input the preprocessed image into the convolutional neural network model; Through the forward propagation calculation of the convolutional neural network model, extract the features of the image layer by layer, and generate detection results containing object detection boxes and class confidence levels at the output layer of the model; Based on the detection results, determine the positions of the jacket and the floatover installation vessel in the image, and extract the contour and key structure point features of the jacket and the floatover installation vessel according to the object detection boxes.

4. The dynamic positioning floating installation ship control method based on image recognition according to claim 1, wherein The extracting the image features of the jacket and the floatover installation vessel includes using the SIFT algorithm or the SURF algorithm to extract the local invariant features of the images of the jacket and the floatover installation vessel. Specifically, it includes the following steps: In the images of the jacket and the floatover installation vessel, respectively detect significant feature points, where the feature points are local extreme points or edge points in the images; For each detected feature point, calculate the histogram of gradient directions of its local image region to generate a feature descriptor, which is a vector representing the local characteristics of the image around the feature point; Between images from different perspectives, perform feature matching based on the feature descriptors to find the pair of matching points with the closest distance between the descriptors, and obtain the corresponding relationship of feature points between the images.

5. The dynamic positioning floating installation ship control method based on image recognition according to claim 1, wherein The dynamic positioning control algorithm is a model predictive control algorithm, and the model predictive control algorithm includes the following steps: Establish a dynamic model of the floating installation vessel; Based on the dynamic model of the floating installation vessel, predict the motion trajectory of the floating installation vessel within the prediction time domain according to the rolling time domain strategy, and generate the optimal control instruction within the control period.

6. The dynamic positioning floating installation ship control method based on image recognition according to claim 1, wherein The image preprocessing further includes the step of adaptively adjusting the camera exposure time and image contrast according to the ambient light intensity, specifically including: Detect the ambient light intensity, and when the detected ambient light intensity decreases, increase the camera exposure time; When the detected image contrast decreases, increase the image contrast enhancement coefficient.

7. The dynamic positioning floating dock-in ship control method based on image recognition according to claim 1, wherein It also includes a sensor data fusion step. In the position and attitude estimation step, fuse the data from the sensors, and the sensors include one or more of lidar, infrared thermal imager, GPS, and IMU.

8. The dynamic positioning floating installation ship control method based on image recognition according to claim 8, characterized in that, In the sensor data fusion step, use the Kalman filter algorithm or the extended Kalman filter algorithm to fuse the position and attitude estimation results of image recognition with other sensor data.

9. A dynamic positioning floating jacking ship control system based on image recognition, which is used to implement the dynamic positioning floating jacking ship control method according to claim 1, and is characterized in that, Including: An image acquisition module, arranged on the jacket, including at least one camera, the camera's view facing the floating installation vessel, for real-time acquisition of image data including the jacket and the floating installation vessel; An image processing module, used for preprocessing the acquired image data, including at least performing image distortion correction and image enhancement processing to improve the image quality; A position and attitude estimation module, used for performing image recognition processing on the preprocessed image, recognizing the jacket and the floating installation vessel in the image, extracting the image features of the jacket and the floating installation vessel, and obtaining the relative position and attitude information of the floating installation vessel relative to the jacket based on the image features, where the image features include the contours and key structure points of the jacket and the floating installation vessel; A control instruction generation module, used for generating control instructions based on the estimated relative position and attitude information of the floating installation vessel relative to the jacket and the preset target position and attitude, and the control instructions are used to control the movement of the floating installation vessel; A dynamic positioning control module, used for receiving the control instructions and controlling the dynamic positioning system of the floating installation vessel to drive the floating installation vessel to move according to the control instructions, realizing the automatic approach control of the floating installation vessel to the predetermined position of the jacket; A feedback adjustment module, used for continuously monitoring and updating the relative position and attitude information of the floating installation vessel relative to the jacket using the image recognition method, and adjusting the control instructions in real time for closed-loop control to ensure that the floating installation vessel accurately approaches the jacket to complete the floating approach task.

10. The dynamic positioning floating installation ship control system based on image recognition according to claim 9, characterized in that, It also includes a fault diagnosis and redundancy module, used for real-time monitoring of the working states of each module of the system and performing fault diagnosis.

Images