Six-Degree-of-Freedom Prediction Method and System for Unmanned Vessels Based on Visual Analysis

Abstract

The present invention provides a six-degree-of-freedom prediction method and system for an unmanned ship based on visual analysis, which relates to the field of automated control of unmanned ships. By collecting visual data, inertial measurement unit data and historical navigation data of the unmanned ship, multimodal data of the unmanned ship is obtained; the acquired multimodal data is fused and preprocessed, and feature extraction is performed on the visual data. Time series analysis is performed on the inertial measurement unit data and the historical navigation data. Position and attitude prediction are performed separately using a multi-task learning framework, and a six-degree-of-freedom prediction model for the unmanned ship is constructed; the parameters and data fusion strategy of the six-degree-of-freedom prediction model are adjusted according to real-time environmental data, and the multimodal data processing, six-degree-of-freedom prediction model and adaptive real-time correction mechanism are integrated into the unmanned ship control system to predict the six-degree-of-freedom state of the unmanned ship and perform precise navigation control, thereby realizing high-precision, real-time and intelligent attitude and position prediction of the unmanned ship.

Description

Technical Field

[0001] The present invention relates to the field of automated control of unmanned ships, and in particular to a six-degree-of-freedom prediction method and system for unmanned ships based on visual analysis. Background Art

[0002] In the actual application scenarios of unmanned vessels, the water environment is often highly uncertain (such as random changes in waves, wind speed, and obstacles). This places higher demands on the six-degree-of-freedom (6DOF) prediction methods for unmanned vessels. Existing 6DOF prediction methods for unmanned vessels mostly rely on single sensor data (such as visual or inertial sensors), which easily affects navigation accuracy in complex water environments.

[0003] While some systems have incorporated multi-sensor fusion technology, they still face deficiencies in multimodal data processing and real-time performance. Furthermore, resource-constrained unmanned aerial vehicles struggle to implement complex, high-precision prediction algorithms, hindering their widespread adoption in practical applications. Therefore, an innovative approach based on multimodal data fusion and AI-powered prediction models is urgently needed to improve the accuracy and adaptability of six-degree-of-freedom state prediction. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to propose a six-degree-of-freedom prediction method and system for unmanned ships based on visual analysis, aiming to improve the navigation performance of unmanned ships in complex water environments, and to achieve high-precision, real-time and intelligent posture and position prediction of unmanned ships through multimodal data fusion, AI-based six-degree-of-freedom prediction model and adaptive real-time correction strategy.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] Based on the above objectives, in a first aspect, the present invention provides a six-degree-of-freedom prediction method for an unmanned ship based on visual analysis, comprising the following steps:

[0007] Collect the unmanned ship's visual data, inertial measurement unit (IMU) data, and historical navigation data to obtain multimodal data of the unmanned ship;

[0008] The acquired multimodal data is fused and preprocessed, and features are extracted from the visual data. Time series analysis is performed on the IMU data and historical navigation data. Position and attitude prediction are performed using a multi-task learning framework, and a six-degree-of-freedom prediction model for the unmanned vessel is constructed.

[0009] Adjust the parameters and data fusion strategy of the six-degree-of-freedom prediction model based on real-time environmental data, and correct the prediction results through an adaptive real-time correction mechanism to optimize the navigation accuracy and path planning accuracy of the unmanned vessel;

[0010] Multimodal data processing, six-degree-of-freedom prediction model and adaptive real-time correction mechanism are integrated into the unmanned ship control system to predict the six-degree-of-freedom state of the unmanned ship and perform precise navigation control.

[0011] As a further solution of the present invention, the visual data collects images or point cloud data of the external environment in real time through the visual sensors carried on the unmanned ship, the IMU data collects dynamic information of the acceleration, angular velocity and direction of the ship through inertial sensors, and the historical navigation data is used to provide prior knowledge support by collecting historical navigation data of the unmanned ship.

[0012] As a further solution of the present invention, the visual data includes images or point cloud data obtained by cameras, lidars or other visual sensors, and the IMU data includes the acceleration, angular velocity and direction data of the unmanned ship; the historical navigation data includes the navigation track, speed, heading, wind speed and wave height information of the unmanned ship.

[0013] As a further solution of the present invention, a convolutional neural network (CNN) is used to extract features from the visual data to identify and capture key features of the environment, including obstacles, tracks, and water surface information; a long short-term memory network (LSTM) is combined to perform time series analysis on the IMU data and historical navigation data to capture the dynamic changes of the unmanned ship and trends in its future state.

[0014] As a further solution of the present invention, when fusing and preprocessing the acquired multimodal data, the visual features, IMU features and historical data features are spliced to generate a multimodal feature vector. Different weights are assigned to each data source through weighted fusion. The weights are dynamically adjusted according to the real-time environmental data. The multimodal fusion feature vector F fuse The calculation method is as follows, using the weight α v , α i and α h Weighted superposition visual feature F vis 、IMU feature F imu and historical characteristics F hist :

[0015] F fuse =α v ·F vis +α i ·F imu +α h ·F hist

[0016] Among them, α v , α i and α h Represent the weights of vision, IMU and historical data respectively.

[0017] As a further solution of the present invention, the current environmental state is classified based on the real-time collected environmental data, and the weight of the IMU data is increased according to a preset rule to maintain posture stability; wherein:

[0018] When the light intensity is lower than the threshold L min When , reduce the weight of visual data α v , the calculation formula is:

[0019]

[0020] When the wave height exceeds the threshold H max When , increase the weight α of IMU data i :

[0021] α i =α i +β·(HH max )

[0022] When the wind speed exceeds the set value W max When , reduce the historical data weight α h , enhancing the impact of real-time data on predictions:

[0023]

[0024] As a further solution of the present invention, the multi-task learning framework is further used to divide the six-degree-of-freedom prediction task into two parts: position prediction and attitude prediction, predict the six-degree-of-freedom state of the unmanned ship in real time, and output estimated values of position (x, y, z) and attitude (roll angle, pitch angle, yaw angle) respectively; including:

[0025] Position prediction: Estimate the current position of the unmanned vessel in the water using the x, y, and z coordinates output by the model;

[0026] Attitude prediction: Estimate the attitude state of the unmanned vessel through the roll angle, pitch angle, and yaw angle output by the model.

[0027] As a further solution of the present invention, the six-degree-of-freedom prediction method for unmanned ships based on visual analysis also includes synchronously marking the six-degree-of-freedom labels of each data through GPS and attitude sensors, including position (x, y, z) and attitude (roll angle, pitch angle, yaw angle), and each data records a set of position and attitude parameters, wherein obstacles in the visual data are marked with the position, shape and size information of the obstacles to identify surface obstacles; historical navigation data are segmented by trajectory and marked with path labels to identify the target position, path curvature and navigation difficulty of each trajectory.

[0028] As a further solution of the present invention, in the constructed six-degree-of-freedom prediction model of the unmanned ship, the visual data is input in the form of RGB images, each frame of the image is 224x224 pixels in size, the data format is a four-dimensional tensor, the data type is float32, and all pixel values are normalized to the range of [0,1]; the IMU data includes acceleration and angular velocity, which are three-dimensional vectors respectively, representing dynamic changes on the x, y, and z axes. The data format of the IMU data is a two-dimensional tensor, and the data type is float32; the data input format of the historical data is a two-dimensional tensor, and the data type is float32.

[0029] As a further solution of the present invention, when the six-degree-of-freedom prediction model of the unmanned ship is constructed to perform position prediction output, the six-degree-of-freedom position prediction output is a three-dimensional vector, representing the coordinate prediction values of the unmanned ship in the x, y, and z directions; the data format is a two-dimensional tensor, defined as [batch_size,3], where each sample contains three coordinate parameters, the data type is float32, and the output value is usually not normalized.

[0030] As a further solution of the present invention, when the six-degree-of-freedom prediction model of the unmanned ship is constructed to perform attitude prediction output, the six-degree-of-freedom attitude prediction output is a three-dimensional vector, representing the roll angle, pitch angle and yaw angle of the unmanned ship, defined as [roll, pitch, yaw]; the data format is a two-dimensional tensor, defined as [batch_size, 3], each sample contains three angle parameters, the data type is float32, and the output angle value is usually expressed in radians.

[0031] As a further solution of the present invention, the six-degree-of-freedom prediction model of the unmanned ship is constructed. In the six-degree-of-freedom prediction stage, when predicting position and attitude, the six-degree-of-freedom prediction model outputs position and attitude, namely [x, y, z, roll, pitch, yaw]. A multi-task loss function is used, and the position prediction uses the mean square error (MSE) loss L position , pose prediction uses cosine similarity loss L attitude ,in:

[0032]

[0033] Total loss function L total for:

[0034] L total =λ1·L position +λ2·L attitude

[0035] Among them, λ1 and λ2 are loss weights.

[0036] As a further solution of the present invention, by weighted fusion, each data source is assigned different weights to perform environmental perception and weight adjustment, collect environmental information including light intensity, wave height, and obstacle distance in real time, and calculate weight adjustment parameters; set the environmental impact factor E to dynamically adjust the fusion weight of vision, IMU, and historical data. The formula is as follows

[0037] E=α·L+β·W+γ·D

[0038] Where L is the light intensity, W is the wave height, D is the distance to the nearest obstacle, and α, β, and γ are the corresponding adjustment coefficients, which are determined based on model experiments;

[0039] The weight update mechanism is to update the visual data weight α according to the value of the environmental factor E. v , IMU data weight α i and historical data weight α h Dynamic update: If the visual data has a greater impact in low-light conditions, the weight adjustment formula is:

[0040]

[0041] When the light intensity L is lower than the threshold L threshold When the visual data weight α v Reduce, while the IMU data weight α i Increase to ensure overall prediction accuracy.

[0042] In a second aspect, the present invention provides a six-degree-of-freedom prediction system for an unmanned ship based on visual analysis, comprising:

[0043] Data acquisition module: used to collect the unmanned ship's visual data, inertial measurement unit (IMU) data, and historical navigation data to obtain the unmanned ship's multimodal data;

[0044] A model building module is used to fuse and preprocess the acquired multimodal data, extract features from the visual data, perform time series analysis on the IMU data and historical navigation data, predict the position and attitude of the unmanned vessel using a multi-task learning framework, and build a six-degree-of-freedom prediction model for the unmanned vessel;

[0045] The model optimization module is used to adjust the parameters and data fusion strategy of the six-degree-of-freedom prediction model according to real-time environmental data, and correct the prediction results through an adaptive real-time correction mechanism to optimize the navigation accuracy and path planning accuracy of the unmanned ship;

[0046] The integrated deployment module is used to integrate multimodal data processing, six-degree-of-freedom prediction model and adaptive real-time correction mechanism into the unmanned ship control system to predict the six-degree-of-freedom state of the unmanned ship and perform precise navigation control.

[0047] In another aspect of the present invention, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, any one of the above-mentioned six-degree-of-freedom prediction methods of an unmanned ship based on visual analysis according to the present invention is executed.

[0048] In another aspect of the present invention, a computer-readable storage medium is provided, which stores computer program instructions, and when the computer program instructions are executed, any one of the above-mentioned six-degree-of-freedom prediction methods of an unmanned ship based on visual analysis according to the present invention is implemented.

[0049] Compared with the existing technology, the six-degree-of-freedom prediction method and system for unmanned ships based on visual analysis proposed in the present invention have the following beneficial effects:

[0050] 1. Improved navigation accuracy and safety for unmanned vessels. By fusing visual data with inertial measurement unit (IMU) data and historical navigation data, the six degrees of freedom (DOF) of the unmanned vessel (the vessel) can be more accurately estimated. This provides precise attitude and motion state prediction for unmanned vessels navigating complex waters, effectively improving navigation accuracy and significantly enhancing navigation safety.

[0051] 2. Enhanced adaptability and robustness. The method of this invention can cope with complex and dynamic water environments. Under varying sea conditions, weather conditions, and navigation channels, it achieves dynamic prediction of six degrees of freedom through real-time visual analysis and data fusion. Especially when encountering interference from external factors such as water surface fluctuations and wind speed changes, the deep learning-based model can adaptively adjust to maintain prediction accuracy and stability, ensuring the robustness of the unmanned vessel in uncertain environments.

[0052] 3. Improved autonomous navigation capabilities for unmanned vessels. By acquiring and processing visual data, IMU data, and historical navigation data in real time, the method of this invention enables autonomous navigation and decision support for unmanned vessels in various environments. Especially in remote or signal-deprived areas, unmanned vessels can independently plan navigation paths and avoid obstacles, reducing reliance on manual operation and external navigation systems, significantly improving their autonomy.

[0053] 4. Improved prediction accuracy through data fusion. The method of this invention utilizes multimodal data fusion, combining visual perception with inertial navigation information, to maximize the complementary nature of various sensor data. Visual data provides rich environmental information, while IMU data supplements the ship's motion information. Historical navigation data further provides a time series pattern of navigation status. The combination of these three significantly improves the accuracy of six-degree-of-freedom predictions.

[0054] 5. Optimizes the performance of the unmanned vessel's control system. By accurately predicting the unmanned vessel's six degrees of freedom (DOF) state, the method of this invention provides precise attitude and motion feedback for the unmanned vessel's control system, optimizing control strategies and improving maneuvering accuracy. Real-time 6DOF predictions provide data support for path planning, heading adjustment, speed control, and other functions, ensuring the unmanned vessel's efficient navigation according to its intended destination.

[0055] 6. Enhanced adaptability to complex water environments: The method of this invention is particularly suitable for applications in complex water environments, such as those with narrow waterways, large surface fluctuations, or dense obstacles. Visual analysis can identify factors such as obstacles on the water surface and changing water flow directions, while the deep learning model can adaptively adjust the ship's motion prediction based on environmental changes, helping the unmanned vessel better cope with complex marine or inland water environments.

[0056] In summary, the unmanned ship six-degree-of-freedom prediction method and system based on visual analysis of the present invention, through multimodal data fusion and advanced deep learning technology, not only improves the navigation accuracy and safety of unmanned ships in complex environments, but also promotes the development of unmanned ship technology towards a more intelligent, autonomous and reliable direction. It has significant application value and broad market prospects.

[0057] These and other aspects of the present application will be more clearly understood in the following description of the embodiments. It should be understood that the above general description and the following detailed description are merely exemplary and explanatory and cannot limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or related technologies, the following briefly introduces the drawings required for the exemplary embodiments or related technical descriptions. The drawings are used to provide a further understanding of the present invention and constitute part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the drawings:

[0059] Figure 1 Flowchart of a six-degree-of-freedom prediction method for an unmanned ship based on visual analysis according to an embodiment of the present invention.

[0060] Figure 2 This is a structural block diagram of an unmanned ship six-degree-of-freedom prediction system based on visual analysis according to an embodiment of the present invention. DETAILED DESCRIPTION

[0061] Below, the present application is further described in conjunction with the accompanying drawings and specific implementation methods. It should be noted that, under the premise of no conflict, the various embodiments or technical features described below can be arbitrarily combined to form new embodiments.

[0062] To make the purpose, technical solutions and advantages of the present invention more clearly understood, the following is a further detailed description of the embodiments of the present invention in conjunction with specific embodiments and with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application.

[0063] It should be noted that all expressions using "first" and "second" in the embodiments of the present invention are intended to distinguish two non-identical entities or non-identical parameters with the same name. Therefore, "first" and "second" are used for convenience of expression only and should not be understood as limitations on the embodiments of the present invention. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, other steps or units inherent to a process, method, system, product, or device that includes a series of steps or units.

[0064] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0065] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, combined, or partially merged, so the actual execution order may vary depending on the actual situation.

[0066] The following describes some embodiments of the present application in detail with reference to the accompanying drawings. In the absence of conflict, the following embodiments and features therein may be combined with each other.

[0067] In view of the fact that existing six-degree-of-freedom prediction methods for unmanned vessels are easily affected by navigation accuracy in complex water environments, multi-sensor fusion technology has been introduced. However, there are still deficiencies in the processing and real-time performance of multimodal data. This paper proposes a six-degree-of-freedom prediction method and system for unmanned vessels based on visual analysis, aiming to improve the navigation performance of unmanned vessels in complex water environments. Through multimodal data fusion, an AI-based six-degree-of-freedom prediction model, and an adaptive real-time correction strategy, high-precision, real-time, and intelligent attitude and position prediction of unmanned vessels is achieved.

[0068] See also Figure 1 As shown, an embodiment of the present invention provides a six-degree-of-freedom prediction method for an unmanned ship based on visual analysis, the method comprising the following steps:

[0069] Step S10: collecting visual data, inertial measurement unit (IMU) data, and historical navigation data of the unmanned ship to obtain multimodal data of the unmanned ship.

[0070] In this step, the visual data collects images or point cloud data of the external environment in real time through the visual sensors carried by the unmanned ship, the IMU data collects dynamic information of the acceleration, angular velocity and direction of the ship through the inertial sensor, and the historical navigation data is used to provide prior knowledge support by collecting the historical navigation data of the unmanned ship.

[0071] Among them, the visual data includes images or point cloud data obtained by cameras, lidar or other visual sensors, the IMU data includes the acceleration, angular velocity and direction data of the unmanned ship; the historical navigation data includes the navigation track, speed, heading, wind speed and wave height information of the unmanned ship.

[0072] In this embodiment, data collected by multiple sensors on the unmanned ship are integrated, including visual data, inertial measurement unit (IMU) data, and historical navigation data.

[0073] Step S20: fuse and preprocess the acquired multimodal data, extract features from the visual data, perform time series analysis on the IMU data and historical navigation data, use a multi-task learning framework to predict position and attitude respectively, and build a six-degree-of-freedom prediction model for the unmanned ship.

[0074] In this embodiment, during multimodal data collection and preprocessing, the prediction accuracy and robustness of the six-degree-of-freedom state are improved by combining multiple sensor data on the unmanned vessel (such as vision, IMU, historical data, etc.). The first step is data collection and preprocessing.

[0075] Visual Data Acquisition: The unmanned vessel is equipped with cameras to capture images and videos of the surrounding environment. This visual data is used to capture obstacles, waterways, and surface features in the external environment. Processing steps include denoising, grayscale conversion, and scale normalization to enhance image quality and consistency.

[0076] IMU data acquisition: The inertial measurement unit (IMU) captures the vessel's acceleration and angular velocity in real time. This data reflects the vessel's attitude changes, such as roll, pitch, and yaw. IMU data is filtered (such as by a Kalman filter or low-pass filter) to remove high-frequency noise and improve data stability.

[0077] Historical data collection: The UAV's past navigation data, such as historical navigation trajectories, attitude angles, and speeds, is used as a reference for model training and calibration. Key features of the navigation trajectories are extracted through cluster analysis or regression analysis.

[0078] Data preprocessing: Vision, IMU, and historical data require consistent normalization and filtering to ensure data quality and format consistency. In particular, in vision data processing, convolutional neural networks (CNNs) process image data through feature extraction, while IMU and historical data are processed as time series.

[0079] In this step, a convolutional neural network (CNN) is used to extract features from the visual data to identify and capture key features of the environment, including obstacles, tracks, and water surface information. A long short-term memory network (LSTM) is combined to perform time series analysis on the IMU data and historical navigation data to capture the dynamic changes of the unmanned ship and trends in its future state.

[0080] During feature extraction, visual feature extraction uses a convolutional neural network (CNN) to process image data and extract high-dimensional features from the environment, such as the location, shape, texture, and navigation path of obstacles. CNN automatically learns these features and converts them into feature vectors. MU data feature extraction uses a long short-term memory network (LSTM) or a recurrent neural network (RNN) to process IMU data and extract the dynamic motion characteristics of the ship. Historical data feature extraction uses clustering or regression analysis to extract motion patterns from past navigation data, providing prior knowledge support for the model.

[0081] In this embodiment, a multimodal data fusion strategy is proposed. The fusion process not only leverages the strengths of various sensor types but also dynamically adjusts different data sources through methods such as weighted averaging and adaptive updating, making the predicted input data more robust and accurate. During the fusion process, visual data provides information about the unmanned vessel's external environment, IMU data captures the vessel's dynamic motion characteristics, and historical navigation data provides prior knowledge support for the model. By combining multimodal data, the present invention can effectively improve the accuracy of predicting the six-degree-of-freedom state of an unmanned vessel in complex dynamic environments.

[0082] Among them, the visual features, IMU features and historical data features are spliced together to generate a multimodal feature vector. These spliced feature vectors are further processed through a fully connected layer (multi-layer perceptron, MLP) to capture the correlation between different data sources. When fusing and preprocessing the acquired multimodal data, the visual features, IMU features and historical data features are spliced together to generate a multimodal feature vector. Through weighted fusion, different weights are assigned to each data source. The weights are dynamically adjusted according to the real-time environmental data. The multimodal fusion feature vector F fuse The calculation method is as follows, using the weight α v , α i and α h Weighted superposition visual feature F vis 、IMU feature Fimu and historical characteristics F h ist :

[0083] F fuse =α v ·F vis +α i ·F imu +α h ·F h ist

[0084] Among them, α v , α i and α h Represent the weights of vision, IMU and historical data respectively.

[0085] Based on the real-time collected environmental data, the current environmental status is classified and the weight of the IMU data is increased according to the preset rules to maintain posture stability.

[0086] When the light intensity is lower than the threshold L min When , reduce the weight of visual data α v , the calculation formula is:

[0087]

[0088] When the wave height exceeds the threshold H max When , increase the weight α of IMU data i :

[0089] α i =α i +β·(HH max )

[0090] When the wind speed exceeds the set value W max When , reduce the historical data weight α h , enhancing the impact of real-time data on predictions:

[0091]

[0092] In this embodiment, the multi-task learning framework is also used to divide the six-degree-of-freedom prediction task into two parts: position prediction and attitude prediction, predicting the six-degree-of-freedom state of the unmanned ship in real time, and outputting estimated values of position (x, y, z) and attitude (roll angle, pitch angle, yaw angle) respectively; including:

[0093] Position prediction: Estimate the current position of the unmanned vessel in the water using the x, y, and z coordinates output by the model;

[0094] Attitude prediction: Estimate the attitude state of the unmanned vessel through the roll angle, pitch angle, and yaw angle output by the model.

[0095] In this embodiment, the six-degree-of-freedom prediction method for unmanned ships based on visual analysis also includes synchronously marking the six-degree-of-freedom labels of each data through GPS and attitude sensors, including position (x, y, z) and attitude (roll angle, pitch angle, yaw angle), and each data records a set of position and attitude parameters, wherein obstacles in the visual data are marked with the position, shape and size information of the obstacles to identify surface obstacles; historical navigation data are segmented by trajectory and marked with path labels to identify the target position, path curvature and navigation difficulty of each trajectory.

[0096] Among them, the visual data in the constructed six-degree-of-freedom prediction model of the unmanned ship is input in the form of RGB images, each frame of the image is 224x224 pixels in size, the data format is a four-dimensional tensor, the data type is float32, and all pixel values are normalized to the range of [0,1]; the IMU data contains acceleration and angular velocity, which are three-dimensional vectors respectively, representing dynamic changes on the x, y, and z axes. The data format of the IMU data is a two-dimensional tensor, and the data type is float32; the data input format of the historical data is a two-dimensional tensor, and the data type is float32.

[0097] Among them, when the constructed six-degree-of-freedom prediction model of the unmanned ship performs position prediction output, the six-degree-of-freedom position prediction output is a three-dimensional vector, which represents the coordinate prediction value of the unmanned ship in the x, y, and z directions; the data format is a two-dimensional tensor, defined as [batch_size,3], where each sample contains three coordinate parameters, the data type is float32, and the output value is usually not normalized.

[0098] In this embodiment, when the six-degree-of-freedom prediction model of the constructed unmanned ship performs attitude prediction output, the six-degree-of-freedom attitude prediction output is a three-dimensional vector, representing the roll angle, pitch angle and yaw angle of the unmanned ship, defined as [roll, pitch, yaw]; the data format is a two-dimensional tensor, defined as [batch_size, 3], each sample contains three angle parameters, the data type is float32, and the output angle value is usually expressed in radians.

[0099] The six-degree-of-freedom prediction model of the unmanned ship is constructed. In the six-degree-of-freedom prediction stage, when predicting position and attitude, the six-degree-of-freedom prediction model outputs position and attitude, namely [x, y, z, roll, pitch, yaw]. The multi-task loss function is used, and the position prediction uses the mean square error (MSE) loss L position , pose prediction uses cosine similarity loss L attitude ,in:

[0100]

[0101] Total loss function L total for:

[0102] L total =λ1·L position +λ2·L attitude

[0103] Among them, λ1 and λ2 are loss weights.

[0104] Step S30: Adjust the parameters and data fusion strategy of the six-degree-of-freedom prediction model according to the real-time environmental data, correct the prediction results through the adaptive real-time correction mechanism, and optimize the navigation accuracy and path planning accuracy of the unmanned ship.

[0105] In this step, environmental perception and weight adjustment are performed by assigning different weights to each data source through weighted fusion. Environmental information including light intensity, wave height, and obstacle distance is collected in real time, and weight adjustment parameters are calculated. The environmental impact factor E is set to dynamically adjust the fusion weight of vision, IMU, and historical data. The formula is as follows

[0106] E=α·L+β·W+γ·D

[0107] Where L is the light intensity, W is the wave height, D is the distance to the nearest obstacle, and α, β, and γ are the corresponding adjustment coefficients, which are determined based on model experiments;

[0108] The weight update mechanism is to update the visual data weight α according to the value of the environmental factor E. v , IMU data weight α i and historical data weight α h Dynamic update: If the visual data has a greater impact in low-light conditions, the weight adjustment formula is:

[0109]

[0110] When the light intensity L is lower than the threshold L threshold When the visual data weight α v Reduce, while the IMU data weight α i Increase to ensure overall prediction accuracy.

[0111] The dynamic weight adjustment mechanism includes:

[0112] Environmental perception and weight adjustment: Real-time collection of environmental information (such as light intensity, wave height, obstacle distance, etc.) and calculation of weight adjustment parameters. Set the environmental impact factor E to dynamically adjust the fusion weight of vision, IMU, and historical data. The formula is as follows:

[0113] E=α·L+β·W+γ·D

[0114] Where L is the light intensity, W is the wave height, D is the distance to the nearest obstacle, and α, β, and γ are the corresponding adjustment coefficients, which are determined based on model experiments.

[0115] Weight update mechanism: According to the value of the environmental factor E, the visual data weight α v , IMU data weight α i and historical data weight α h Dynamic update. Assuming that visual data has a greater impact in low-light conditions, the weight adjustment formula is:

[0116]

[0117] When the light intensity L is lower than the threshold L threshold When the visual data weight α v Reduce, while the IMU data weight α i Increase to ensure overall prediction accuracy.

[0118] When calculating weight update, initialize weight α v =0.4,α i =0.3,α h =0.3; when the environmental factor E changes, the weight is dynamically updated using the following formula:

[0119]

[0120] Where ΔE=E current -E base , E base This is the baseline value under normal conditions.

[0121] The updated weights need to be normalized to ensure that the sum is 1, that is:

[0122] α v +α i +α h =1.

[0123] By introducing an adaptive real-time correction mechanism and continuously collecting environmental data (such as wind speed, wave height, obstacle location, etc.), the model parameters and data fusion strategy are adjusted in real time to enhance the system's navigation performance in complex environments. The correction process adopts a rule-driven and AI-based approach. First, a preliminary correction is performed based on real-time environmental data, and then the AI algorithm is used to further optimize the correction results, enabling the unmanned vessel to respond quickly to drastic environmental changes and dynamically adjust the six-degree-of-freedom state. This adaptive correction mechanism significantly improves the navigation accuracy and path planning capabilities of the unmanned vessel, especially under sudden environmental changes (such as sudden obstacles or bad weather).

[0124] Step S40: Integrate multimodal data processing, a six-degree-of-freedom prediction model, and an adaptive real-time correction mechanism into the unmanned vessel control system to predict the six-degree-of-freedom state of the unmanned vessel and perform precise navigation control.

[0125] In this step, the weather monitoring and early warning mechanism is activated to adjust strategies for sudden weather changes. This mechanism uses onboard meteorological sensors (such as temperature, humidity, air pressure, and wind speed sensors) as well as onboard environmental sensors (such as water temperature and wave height sensors) to monitor weather parameters in real time. If signs of a dramatic weather change (such as a sharp increase in wind speed or a sudden drop in air pressure) are detected, the system automatically enters early warning mode.

[0126] Weight adjustment strategy under sudden weather changes:

[0127] Rule 1: When the wind speed exceeds a preset threshold (e.g., 15 m / s), increase the weight of the IMU data and reduce the weight of the visual data to ensure more stable attitude and speed adjustments. The specific adjustment formula is:

[0128] α i =α i +δ·(WW threshold )

[0129]

[0130] Where W is the current wind speed, W threshold is the set wind speed threshold.

[0131] Rule 2: When the air pressure drops by more than 20hPa, the system determines that rain or storms may occur, reduces the weight of historical data, and enhances the responsiveness of real-time data to the current environment to cope with sudden changes in the environment:

[0132]

[0133] Among them, P is the current air pressure, P normal It is the reference value of normal air pressure.

[0134] Rule 3: When the wave height exceeds the set value (such as 3 meters), increase the proportion of IMU data and introduce multi-frame image weighted processing to weighted average the frame information in the visual data to reduce the impact of single-frame errors on judgment.

[0135] In the six-degree-of-freedom prediction method of the unmanned ship based on visual analysis of the present invention, the preprocessing of the image data includes:

[0136] Denoising: First, a Gaussian filter is applied to the collected image data for denoising, using a Gaussian kernel with a standard deviation of 1 (e.g., a 3x3 or 5x5 matrix). The specific operation is to convolve the image under the Gaussian kernel to remove noise interference while retaining the main edge features of the image to reduce interference caused by ambient lighting changes or water surface reflections.

[0137] Normalization: Image data is normalized to the range [0, 1]. This normalization process involves subtracting the image's average grayscale value from each pixel, then dividing it by the standard deviation of the image's pixel values. This ensures a uniform distribution of values when input into the model, minimizing the impact of inconsistent brightness.

[0138] Scaling: Use bilinear interpolation to resize images to the standard size required by the model (e.g., 224x224 pixels), ensuring that images of different scenes have the same resolution. Maintaining a uniform resolution helps stabilize the model during the feature extraction phase.

[0139] Time series smoothing: Time series smoothing is performed on continuous frame images. The specific method is to fuse the pixel values of each frame image with the pixel values of the previous and next frames in a weighted average manner (for example, the weight of the previous frame is 0.2, the weight of the current frame is 0.6, and the weight of the next frame is 0.2) to reduce the inter-frame mutation caused by device jitter and improve the accuracy of feature extraction.

[0140] The preprocessing of IMU data involves applying a Kalman filter to remove random noise from the IMU data. Assuming that acceleration and angular velocity changes are state variables, the data is processed in two steps: prediction and update:

[0141] Prediction step: Predict the current state based on the state estimate and control input at the previous moment.

[0142] Update step: The state prediction is adjusted based on the actual measured value at the current moment to obtain filtered acceleration and angular velocity values. Kalman filtering can significantly reduce noise interference and make IMU data smoother, making it particularly suitable for attitude estimation.

[0143] Smooth high-frequency noise in IMU data. A Butterworth low-pass filter with a cutoff frequency of 5 Hz is used to remove high-frequency components from the acceleration and angular velocity data, retaining only the low-frequency components to ensure data stability. This method involves determining the noise frequency range through frequency domain transformation and then applying a filtering formula to reduce high-frequency components. Sensor zero-bias calibration and temperature compensation are performed on IMU data to avoid errors caused by temperature changes or sensor drift. For example, by regularly collecting zero-input data, calculating the offset value, and subtracting it from the actual data, data accuracy is improved.

[0144] For preprocessing of historical navigation data, key feature extraction uses K-means clustering to extract features from historical navigation trajectory data. First, the historical data is clustered based on features such as speed, attitude angle, and path curvature. The features of each cluster center are then extracted and used as prior inputs for the model. This method effectively summarizes historical navigation patterns and reduces data redundancy. Numerical features such as speed and attitude are scaled to the range [0, 1]. The normalization formula is: Where x represents the data feature value, ensuring that different data sources are scaled consistently during fusion to prevent any one feature from being overweighted. The autoregressive integrated moving average (ARIMA) model is used to model and analyze the time series characteristics of historical trajectories, extracting trends in speed and attitude. This is used to predict the unmanned vessel's motion patterns in the current environment, providing a reference for real-time navigation.

[0145] During data consistency processing, timestamp alignment is performed. For multi-sensor data synchronization, bilinear interpolation is used to align timestamps from different data sources to ensure temporal consistency among visual, IMU, and historical data. During format conversion, the processed data is uniformly converted to the tensor format required for model input. For example, visual data is a four-dimensional tensor of [batch, height, width, channels], while IMU and historical data are two-dimensional tensors of [batch, features]. This ensures seamless data integration when input into the fusion model.

[0146] In this embodiment, when acquiring model training data and constructing a dataset, the data sources include:

[0147] Visual data: collected by the high-definition camera on the unmanned vessel in different environments, including open waters, dense obstacle areas, strong light and low light conditions, to cover various situations that the unmanned vessel may encounter.

[0148] IMU data: The acceleration and angular velocity data collected in real time by the inertial measurement unit (IMU) on the unmanned vessel ensures that the data can accurately reflect the motion posture of the vessel.

[0149] Historical navigation data: The trajectory, speed, posture and other data of unmanned vessels sailing in different waters are selected to construct a representative historical data set.

[0150] The data collection methods are:

[0151] Acquisition frequency: The acquisition frequency of visual data and IMU data is set to 10 Hz, that is, 10 frames per second, to ensure that the model can respond to water surface changes and posture adjustments in real time.

[0152] Environmental conditions: Data collection covers weather conditions such as sunny days, cloudy days, and haze, and is collected under conditions of different wind speeds and wave heights to ensure that the data is widely representative.

[0153] Sample size: At least 5,000 data records are collected for model training. Each record contains a visual image frame, corresponding IMU acceleration and angular velocity data, and historical flight trajectory. These data samples ensure sufficient diversity in the training set to avoid model overfitting.

[0154] When implementing data labeling and annotation strategies, position and attitude tags synchronize the GPS and attitude sensors to annotate each piece of data with six degrees of freedom (6DOF) tags, including position (x, y, z) and attitude (roll, pitch, and yaw). Each piece of data records a set of position and attitude parameters. Obstacle tags are manually annotated or automatically annotated using object detection models such as YOLO. The location, shape, and size of obstacles in the visual data are annotated to help the model accurately identify surface obstacles. Path tags segment historical navigation data into segments and add path tags, identifying the target position, path curvature, and navigation difficulty of each trajectory, so that the model can learn navigation patterns in different environments.

[0155] The present invention also includes data enhancement, wherein visual data enhancement includes:

[0156] Brightness change: Adjust the brightness of the image (within the range of ±30%) to simulate different lighting conditions to improve the model's adaptability to lighting changes.

[0157] Rotation and Cropping: Randomly rotate the image (±10°) and perform random cropping to increase the model’s perspective diversity and anti-interference ability.

[0158] Adding noise: Add random Gaussian noise to the image to simulate factors such as water surface reflection and sensor noise, thereby enhancing the model's noise resistance.

[0159] IMU data enhancement includes:

[0160] Data perturbation: Add a small Gaussian perturbation (mean 0, standard deviation 0.1) to the IMU data to simulate jitter and measurement errors in the real environment, ensuring the robustness of the model in different devices and environments.

[0161] Time offset: Apply a time series offset (±1 to 2 time steps) to the IMU data to simulate the effects of sensor delay or data synchronization errors and improve the temporal robustness of the model.

[0162] Historical data enhancements include:

[0163] Trajectory smoothing: Smoothing the original trajectory data through interpolation enhances data continuity and enables the model to learn historical navigation patterns more smoothly.

[0164] Random Path Deformation: Randomly deform the trajectory data to make the navigation path slightly deviate from the original route. This simulates the small route adjustments in real navigation and improves the model's ability to cope with trajectory changes.

[0165] In this embodiment, the output data format and content of the six-degree-of-freedom prediction model, the six-degree-of-freedom position prediction output is:

[0166] The position output is a three-dimensional vector, which represents the predicted coordinates of the unmanned ship in the x, y, and z directions.

[0167] The data format is a two-dimensional tensor defined as [batch_size,3], where each sample contains three coordinate parameters.

[0168] The data type is float32, and the output value is usually not normalized so that it can be directly used for subsequent navigation and control.

[0169] For example, the tensor format of a single position prediction output is [1,3], containing [x,y,z].

[0170] The six-degree-of-freedom posture prediction output is:

[0171] The attitude output is a three-dimensional vector representing the roll angle, pitch angle, and yaw angle of the unmanned ship, defined as [roll, pitch, yaw].

[0172] The data format is a two-dimensional tensor defined as [batch_size,3], and each sample contains three angle parameters.

[0173] The data type is float32, and the output angle value is usually expressed in radians to be consistent with the sensor data.

[0174] For example, the tensor format of a single pose prediction output is [1,3], containing [roll, pitch, yaw].

[0175] For example, the model's input data consists of [visual data (224, 224, 3), IMU data (10, 6), and historical data (50, 5)], with the total input format being [1, 224, 224, 3], [1, 10, 6], and [1, 50, 5]. The model's output data format is position output [1, 3] and attitude output [1, 3].

[0176] In this embodiment, the six-degree-of-freedom prediction model also includes model pruning, which uses structured pruning to reduce model parameters by removing convolution kernels or neurons. The main steps are as follows:

[0177] Initial model training: First train the model on the complete dataset until convergence to obtain a baseline model.

[0178] Evaluate weight importance: Calculate the L1 norm of each convolution kernel. Assuming the weight matrix is W, its L1 norm is:

[0179]

[0180] Convolution kernels with weight norms less than the threshold are considered unimportant and can be removed.

[0181] Gradual pruning: Remove unimportant convolution kernels layer by layer according to a set pruning rate (such as 30%). For example, suppose a convolution layer initially has 64 convolution kernels, and after pruning, only 45 convolution kernels are retained.

[0182] Post-pruning fine-tuning: After pruning, the model is re-fine-tuned to restore accuracy.

[0183] Effect and parameter settings:

[0184] When the pruning rate is set to 30%, the convolution kernel parameters are reduced by about 30% and the memory usage is saved by about 40%.

[0185] The fine-tuned model can maintain close prediction performance with accuracy loss within 1%.

[0186] The present invention's six-degree-of-freedom prediction method for unmanned vessels based on visual analysis fuses visual data with inertial measurement unit (IMU) data and historical navigation data to more accurately estimate the six degrees of freedom (position, velocity, acceleration, pitch angle, roll angle, and yaw angle) of the unmanned vessel. This provides precise attitude and motion state prediction for unmanned vessels navigating in complex waters, effectively improving navigation accuracy and significantly enhancing navigation safety.

[0187] The method of the present invention can cope with complex and dynamic water environments, and realize dynamic prediction of six degrees of freedom through real-time visual analysis and data fusion under different sea conditions, weather, waterway conditions, etc. In particular, when encountering interference from external factors such as water surface fluctuations and wind speed changes, the model based on deep learning can adaptively adjust to maintain the accuracy and stability of the prediction, and ensure the robustness of the unmanned ship in an uncertain environment. By acquiring and processing visual data, IMU data and historical navigation data in real time, the method of the present invention can realize autonomous navigation and decision support of unmanned ships in different environments. Especially in remote or signal-free areas, unmanned ships can independently complete navigation path planning and obstacle avoidance, reduce dependence on manual operation and external navigation systems, and significantly improve the autonomy of unmanned ships.

[0188] The method of the present invention maximizes the complementarity of various sensor data through multimodal data fusion, combining visual perception and inertial navigation information. Visual data provides rich environmental information, while IMU data supplements the motion information of the hull, and historical navigation data further provides a time series pattern of the navigation status. The combination of the three significantly improves the accuracy of the six-degree-of-freedom prediction. By accurately predicting the six-degree-of-freedom state of the unmanned ship, the method of the present invention can provide accurate attitude and motion feedback for the control system of the unmanned ship, optimize the control strategy, and improve the control accuracy. Real-time six-degree-of-freedom prediction can provide data support for path planning, heading adjustment, speed control, etc., to ensure that the unmanned ship travels efficiently according to the predetermined target.

[0189] The method of this invention is particularly suitable for use in complex water environments, such as those with narrow waterways, large surface fluctuations, or dense obstacles. Visual analysis can identify factors such as obstacles on the water surface and changing current direction, while the deep learning model can adaptively adjust the ship's motion prediction based on environmental changes, helping unmanned vessels better cope with complex marine or inland water environments.

[0190] It should be noted that the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of the present invention and are not intended to be limiting. It is readily understood that the processes illustrated in the above figures do not indicate or limit the temporal order of these processes. Furthermore, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0191] It should be understood that, although the above is described in a certain order, these steps are not necessarily performed in sequence according to the above order. Unless clearly stated herein, the execution of these steps does not have strict order restrictions, and these steps can be performed in other orders. Moreover, a part of the steps of the present embodiment may include multiple steps or multiple stages, and these steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of the steps or stages in other steps or other steps.

[0192] The second aspect of the embodiment of the present invention is Figure 2 As shown, the present invention also provides a six-degree-of-freedom prediction system for an unmanned ship based on visual analysis, comprising:

[0193] Data acquisition module 100: used to collect visual data, inertial measurement unit (IMU) data and historical navigation data of the unmanned ship to obtain multimodal data of the unmanned ship;

[0194] A model building module 200 is used to fuse and preprocess the acquired multimodal data, extract features from the visual data, perform time series analysis on the IMU data and historical navigation data, predict the position and attitude of the unmanned vessel using a multi-task learning framework, and build a six-degree-of-freedom prediction model for the unmanned vessel;

[0195] The model optimization module 300 is used to adjust the parameters and data fusion strategy of the six-degree-of-freedom prediction model according to real-time environmental data, and to correct the prediction results through an adaptive real-time correction mechanism to optimize the navigation accuracy and path planning accuracy of the unmanned vessel;

[0196] The integrated deployment module 400 is used to integrate multimodal data processing, six-degree-of-freedom prediction model and adaptive real-time correction mechanism into the unmanned ship control system to predict the six-degree-of-freedom state of the unmanned ship and perform precise navigation control.

[0197] Through the above detailed steps, the unmanned ship six-degree-of-freedom prediction system based on visual analysis of the present invention is used to execute the steps of the unmanned ship six-degree-of-freedom prediction method based on visual analysis in the above embodiment, which will not be repeated here.

[0198] The unmanned ship six-degree-of-freedom prediction system based on visual analysis of the present invention not only improves the navigation accuracy and safety of unmanned ships in complex environments through multimodal data fusion and advanced deep learning technology, but also promotes the development of unmanned ship technology towards a more intelligent, autonomous and reliable direction. It has significant application value and broad market prospects.

[0199] Examples of application scenarios of the unmanned ship six-degree-of-freedom prediction method and system based on visual analysis of the present invention are as follows:

[0200] Application scenario 1: Inspection tasks in marine aquaculture areas

[0201] Scenario Description: Unmanned vessels are used for automated inspections in marine aquaculture areas. While the waters in these areas are relatively calm, the densely populated aquaculture facilities create complex and densely populated obstacles. The unmanned vessel must accurately predict its position and attitude to achieve stable navigation and obstacle avoidance.

[0202] 6DOF prediction effect: The 6DOF model based on multimodal fusion controls the errors of position and posture within 0.2 meters and 2 degrees during the inspection process, ensuring high-precision navigation of the unmanned vessel in densely populated areas and avoiding collisions.

[0203] Experimental results: In actual missions, after using the 6DOF model, the inspection time of the unmanned boat in the aquaculture area was reduced by 15%, and the probability of obstacle collision was reduced to less than 0.1%.

[0204] Application scenario 2: garbage cleaning in inland waters

[0205] Scenario Description: Unmanned boats are used for garbage removal in inland waterways. Currents in these waters vary greatly and are affected by wind speed and obstacles (such as bridge piers and ships). Unmanned boats must adjust their attitude and speed in real time to maintain stable operation and remove floating debris.

[0206] 6DOF prediction effect: The real-time attitude error predicted by the model is kept within 1.5 degrees, and the position deviation is within 0.1 meter, ensuring the cleaning accuracy and stability of the unmanned boat in unstable water flow.

[0207] Simulation data: Simulation results in this environment show that after applying the 6DOF prediction model, the cleaning path deviation is reduced to within 0.05 meters, and the cleaning efficiency is improved by 20%.

[0208] Experimental results or simulation data are being supplemented. The experimental setup involved conducting unmanned vessel navigation tests in three environments: inland rivers, marine aquaculture areas, and open waters. Test metrics included position error, attitude error, path deviation, and cleaning / inspection efficiency.

[0209] Experimental data:

[0210] Position prediction error: In inland waters, the average position error of the 6DOF model is 0.15 meters; in marine aquaculture areas, it is 0.18 meters; and in open waters, it is 0.12 meters.

[0211] Pose prediction error: Under different environments, the average pose error of the 6DOF model is 1.2 degrees, and the maximum error is 2.5 degrees.

[0212] Improved efficiency: Compared with traditional single-sensor models, the multimodal 6DOF model improves inspection efficiency by approximately 15%-20% and cleaning efficiency by 18%-22%.

[0213] The 6DOF model's performance in real-world applications demonstrates its exceptional adaptability in complex environments. For example, in high-wave conditions, the unmanned vessel, thanks to its adaptive correction strategy, is able to smoothly adjust its attitude and maintain its course. Test results demonstrate that the 6DOF model effectively reduces yaw caused by sudden changes in currents and wind speeds, enabling the unmanned vessel to achieve a mission completion rate exceeding 98% in complex environments.

[0214] According to a third aspect of an embodiment of the present invention, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the method of any one of the above embodiments is implemented.

[0215] The computer device includes a processor and a memory, and may also include an input system and an output system. The processor, memory, input system, and output system may be connected via a bus or other means. The input system may receive digital or character input and generate signal input related to the six-degree-of-freedom predicted migration of the unmanned vessel based on visual analysis. The output system may include a display device such as a display screen.

[0216] As a non-volatile computer-readable storage medium, the memory can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as the program instructions / modules corresponding to the unmanned ship six-degree-of-freedom prediction method based on visual analysis in the embodiment of the present application. The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created by the use of the unmanned ship six-degree-of-freedom prediction method based on visual analysis, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory may optionally include a memory remotely located relative to the processor, and these remote memories may be connected to the local module via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0217] In some embodiments, the processor may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor is generally used to control the overall operation of the computer device. In this embodiment, the processor is used to run the program code stored in the memory or process data. The processors of the multiple computer devices of the computer device of this embodiment execute various functional applications and data processing of the server by running the non-volatile software programs, instructions and modules stored in the memory, that is, implementing the steps of the six-degree-of-freedom prediction method of the unmanned ship based on visual analysis of the above-mentioned method embodiment.

[0218] It should be understood that, to the extent that they do not conflict with each other, all the embodiments, features and advantages described above for the unmanned ship six-degree-of-freedom prediction method based on visual analysis according to the present invention are also applicable to the unmanned ship six-degree-of-freedom prediction and storage medium based on visual analysis according to the present invention.

[0219] It will also be appreciated by those skilled in the art that the various exemplary logic blocks, modules, circuits and algorithmic steps described in conjunction with the disclosure herein can be implemented as electronic hardware, computer software or a combination of the two. In order to clearly illustrate this interchangeability of hardware and software, a general description has been given of the functions of various schematic components, blocks, modules, circuits and steps. Whether this function is implemented as software or hardware depends on specific applications and the design constraints imposed on the entire system. Those skilled in the art can implement the function in various ways for each specific application, but this implementation decision should not be interpreted as causing a departure from the disclosed scope of the embodiments of the present invention.

[0220] Finally, it should be noted that the computer-readable storage medium (e.g., memory) herein may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. By way of example and not limitation, non-volatile memory may include a read-only memory (ROM), a programmable ROM (PROM), an electrically programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), or a flash memory. Volatile memory may include a random access memory (RAM), which may act as an external cache memory. By way of example and not limitation, RAM may be obtained in a variety of forms, such as synchronous RAM (DRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct Rambus RAM (DRRAM). The storage devices of the disclosed aspects are intended to include, but are not limited to, these and other suitable types of memory.

[0221] The various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure herein may be implemented or executed using a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination of these components. A general-purpose processor may be a microprocessor, but alternatively, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, for example, a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP, and / or any other such configuration.

[0222] The above are exemplary embodiments disclosed in the present invention, but it should be noted that various changes and modifications may be made without departing from the scope of the embodiments disclosed in the claims. The functions, steps and / or actions of the method claims according to the disclosed embodiments described herein do not need to be performed in any particular order. In addition, although the elements disclosed in the embodiments of the present invention may be described or required in individual form, they may also be understood as multiple unless expressly limited to the singular.

[0223] It should be understood that, as used herein, the singular form "a" or "an" is intended to include the plural form as well, unless the context clearly supports an exception. It should also be understood that, as used herein, "and / or" refers to any and all possible combinations of one or more of the items listed in association. The serial numbers of the embodiments disclosed in the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0224] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the disclosure of the embodiments of the present invention (including the claims) is limited to these examples. Within the spirit of the embodiments of the present invention, the technical features of the above embodiments or different embodiments may be combined, and there are many other variations of different aspects of the above embodiments of the present invention, which are not provided in detail for the sake of simplicity. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present invention should be included in the scope of protection of the embodiments of the present invention.

Claims

1. A six-degree-of-freedom prediction method for unmanned ships based on visual analysis, characterized in that: The method comprises the following steps: Collect visual data, inertial measurement unit data, and historical navigation data of the unmanned vessel to obtain multimodal data of the unmanned vessel, wherein the visual data includes images or point cloud data acquired by a camera, lidar, or visual sensor; the inertial measurement unit data includes acceleration, angular velocity, and direction data of the unmanned vessel; and the historical navigation data includes navigation track, speed, heading, wind speed, and wave height information of the unmanned vessel; The acquired multimodal data is fused and preprocessed, and features are extracted from the visual data. Time series analysis is performed on the inertial measurement unit data and historical navigation data. Position and attitude prediction are performed using a multi-task learning framework, and a six-degree-of-freedom prediction model for the unmanned vessel is constructed. Adjust the parameters and data fusion strategy of the six-degree-of-freedom prediction model based on real-time environmental data, and correct the prediction results through an adaptive real-time correction mechanism to optimize the navigation accuracy and path planning accuracy of the unmanned vessel; Integrate multimodal data processing, six-degree-of-freedom prediction model and adaptive real-time correction mechanism into the unmanned vessel control system to predict the six-degree-of-freedom state of the unmanned vessel and perform precise navigation control; Based on the real-time collected environmental data, the current environmental status is classified and the weight of the inertial measurement unit data is increased according to the preset rules to maintain attitude stability. When the light intensity is lower than the threshold L min When , reduce the weight of visual data α v , the calculation formula is: When the wave height exceeds the threshold H max When α is increased, the weight of the inertial measurement unit data is increased. i : a i =a i +β·(HH max ) Where L is the light intensity, and β is the adjustment coefficient corresponding to the wave height, which is determined based on model experiments; When the wind speed exceeds the set value W max When , reduce the historical data weight α h , enhancing the impact of real-time data on predictions:

2. The six-degree-of-freedom prediction method for an unmanned ship based on visual analysis according to claim 1, characterized in that: A convolutional neural network is used to extract features from the visual data to identify and capture key features of the environment, including obstacles, tracks, and water surface information. A long short-term memory network is combined to perform time series analysis on the inertial measurement unit data and historical navigation data to capture the dynamic changes of the unmanned ship and trends in its future state.

3. The six-degree-of-freedom prediction method for an unmanned ship based on visual analysis according to claim 2, characterized in that: When fusing and preprocessing the acquired multimodal data, the visual features, inertial measurement unit features and historical data features are spliced to generate a multimodal feature vector. Different weights are assigned to each data source through weighted fusion. The weights are dynamically adjusted according to the real-time environmental data. The multimodal fusion feature vector F fuse The calculation method is as follows, using the weight α v , α i and α h Weighted superposition visual feature F vis , Inertial Measurement Unit Characteristics F imu and historical characteristics F hist : F fuse =a v ·F vis +a i ·F imu +a h ·F hist Among them, α v , α i and α h Represent the weights of visual data, inertial measurement unit data, and historical navigation data respectively.

4. The six-degree-of-freedom prediction method for an unmanned ship based on visual analysis according to claim 1, characterized in that: The multi-task learning framework is also used to split the 6DOF prediction task into two parts: position prediction and attitude prediction. It predicts the 6DOF state of the UAV in real time and outputs the estimated values of position and attitude respectively. This includes: Position prediction: Estimate the current position of the unmanned vessel in the water using the x, y, and z coordinates output by the model; Attitude prediction: Estimate the attitude state of the unmanned vessel through the roll angle, pitch angle, and yaw angle output by the model.

5. The six-degree-of-freedom prediction method for an unmanned ship based on visual analysis according to claim 4, characterized in that: The six-degree-of-freedom prediction method for unmanned ships based on visual analysis also includes synchronously marking the six-degree-of-freedom tags of each data through GPS and attitude sensors, including position and attitude. Each data records a set of position and attitude parameters. Among them, obstacles in the visual data are marked with the position, shape and size information of the obstacles to identify surface obstacles; historical navigation data are segmented by trajectory and marked with path labels to identify the target position, path curvature and navigation difficulty of each trajectory.

6. The six-degree-of-freedom prediction method for an unmanned ship based on visual analysis according to claim 5, characterized in that: The six-degree-of-freedom prediction model of the unmanned ship is constructed. In the six-degree-of-freedom prediction stage, when predicting position and attitude, the six-degree-of-freedom prediction model outputs position and attitude, namely [x, y, z, roll, pitch, yaw]. The multi-task loss function is used, and the position prediction uses the mean square error (MSE) loss L position , pose prediction uses cosine similarity loss L attitude ,in: Total loss function L total for: L total =λ1·L position +λ2·L attitude Among them, λ1 and λ2 are loss weights.

7. A six-degree-of-freedom prediction system for unmanned ships based on visual analysis, characterized in that: The system is used to execute the six-degree-of-freedom prediction method of an unmanned ship based on visual analysis according to any one of claims 1 to 6, comprising: Data acquisition module: used to collect the unmanned ship's visual data, inertial measurement unit data, and historical navigation data to obtain the unmanned ship's multimodal data; A model building module is used to fuse and preprocess the acquired multimodal data, extract features from the visual data, perform time series analysis on the inertial measurement unit data and historical navigation data, predict the position and attitude of the unmanned vessel using a multi-task learning framework, and build a six-degree-of-freedom prediction model for the unmanned vessel; The model optimization module is used to adjust the parameters and data fusion strategy of the six-degree-of-freedom prediction model according to real-time environmental data, and correct the prediction results through an adaptive real-time correction mechanism to optimize the navigation accuracy and path planning accuracy of the unmanned ship; The integrated deployment module is used to integrate multimodal data processing, six-degree-of-freedom prediction model and adaptive real-time correction mechanism into the unmanned ship control system to predict the six-degree-of-freedom state of the unmanned ship and perform precise navigation control.

Images