Positioning method and system based on multi-modal data fusion and ensemble learning

Through multimodal data fusion and integrated learning methods, the data of acoustic Doppler speed measurement, strap-inert inertial measurement and geomagnetic navigation units are used, combined with Transformer neural network, and the impact of position error accumulation and environmental noise in underwater positioning technology is solved. Problem, improving positioning accuracy and safety.

CN118960719APending Publication Date: 2024-11-15HUNAN UNIV +1
View PDF 0 Cites 7 Cited by

Patent Information

Application Number
CN202411063429.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2024-06-13
Filing Date
2024-08-05
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

When using an inertial navigation system, existing underwater positioning technology has problems such as accumulation of position errors, environmental noise affecting positioning accuracy, acoustic signal attenuation and multipath effect leading to signal weakening and interference, affecting positioning accuracy.

Method used

Multimodal data fusion and integrated learning methods are adopted to obtain multimodal data through acoustic Doppler speed measurement unit, strap-inert inertial measurement unit and geomagnetic navigation unit, perform mixed coding and feature extraction, and use a deep neural network based on Transformer for Training to improve positioning accuracy.

Benefits of technology

It improves the accuracy of positioning, reduces the possibility of misaligned positioning, ensures the safety of underwater submarine driving, and enhances data utilization through multimodal data normalization encoding and feature learning.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118960719A_ABST
    Figure CN118960719A_ABST
Patent Text Reader

Abstract

The invention discloses a positioning method and system based on multi-modal data fusion and ensemble learning. The method comprises the following steps: acquiring speed information and motion track information of an underwater vehicle through an acoustic Doppler speed measurement unit; obtaining three-axis linear acceleration data, three-axis angular velocity data and attitude information of the underwater vehicle through a strapdown inertial measurement unit; position information of the underwater vehicle is obtained through the geomagnetic navigation unit; performing hybrid coding on the three types of multi-modal data obtained by the Doppler velocity measurement unit, the strapdown inertial measurement unit and the geomagnetic navigation unit to obtain fusion feature data; inputting the fused feature data into a deep neural network based on Transform for training; and in the underwater positioning navigation, the trained deep neural network based on the Transform is used for positioning the underwater vehicle. According to the invention, feature learning is carried out through multi-modal fusion data, the positioning result is output by using the neural network, and the underwater positioning precision is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of underwater positioning technology, and in particular to a positioning method and system based on multimodal data fusion and integrated learning. Background Art

[0002] With the continuous advancement of marine strategy, underwater positioning technology will become an important means to ensure national security, promote marine economic development, protect the marine environment and support scientific research, with broad application prospects and important significance. China's continuous investment and development in underwater positioning technology can not only enhance the country's comprehensive marine strength, but also make important contributions to the global marine cause.

[0003] At present, although underwater positioning technology has made significant progress, there are still some defects and challenges in practical applications. For example, when using an inertial navigation system, sensor noise and drift will cause position errors to accumulate over time, requiring frequent calibration; changes in environmental noise, temperature, and water density will affect the speed of sound, thereby affecting positioning accuracy; when propagating underwater, acoustic signals will be affected by attenuation and multipath effects, resulting in signal weakening and interference; these problems will affect the accuracy of underwater positioning. Summary of the invention

[0004] The object of the present invention is to provide a positioning method and system based on multimodal data fusion and integrated learning, so as to at least partially solve the above-mentioned problems of the prior art.

[0005] To achieve the above object, one aspect of the present invention provides a positioning method based on multimodal data fusion and integrated learning, which includes:

[0006] The speed information and motion trajectory information of the underwater vehicle are obtained through an acoustic Doppler velocity measurement unit;

[0007] The three-axis linear acceleration data, three-axis angular velocity data and attitude information of the underwater submersible are obtained through the strapdown inertial measurement unit;

[0008] Obtaining the position information of the underwater submersible through the geomagnetic navigation unit;

[0009] The three types of multimodal data obtained by the Doppler velocity measurement unit, the strapdown inertial measurement unit and the geomagnetic navigation unit are mixed and encoded to obtain fused feature data;

[0010] Input the fused feature data into the Transformer-based deep neural network for training;

[0011] In underwater positioning and navigation, a trained Transformer-based deep neural network is used to locate the underwater submersible.

[0012] Preferably, three types of multimodal data obtained by the Doppler velocity measurement unit, the strapdown inertial measurement unit and the geomagnetic navigation unit are mixed and encoded to obtain fused feature data, including:

[0013] The mean and standard deviation of the three types of multimodal data obtained by the Doppler velocity measurement unit, the strapdown inertial measurement unit and the geomagnetic navigation unit were calculated and normalized;

[0014] Different multi-layer perceptrons (MLPs) are built for the three types of multimodal data. The MLPs include a linear layer consisting of a randomly initialized weight matrix, a ReLU activation function, and a Dropout layer.

[0015] The normalized data are input into the corresponding MLP to abstract the hidden features;

[0016] The three hidden features are concatenated to obtain fused feature data.

[0017] Preferably, the three hidden features are concatenated to obtain fused feature data, including:

[0018] Residual connections are added to the three hidden features respectively to obtain three eigenvalues, and the three eigenvalues ​​are fused to obtain fused feature data.

[0019] Preferably, three types of multimodal data obtained by the Doppler velocity measurement unit, the strapdown inertial measurement unit and the geomagnetic navigation unit are mixed and encoded to obtain fused feature data, including:

[0020] The three types of multimodal data are x1, x2 and x3, with means μ1, μ2 and μ3, and standard deviations σ1, σ2 and σ3, respectively. The normalized calculation is:

[0021]

[0022] Different multi-layer perceptrons MLP1, MLP2 and MLP3 are established for the three types of multimodal data, where each MLP includes a linear layer consisting of a randomly initialized weight matrix, a ReLU activation function and a Dropout layer;

[0023] The normalized data are input into the corresponding multi-layer perceptron MLP to abstract the hidden features.

[0024]

[0025] Adding residual connection, we get:

[0026]

[0027] Concatenate y1, y2, and y3 to obtain the fused feature data y as the output of the encoding layer.

[0028] Preferably, inputting the fused feature data into a Transformer-based deep neural network for training includes:

[0029] Linear mapping of the fused feature data y:

[0030] q=W Q y,k=W K y,v=W V y

[0031] Among them, W Q , W K , W V is a randomly initialized weight matrix;

[0032] Perform self-attention calculation:

[0033]

[0034] Among them, (k) Τ represents the transpose of matrix k, dk represents the dimension of the model hidden layer;

[0035] The self-attention calculation result is input into the feedforward neural network, which consists of a linear layer, Dropout, and Relu activation function, expressed as:

[0036] c`=Linear(Dropout(Relu(c)))

[0037] Among them, Linear represents the randomly initialized weight matrix, Dropout represents the neuron abandonment rate, and the Relu activation function is expressed as:

[0038]

[0039] Preferably, inputting the fused feature data into a Transformer-based deep neural network for training further includes:

[0040] Input c` into the linear decoder to decode:

[0041] r = MLPDecoder(c`)

[0042] Among them, MLPDecoder is the same as MLP, including a linear layer composed of a randomly initialized weight matrix, a ReLU activation function, and a Dropout layer; r is the positioning data of the underwater submersible obtained by decoding;

[0043] The root mean square error RMSE of the output r and the sample label value r` is calculated as the loss function L. The gradient descent of L is iterated until the function reaches the preset convergence condition, and the parameter setting in the Transformer-based deep neural network is obtained to complete the training.

[0044] Preferably, iterating the gradient descent on L includes: calculating the gradient descent using the Adam optimization algorithm.

[0045] Another aspect of the present invention provides a positioning system based on multimodal data fusion and integrated learning, which is used to execute the above-mentioned positioning method based on multimodal data fusion and integrated learning, and the system comprises:

[0046] Acoustic Doppler speed measurement unit, used to obtain speed information and motion trajectory information of underwater submersibles;

[0047] Strapdown inertial measurement unit, used to obtain three-axis linear acceleration data, three-axis angular velocity data and attitude information of the underwater submersible;

[0048] A geomagnetic navigation unit, used to obtain the position information of the underwater submersible;

[0049] A multi-modal feature fusion encoding unit is used to perform mixed encoding on three types of multi-modal data obtained by a Doppler velocity measurement unit, a strapdown inertial measurement unit and a geomagnetic navigation unit to obtain fused feature data;

[0050] The underwater positioning unit is used to input the fused feature data into the Transformer-based deep neural network for training, and use the trained Transformer-based deep neural network to locate the underwater submersible.

[0051] The present invention has the following advantages due to the adoption of the above technical solution:

[0052] The use of a multimodal data normalization encoding method improves data utilization; the use of a Transformer-based neural network for feature learning increases positioning accuracy, reduces the possibility of misaligned positioning, and ensures the safety of underwater submersibles. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 It is a flow chart of a positioning method based on multimodal data fusion and integrated learning provided by an embodiment of the present invention.

[0054] Figure 2 It is a structural diagram of a positioning system based on multimodal data fusion and integrated learning provided by an embodiment of the present invention.

[0055] Figure 3Schematic diagram of a four-beam Janus array structure of a Doppler velocimeter according to an embodiment of the present invention.

[0056] Figure 4 It is a schematic diagram of the working principle of the strapdown inertial navigation system according to an embodiment of the present invention.

[0057] Figure 5 It is a schematic diagram of the working principle of measuring underwater geomagnetic field data in real time by a magnetometer according to an embodiment of the present invention.

[0058] Figure 6 It is a schematic diagram of the structure of a Transformer-based deep neural network according to an embodiment of the present invention. DETAILED DESCRIPTION

[0059] In the drawings, the same or similar reference numerals are used to represent the same or similar elements or elements with the same or similar functions. The embodiments of the present invention are described in detail below in conjunction with the drawings.

[0060] In the description of the present invention, the terms "center", "longitudinal", "lateral", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate directions or positional relationships based on the directions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore should not be understood as limiting the scope of protection of the present invention.

[0061] At present, as a strapdown inertial navigation system for underwater positioning, an inertial measurement unit is used, which consists of three accelerometers and three gyroscopes, respectively measuring the linear acceleration and angular velocity of the carrier on three orthogonal axes. The attitude solution calculates the attitude change of the carrier through the angular velocity measured by the gyroscope. The attitude can be expressed by Euler angles, direction cosine matrix (DCM) or quaternion.

[0062] The following is the attitude solution process represented by quaternion: 1. Angular velocity integration: convert the angular velocity measured by the gyroscope into the angular velocity increment represented by the quaternion, and obtain the quaternion increment by integration; 2. Attitude update: multiply the initial attitude quaternion by the angular velocity increment quaternion to obtain the updated attitude quaternion; 3. Normalization: in order to avoid numerical drift, the updated quaternion is normalized. Velocity solution calculates the velocity change of the carrier through the linear acceleration measured by the accelerometer. The acceleration measured by the accelerometer is in the carrier coordinate system and needs to be converted to the inertial coordinate system for integration. Velocity solution includes: 1. Acceleration conversion: using the attitude quaternion, the acceleration measured by the accelerometer is converted from the carrier coordinate system to the inertial coordinate system; 2. Gravity compensation: subtract the influence of gravity acceleration to obtain pure motion acceleration; 3. Acceleration integration: integrate the motion acceleration to obtain the velocity.

[0063] However, strapdown inertial navigation systems rely on gyroscopes and accelerometers to measure angular velocity and acceleration. However, these sensors inevitably have zero bias and noise. Over time, these small errors will accumulate, resulting in a gradual decrease in navigation accuracy. The calculation of attitude, velocity, and position all rely on the integration of angular velocity and acceleration. Any tiny error will be amplified during the integration process, resulting in the accumulation of position and velocity estimation errors. This accumulated error cannot be completely eliminated by simple algorithm adjustments. In order to improve navigation accuracy, strapdown inertial navigation systems are often used in combination with other navigation systems (such as GPS, magnetometers, sonar, etc.). This requires complex data fusion algorithms such as Kalman filtering and particle filtering, which increases the complexity of the system.

[0064] The underwater geomagnetic navigation system is also an underwater positioning device. Its working principle can be divided into the following steps: geomagnetic field measurement, geomagnetic field model establishment, matching positioning and path planning. Geomagnetic field measurement is the basis of geomagnetic navigation. The geomagnetic field data underwater, including magnetic field strength and direction, are measured in real time by a magnetometer. These data need to be preprocessed, such as filtering and correction, to remove noise and interference. The geomagnetic field model is the key to navigation. The geomagnetic field model can be established based on the global geomagnetic field model (such as the World Geomagnetic Model, WMM) and local high-precision geomagnetic field measurement data. The accuracy of the model directly affects the accuracy of navigation. Matching positioning is the core step of geomagnetic navigation. The current position is determined by comparing the real-time measured geomagnetic field data with the pre-established geomagnetic field model.

[0065] Path planning is the application part of the geomagnetic navigation system. Through the geomagnetic field data and model, the optimal navigation path is planned to ensure navigation accuracy and efficiency. In practical applications, path planning also needs to be combined with other sensor data, such as sonar, inertial navigation system (INS), etc., for comprehensive planning and adjustment. However, underwater geomagnetic navigation needs to rely on a priori underwater geomagnetic field information map. However, due to the complexity of the underwater magnetic field environment, it is very difficult to obtain a high-precision prior underwater geomagnetic field information map. This is because the underwater magnetic field is easily affected by various magnetic field environments, which may come from the magnetic field changes of the earth itself, or due to magnetic field disturbances caused by artificial structures and natural phenomena in the underwater environment. The complexity of the underwater magnetic field environment makes it difficult to guarantee the accuracy of the geomagnetic navigation system in practical applications. Although geomagnetic navigation can provide higher navigation accuracy in theory, in actual operation, due to the limitations of the above factors, its application effect may not be as expected. This complexity includes but is not limited to daily changes in the geomagnetic field, seasonal changes, and local magnetic field anomalies caused by differences in geological structures.

[0066] The acoustic Doppler velocimeter is an underwater positioning device. Its basic working principles include: the transducer in the instrument emits high-frequency sound waves (usually between 1MHz and 10MHz), which propagate into the water at a certain angle; the emitted sound waves encounter suspended particles or bubbles in the water during propagation, and these particles reflect part of the sound waves; the instrument receives the reflected sound waves and measures their frequency; the frequency shift is calculated by comparing the frequency of the emitted sound wave with the frequency of the received sound wave; and the frequency shift is converted into water flow velocity according to the Doppler effect formula.

[0067] In underwater positioning tasks, the multimodal data fusion technology solution enhances positioning accuracy and reliability by integrating multiple sensor data. The main multimodal data include sonar data, inertial navigation data, underwater magnetometer data, and environmental sensor data. Including data from temperature, pressure, and salinity sensors, this information can be used to correct changes in sound velocity profiles and improve sonar ranging accuracy. In addition, environmental data can also provide supplementary positioning information, such as correction through underwater terrain matching. Among them, multimodal data fusion involves a lot of data processing and calculation, especially high-precision filtering algorithms (such as particle filtering), which require a lot of computing resources, which may become a bottleneck in real-time applications. Multiple sensors require precise calibration and time synchronization. A slight error will lead to a decrease in data fusion accuracy, increasing system complexity and maintenance difficulty.

[0068] In underwater positioning tasks, ensemble learning technology combines the prediction results of multiple base models to improve the overall accuracy and robustness of the positioning system. Ensemble learning uses the advantages of different models to make up for the shortcomings of a single model, thereby achieving better performance in complex and changing underwater environments. The first step of ensemble learning is to select and train base models. These base models can be different types of machine learning models (such as decision trees, support vector machines, neural networks) or different instances of the same model (such as neural networks with different initializations or different parameter settings). In underwater positioning, base models usually include sonar image processing models, inertial navigation data analysis models, and environmental sensor data fusion models.

[0069] The embodiment of the present invention provides a positioning method based on multimodal data fusion and integrated learning, such as Figure 1 As shown, including:

[0070] Step 101, obtaining speed information and motion trajectory information of an underwater vehicle through an acoustic Doppler speed measurement unit.

[0071] Step 102, obtaining three-axis linear acceleration data, three-axis angular velocity data and attitude information of the underwater submersible through a strapdown inertial measurement unit.

[0072] Step 103, obtaining the position information of the underwater submersible through the geomagnetic navigation unit.

[0073] Step 104 , performing mixed encoding on the three types of multimodal data obtained by the Doppler velocity measurement unit, the strapdown inertial measurement unit and the geomagnetic navigation unit to obtain fused feature data.

[0074] Among them, this step may include:

[0075] The mean and standard deviation of the three types of multimodal data obtained by the Doppler velocity measurement unit, the strapdown inertial measurement unit and the geomagnetic navigation unit were calculated and normalized;

[0076] Different multi-layer perceptrons (MLPs) are built for the three types of multimodal data. The MLPs include a linear layer consisting of a randomly initialized weight matrix, a ReLU activation function, and a Dropout layer.

[0077] The normalized data are input into the corresponding MLP to abstract the hidden features;

[0078] The three hidden features are concatenated to obtain fused feature data.

[0079] The three hidden features are concatenated to obtain fused feature data, which may include:

[0080] Residual connections are added to the three hidden features respectively to obtain three eigenvalues, and the three eigenvalues ​​are fused to obtain fused feature data.

[0081] Step 105: input the fused feature data into a Transformer-based deep neural network for training.

[0082] Step 106, in underwater positioning and navigation, use the trained Transformer-based deep neural network to locate the underwater submersible.

[0083] In a preferred embodiment, three types of multimodal data obtained by the Doppler velocity measurement unit, the strapdown inertial measurement unit and the geomagnetic navigation unit are mixed and encoded to obtain fused feature data, including:

[0084] The three types of multimodal data are x1, x2 and x3, with means μ1, μ2 and μ3, and standard deviations σ1, σ2 and σ3, respectively. The normalized calculation is:

[0085]

[0086] Different multi-layer perceptrons MLP1, MLP2 and MLP3 are established for the three types of multimodal data, where each MLP includes a linear layer consisting of a randomly initialized weight matrix, a ReLU activation function and a Dropout layer;

[0087] The normalized data are input into the corresponding multi-layer perceptron MLP to abstract the hidden features.

[0088]

[0089] Adding residual connection, we get:

[0090]

[0091] Concatenate y1, y2, and y3 to obtain the fused feature data y as the output of the encoding layer.

[0092] Among them, inputting the fused feature data into the Transformer-based deep neural network for training includes:

[0093] Linear mapping of the fused feature data y:

[0094] q=W Q y,k=W K y,v=W V y

[0095] Among them, W Q , W K , W V is a randomly initialized weight matrix;

[0096] Perform self-attention calculation:

[0097]

[0098] Among them, (k) Τ represents the transpose of matrix k, dk represents the dimension of the model hidden layer, and in the preferred implementation, d k Set to 64. In other implementations, d k It can also be set to other possible values ​​such as 32, 60, etc.

[0099] The self-attention calculation result is input into the feedforward neural network, which consists of a linear layer, Dropout, and Relu activation function, expressed as:

[0100] c`=Linear(Dropout(Relu(c)))

[0101] Among them, Linear represents the randomly initialized weight matrix, Dropout represents the neuron abandonment rate, and the Relu activation function is expressed as:

[0102]

[0103] Among them, inputting the fused feature data into the Transformer-based deep neural network for training also includes:

[0104] Input c` into the linear decoder to decode:

[0105] r = MLPDecoder(c`)

[0106] Among them, MLPDecoder is the same as MLP, including a linear layer composed of a randomly initialized weight matrix, a ReLU activation function, and a Dropout layer; r is the positioning data of the underwater submersible obtained by decoding;

[0107] The root mean square error RMSE of the output r and the sample label value r` is calculated as the loss function L. The gradient descent of L is iterated until the function reaches the preset convergence condition, and the parameter setting in the Transformer-based deep neural network is obtained to complete the training.

[0108] The iteration of performing gradient descent on L may include: calculating the gradient descent using the Adam optimization algorithm.

[0109] An embodiment of the present invention further provides a positioning system based on multimodal data fusion and integrated learning, which is used to implement the positioning method based on multimodal data fusion and integrated learning described in the above embodiment and its various implementation methods.

[0110] Figure 2FIG. 2 is a schematic diagram showing the structure of a positioning system based on multimodal data fusion and integrated learning provided by an embodiment of the present invention. Figure 2 As shown, the system includes an acoustic Doppler velocity measurement unit 10, a strapdown inertial measurement unit 20, a geomagnetic navigation unit 30, and an integrated learning unit 40.

[0111] (I) Acoustic Doppler velocity measurement unit 10

[0112] The acoustic Doppler velocity measurement unit includes an acoustic Doppler velocimeter (DVL). The acoustic Doppler velocimeter measures the relative velocity of the water body relative to the device by emitting sound waves and receiving reflected waves, using the Doppler effect, and then calculates the position in combination with the inertial navigation system.

[0113] The Doppler velocity measurement formula is:

[0114]

[0115] The frequency f received at position G t , the frequency f received at position G′ r , where f d =f r -f t is the Doppler shift, and the speed of sound in water c is about 1500m / s.

[0116] Figure 3 FIG. 1 is a schematic diagram of a four-beam Janus array structure of a Doppler velocimeter according to an embodiment of the present invention. Figure 3 As shown, a DVL is set in an underwater submersible (also known as an underwater navigation device such as an underwater vehicle, an underwater robot, etc.), and the DVL adopts a four-beam Janus formation structure. The DVL with this transducer formation structure can simultaneously transmit and receive four narrow beam sound waves, and then the radial flow velocity information relative to the acoustic transducer can be obtained after processing. The velocity measurement system coordinate system D of the DVL is defined, and the origin of D is located at the center point O of the Janus formation structure, and its three axes point to the front, right and bottom directions specified by the flow measurement system respectively. The four-beam Janus formation structure of the DVL is composed of four transducers, and the angle between the axis of each transducer for transmitting and receiving sound waves and the z-axis of the flow measurement system coordinate system is α.

[0117] In addition, the projection of the axis of each transducer receiving and transmitting sound waves on the xoy plane of the array coordinate system forms two mutually perpendicular straight lines, and the angle between the projection of beam 1 and the x-axis is β. Usually, when the flow measurement system is installed in an "X" form, β is equal to 45°, and when it is installed in a "cross" form, β is equal to 0°. When the DVL equipped with a four-beam Janus array structure is working, the narrow beam sound waves emitted simultaneously by the transducers propagate along the axis direction of each transducer. During the propagation process, each sound wave is scattered by the water body, and a scattered echo will be received by each transducer. The radial movement between the DVL transducer and the water body causes a Doppler frequency shift between the frequencies of the transmitted and received sound waves. By measuring the Doppler frequency shift, the radial flow velocities v1, v2, v3, and v4 can be calculated.

[0118] Assume that the velocity vector of the water bottom in the velocity measurement system coordinate system D is:

[0119] V d =[v dx ,v dy ,v dz ] T

[0120] Assuming that the water bottom measured by DVL is uniform and horizontal, the velocity measured by any three sound waves of DVL equipped with a four-beam Janus array structure can be converted into the velocity vector V in the velocity measurement system coordinate system D. d , the system can still work even if a beam cannot be measured normally due to interference, thus improving the reliability of measurement.

[0121] The velocity column vector V measured by the 1st, 2nd and 3rd beams b =[v1,v2,v3] T For example, we get V b With V d The relationship is:

[0122] V b =AV d

[0123] V d =(A T A) -1 A T V d

[0124] in:

[0125]

[0126]

[0127] In fact, the measured bottom is rarely completely uniform, that is, the velocity magnitude and direction of the DVL relative to the bottom at the same depth are often different. The unevenness of the bottom will introduce different degrees of inherent velocity errors. The four-beam Janus formation structure can obtain more redundant information from the fourth beam than the three-beam Convex formation structure, so the uniformity of the bottom can be verified.

[0128] In order to quantify the impact of bottom heterogeneity, the velocity inherent error formula can be introduced, which is an important factor in evaluating whether the data quality is effective. Specifically:

[0129] v error =v1-v2+v3-v4

[0130] Through analysis, it can be seen that as long as the measured water bottom is uniform, no matter how the system swings, it will basically approach zero, and the four-way echo is considered to be valid data; if the measured flow field is uneven, the degree of unevenness can be determined to judge the validity of the four-way echo data.

[0131] The four-beam Janus array structure has another advantage over the three-beam Convex array structure, which is that it can more effectively reduce the velocity measurement error introduced by the DVL array swing. This is achieved by converting the velocity measured by the four beams into the velocity vector Vd in the velocity measurement system coordinate system D. Suppose the column vector composed of the four radial velocities of the bottom is V b =[v1,v2,v3,v4] T , since the velocity column vector of the water bottom in the velocity measurement system coordinate system D is V d =[v dx ,v dy ,v dz ] T , so we get V b With V d The relationship between them is:

[0132] V b =BV d

[0133] V d =(B T B) -1 B T V d

[0134] The conversion matrices of the above two equations are:

[0135]

[0136]

[0137] According to the relationship between the above two equations, the measured radial velocity V can beb Converted to the speed V in the speed measurement system coordinate system D d .

[0138] In this embodiment, the velocity information in each direction and the continuity information of the motion trajectory are obtained by an acoustic Doppler velocimeter.

[0139] (II) Strapdown Inertial Measurement Unit 20

[0140] The core component of the strapdown inertial navigation system is the inertial measurement unit (IMU). Figure 4 FIG. 1 is a schematic diagram of the working principle of the strapdown inertial navigation system according to an embodiment of the present invention. Figure 4 As shown, the IMU is usually composed of three gyroscopes and three accelerometers, which measure the angular velocity and acceleration of the carrier respectively. In addition, the system also includes a navigation computer and related algorithms to process the data of the IMU and calculate the attitude, velocity and position of the carrier. The gyroscope measures the angular velocity of the carrier relative to the inertial space.

[0141] The working principle of the strapdown inertial navigation system can be divided into three main steps: attitude solution, velocity update and position update.

[0142] The purpose of attitude solution is to calculate the attitude of the carrier through the angular velocity measured by the gyroscope. The attitude is usually expressed by Euler angles, quaternions or direction cosine matrices (DCM). Since the gyroscope measures the angular velocity, it is necessary to obtain the angle change through integration and then calculate the current attitude in combination with the initial attitude.

[0143] For Euler angles, the process of attitude solution is: Get the angular velocity ω of the three axes x ,ω y ,ω z , the angle change Δθ is obtained by integration x ,Δθ y ,Δθ z , update the Euler angles.

[0144] For quaternions, the process of attitude solution is: obtain the angular velocity and convert it into the angular velocity represented by the quaternion; update the quaternion using the quaternion differential equation; normalize the quaternion.

[0145] The purpose of velocity update is to calculate the velocity of the carrier using the acceleration measured by the accelerometer. It should be noted that the accelerometer measures the acceleration a in the carrier coordinate system. x ,a y ,a z, so it needs to be converted to the inertial coordinate system: get the linear acceleration measured by the accelerometer; convert the linear acceleration from the carrier coordinate system to the inertial coordinate system, using the attitude matrix for conversion; subtract the gravity acceleration; get the velocity by integration. The purpose of position update is to calculate the position of the carrier through the velocity. The position can be obtained by integrating the velocity: Get the velocity v x ,v y ,v z , and the position is obtained by integration.

[0146] The strapdown inertial measurement unit uses the two inertial elements, the accelerometer and gyroscope on the carrier, to measure the angular motion information and linear motion information relative to the inertial space respectively, and under given initial conditions, the computer calculates the three-axis linear acceleration data, three-axis angular velocity data and attitude information. The apparent acceleration n measured by the accelerometer, that is, the absolute acceleration V of the carrier xg and the vector sum of gravitational acceleration g, θ is the heading angle, where n x and n y are the two components of n on the x-axis and y-axis:

[0147]

[0148] (III) Geomagnetic navigation unit 30

[0149] The geomagnetic navigation unit includes a magnetometer, which is the main sensor for measuring the geomagnetic field. It is usually composed of three mutually perpendicular magnetic sensitive elements, which measure the magnetic field components in the X, Y, and Z directions respectively. Common types of magnetometers include magnetoresistive type, fluxgate type, and superconducting quantum interference device (SQUID) type.

[0150] The strength and direction of the geomagnetic field are unevenly distributed around the world and can be drawn into a geomagnetic map. By comparing the currently measured geomagnetic data with the pre-stored geomagnetic map, the geomagnetic navigation system can infer the current position. Geomagnetic navigation uses a particle filter algorithm based on Monte Carlo random sampling suitable for environments with large interference; the Rao-Blackwellized algorithm divides the system into a nonlinear part and an approximate linear part. For the nonlinear part, a particle filter is used for filtering; and the remaining linear part is filtered using a Kalman filter; the resampling method can restore the positioning result to near the actual position of the underwater carrier.

[0151] The working principle of the underwater geomagnetic navigation system can be divided into the following steps: geomagnetic field measurement, geomagnetic field model establishment, matching positioning and path planning.

[0152] Geomagnetic field measurement is the basis of geomagnetic navigation. Figure 5 FIG. 1 is a schematic diagram of the working principle of measuring underwater geomagnetic field data in real time by a magnetometer according to an embodiment of the present invention. Figure 5 As shown, the underwater geomagnetic field data, including magnetic field strength and direction, are measured in real time by a magnetometer. These data need to be preprocessed, such as filtering and correction, to remove noise and interference. The geomagnetic field model is the key to navigation. The geomagnetic field model can be established based on the global geomagnetic field model (such as the World Geomagnetic Model, WMM) and local high-precision geomagnetic field measurement data. The accuracy of the model directly affects the accuracy of navigation. Matching positioning is based on the state estimation method of the Kalman filter linear dynamic system, which optimizes the position estimate using current and historical data through prediction and update steps. The current position is determined by comparing the real-time measured geomagnetic field data with the pre-established geomagnetic field model.

[0153] Path planning uses geomagnetic field data and models to plan the optimal navigation path to ensure navigation accuracy and efficiency; it also combines other sensor data for comprehensive planning and adjustment.

[0154] (IV) Integrated Learning Unit 40

[0155] Figure 6 Schematic diagram of the structure of a Transformer-based deep neural network according to an embodiment of the present invention. Figure 6 As shown, it includes an embedding layer, a Transformer coding layer, and a decoder.

[0156] 4.1 Embedding Layer

[0157] The embedded layer receives three types of data collected by the acoustic Doppler velocity measurement unit, the strapdown inertial measurement unit, and the geomagnetic navigation unit, that is, the data collected by the acoustic Doppler velocity measurement unit is defined as the first type of data, the data collected by the strapdown inertial measurement unit is defined as the second type of data, and the data collected by the geomagnetic navigation unit is defined as the third type of data.

[0158] Multimodal data mixed encoding is performed on three types of multimodal data.

[0159] In an example, the embedding layer calculates the mean μ1, μ2, and μ3 and the standard deviation σ1, σ2, and σ3 of the three types of multimodal data x1, x2, and x3 collected, and then performs normalization calculation:

[0160]

[0161] Then, different multilayer perceptrons MLP1, MLP2 and MLP3 are established for the three types of input data, where MLP includes a linear layer composed of a randomly initialized weight matrix, a ReLU activation function and a Dropout layer.

[0162] The normalized data are input into the corresponding multi-layer perceptron MLP to abstract the hidden features.

[0163]

[0164] Then, in order to ensure the stability of the gradient, a residual connection is added to obtain:

[0165]

[0166] is the calculated eigenvalue, is the normalized input data.

[0167] Then, y1, y2, and y3 are concatenated to obtain y, which is used as the output of the encoding layer.

[0168] 4.2 Transformer Encoding Layer

[0169] The output y of the multimodal data mixed encoding is used as the input of the Transformer encoder, and a linear mapping is first performed:

[0170] q=W Q y,k=W K y,v=W V y

[0171] Among them, W Q , W K , W V Represent different randomly initialized weight matrices:

[0172]

[0173] Where a and b are obtained by random sampling from a Gaussian distribution with a mean of 0 and a variance of 0.01; q, k, and v represent the query, key, and value in the attention mechanism, respectively;

[0174] Then do the self-attention calculation:

[0175]

[0176] Among them, (k) Τ represents the transpose of the matrix k, dk represents the dimension of the hidden layer of the preset model, for example, it can be preferably set to 64, and can also be set to other values, such as 32, 60, etc.

[0177] The self-attention calculation result is input into the following feedforward neural network, which consists of a linear layer, Dropout, and Relu activation function:

[0178] c`=Linear(Dropout(Relu(c)))

[0179] Where Linear represents the randomly initialized weight matrix, Dropout represents the neuron abandonment rate, which is set to 0.1, and the Relu activation function is expressed as:

[0180]

[0181] 4.3 Decoder

[0182] Input c` into the linear decoder to decode:

[0183] r = MLPDecoder(c`)

[0184] Among them, MLPDecoder is the same as MLP, including a linear layer composed of a randomly initialized weight matrix, a ReLU activation function and a Dropout layer; r is the decoded multimodal data of the underwater submersible.

[0185] 4.4 Accurate positioning

[0186] During the training process, it is necessary to extract sample data containing multiple types of abnormal measurements for training the neural network model; the label value in the training is the real location information; the loss function in the training is the root mean square error RMSE; the optimization algorithm used in the training is the Adam optimization algorithm.

[0187] The output r and the sample label value r' are used to calculate the RMSE, which is used as the loss function L. The gradient descent of L is iterated until the function converges, that is, the training is completed. At this time, the trained MLP, the weight matrix of the Transformer encoding layer, and the weight matrix in the decoder are obtained.

[0188] In order to speed up the convergence and avoid local optimality, the Adam optimizer is used to calculate the gradient descent; the number of iterations during training can be 50 and the sample batch size can be 10.

[0189] Use the trained network weights to calculate the accurate results of underwater positioning.

[0190] The positioning result is calculated by inputting the multimodal data into the trained neural network.

[0191] It should be noted that due to the attention mechanism used in the Transformer encoder, different weights are assigned to data of different modalities, thereby improving the accuracy of positioning.

[0192] Simply modifying the neural network structure in the present invention, or replacing it with other integrated learning algorithms, and adjusting the multimodal encoding method can achieve roughly the same effect, but all belong to the main idea of ​​the present invention.

[0193] In the present invention, since a multimodal data normalization encoding method is adopted, the utilization rate of data is improved; since a Transformer-based neural network is used for feature learning, the accuracy of positioning is increased, the possibility of misplaced positioning is reduced, and the safety of underwater submersible driving is ensured.

[0194] Finally, it should be pointed out that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them. Those skilled in the art should understand that the technical solutions described in the above embodiments may be modified, or some of the technical features thereof may be replaced by equivalents; these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A positioning method based on multimodal data fusion and ensemble learning, characterized in that: include: The speed information and motion trajectory information of the underwater vehicle are obtained through an acoustic Doppler velocity measurement unit; The three-axis linear acceleration data, three-axis angular velocity data and attitude information of the underwater submersible are obtained through the strapdown inertial measurement unit; Obtaining the position information of the underwater submersible through the geomagnetic navigation unit; The three types of multimodal data obtained by the Doppler velocity measurement unit, the strapdown inertial measurement unit and the geomagnetic navigation unit are mixed and encoded to obtain fused feature data; Input the fused feature data into the Transformer-based deep neural network for training; In underwater positioning and navigation, a trained Transformer-based deep neural network is used to locate the underwater submersible.

2. The positioning method based on multimodal data fusion and integrated learning according to claim 1, characterized in that: The three types of multimodal data obtained by the Doppler velocity measurement unit, the strapdown inertial measurement unit and the geomagnetic navigation unit are mixed and encoded to obtain fused feature data, including: The mean and standard deviation of the three types of multimodal data obtained by the Doppler velocity measurement unit, the strapdown inertial measurement unit and the geomagnetic navigation unit were calculated and normalized; Different multi-layer perceptrons (MLPs) are built for the three types of multimodal data. The MLPs include a linear layer consisting of a randomly initialized weight matrix, a ReLU activation function, and a Dropout layer. The normalized data are input into the corresponding MLP to abstract the hidden features; The three hidden features are concatenated to obtain fused feature data.

3. The positioning method based on multimodal data fusion and integrated learning according to claim 2, characterized in that: The three hidden features are concatenated to obtain fused feature data, including: Residual connections are added to the three hidden features respectively to obtain three eigenvalues, and the three eigenvalues ​​are fused to obtain fused feature data.

4. The positioning method based on multimodal data fusion and integrated learning according to claim 1, characterized in that: The three types of multimodal data obtained by the Doppler velocity measurement unit, the strapdown inertial measurement unit and the geomagnetic navigation unit are mixed and encoded to obtain fused feature data, including: The three types of multimodal data are x1, x2 and x3, with means μ1, μ2 and μ3, and standard deviations σ1, σ2 and σ3, respectively. The normalized calculation is: Different multi-layer perceptrons MLP1, MLP2 and MLP3 are established for the three types of multimodal data, where each MLP includes a linear layer consisting of a randomly initialized weight matrix, a ReLU activation function and a Dropout layer; The normalized data are input into the corresponding multi-layer perceptron MLP to abstract the hidden features. Adding residual connection, we get: Concatenate y1, y2, and y3 to obtain the fused feature data y as the output of the encoding layer.

5. The positioning method based on multimodal data fusion and integrated learning according to claim 4 is characterized in that ,Inputting the fused feature data into the Transformer-based deep neural network for training includes: Linear mapping of the fused feature data y: q=W Q y,k=W K y,v=W V y Among them, W Q , W K , W V is a randomly initialized weight matrix; Perform self-attention calculation: Among them, (k) Τ represents the transpose of matrix k, and dk represents the hidden layer dimension of the preset model; The self-attention calculation result is input into the feedforward neural network, which consists of a linear layer, Dropout, and Relu activation function, expressed as: c`=Linear(Dropout(Relu(c))) Among them, Linear represents the randomly initialized weight matrix, Dropout represents the neuron abandonment rate, and the Relu activation function is expressed as:

6. The positioning method based on multimodal data fusion and integrated learning according to claim 5 is characterized in that ,Inputting the fused feature data into the Transformer-based deep neural network for training also includes: Input c` into the linear decoder to decode: r = MLPDecoder(c`) Among them, MLPDecoder is the same as MLP, including a linear layer composed of a randomly initialized weight matrix, a ReLU activation function, and a Dropout layer; r is the positioning data of the underwater submersible obtained by decoding; The root mean square error RMSE of the output r and the sample label value r` is calculated as the loss function L. The gradient descent of L is iterated until the function reaches the preset convergence condition, and the parameter setting in the Transformer-based deep neural network is obtained to complete the training.

7. The positioning method based on multimodal data fusion and integrated learning according to claim 5 is characterized in that ,The iteration of gradient descent on L includes: using Adam optimization algorithm to calculate gradient descent.

8. A positioning system based on multimodal data fusion and integrated learning, used to execute the positioning method based on multimodal data fusion and integrated learning according to any one of claims 1 to 7, characterized in that: include: Acoustic Doppler speed measurement unit, used to obtain speed information and motion trajectory information of underwater submersibles; Strapdown inertial measurement unit, used to obtain three-axis linear acceleration data, three-axis angular velocity data and attitude information of the underwater submersible; A geomagnetic navigation unit, used to obtain the position information of the underwater submersible; A multi-modal feature fusion encoding unit is used to perform mixed encoding on three types of multi-modal data obtained by a Doppler velocity measurement unit, a strapdown inertial measurement unit and a geomagnetic navigation unit to obtain fused feature data; The underwater positioning unit is used to input the fused feature data into the Transformer-based deep neural network for training, and use the trained Transformer-based deep neural network to locate the underwater submersible.

Citation Information

Cited By

  • Underwater vehicle automatic obstacle avoidance strategy training method based on deep reinforcement learning

    CN119336032A

  • Automatic obstacle avoidance strategy training method for underwater vehicles based on deep reinforcement learning

    CN119336032B

  • CNN-LSTM-Attention neural network-based trajectory prediction method and system

    CN120003529A

  • A trajectory prediction method and system based on a CNN-LSTM-attention neural network

    CN120003529B

  • AUV (Autonomous Underwater Vehicle) three-dimensional pose joint estimation method and system based on multi-modal layering

    CN121582540A