Control method for large inertial objects based on ECS-LSTM network

Abstract

This invention discloses a control method for the motion of a large inertial object based on an ECS-LSTM network, comprising the following steps: S1: Constructing an ECS-LSTM network model; S2: Training the ECS-LSTM network model; S3: Acquiring data using sensors installed on the large inertial object; S4: Fusion and cleaning the acquired data to form instantaneous environmental situation image data and motion parameters of the large inertial object; S5: Feeding the environmental situation image data combined with the motion parameters of the large inertial object into the ECS-LSTM network model for continuous iteration until the entire system terminates; S6: Recombining the key node information output by the S-LSTM at each cycle to form executable control action commands, which are then sent to the motion actuator to complete motion control. This invention actively maintains the motion state of the large inertial object by perceiving the environmental situation. Before environmental factors act on the hull, it first forms the current environmental situation and formulates corresponding response strategies.

Description

Technical Field

[0001] This invention relates to the field of automatic control of the motion of large inertia objects, and in particular to a control method for the motion of large inertia objects based on an ECS-LSTM network. Background Technology

[0002] The motion control of aircraft commonly used in aerospace and maritime fields falls under the category of motion control for large inertia objects. These large inertia objects generally exhibit excellent performance in their respective fields. However, when encountering sudden environmental changes, such as the effects of strong winds or waves, or sudden collisions, these large inertia objects are prone to yaw without self-stabilization or attitude control. Most current solutions involve gradually restoring the object to its original course after yaw, thus returning it to its original motion state. This method of gradually restoring the large inertia object to its original motion state after being affected is passive.

[0003] The advantage of the motion control method of the present invention is that it is proactive. A large-inertia moving object utilizes sensors with a certain detection range to sense changes in the surrounding environment in advance, collect environmental factors that may cause changes in the object's motion, and analyze and form the current environmental situation of the large-inertia moving object. Then, based on the environmental situation, the magnitude and direction of the possible forces acting on it are analyzed. The object then uses its own power device to generate force to counteract the influence of these external environmental forces, thereby actively maintaining and stabilizing the current motion state of the large-inertia object. Similarly, through the perception of the environmental situation, the large-inertia object can also adjust its motion posture in advance or temporarily change its original motion posture to cope with the effects of environmental factors. When the environmental factors actually act on the moving object, the object can make secondary adjustments through its pre-adjusted posture and its own sensing system to actively maintain its original motion posture and achieve easier motion control. Summary of the Invention

[0004] Purpose of the invention: To address the above-mentioned problems, this invention provides a control method for the motion of large inertia objects by actively maintaining their motion state through the perception of the environmental situation. This method utilizes continuous sensing by an environmental sensing device with a certain range to first form the current environmental situation and formulate corresponding response strategies before environmental factors act on the hull. Then, the motion of the power system is controlled according to the response strategies, thereby overcoming the shortcomings of the prior art and proposing a control method for the motion of large inertia objects based on ECS-LSTM networks.

[0005] Technical solution: A control method for the motion of large inertia objects based on ECS-LSTM networks, comprising the following steps:

[0006] Step S1: Construct an ECS-LSTM network model, which contains two LSTM networks: the first is an EC-LSTM network, and the second is an S-LSTM network.

[0007] Step S2: Train the ECS-LSTM network model;

[0008] Step S3: Acquire data using sensors mounted on a large inertial object;

[0009] Step S4: The collected data is fused and cleaned to form the current instantaneous environmental situation image data and motion parameters of large inertial objects;

[0010] Step S5: Input the environmental situation image data into the first layer EC-LSTM network and input its output into the second layer S-LSTM network along with the motion parameters of the large inertia object to obtain key node information and global context information containing environmental situation and motion parameters; input it into the two networks in the ECS-LSTM of the next frame, and fuse it with the new environmental situation image data and motion parameters of the large inertia object obtained in the current frame. The key node information and global context information obtained after fusion are cyclically input into the next frame, and the process is iterated until the entire system terminates.

[0011] Step S6: Reassemble the key node information output by S-LSTM at each beat to form executable control action commands, and send them to the motion actuator to complete motion control.

[0012] Furthermore, the EC-LSTM is a CNN-LSTM network used to construct environmental situation and form environmental situation signals. Its model consists of five parts: the first part is the input layer, which takes the input signal dataset as input; the second part is the CNN network, which is used to extract the spatial features of the signal dataset; the third part is the LSTM network, which is used to extract the temporal features of the signal dataset; and the fourth part is the feature fusion layer, which concatenates the spatial features extracted by the CNN network with the temporal features extracted by the LSTM to obtain new signal features.

[0013] Furthermore, the S-LSTM is an LSTM network used to integrate temporal features and motion parameters to ultimately form an action instruction set. It has long-term memory, fully considers the temporal correlation of data, and can automatically adjust the contribution of historical information to the current prediction based on the current state. The LSTM network consists of multiple memory cell layers, where the neurons include a cell state C(t), a forget gate Ft, an input gate IT, and an output gate OT. Throughout the entire cycle, the cell state is updated by maintaining the state of the memory cells, i.e., updating the three gate structures, thereby achieving the purpose of updating motion parameters by integrating temporal features.

[0014] Furthermore, the specific steps of step S2 are as follows:

[0015] 2.1 Initialize the weights and pre-set values ​​of the entire network using a random function;

[0016] 2.2 Extract a valid environmental situation response dataset from the historical database as a training sample and input it into the ECS-LSTM network;

[0017] 2.3 Obtain the result set through forward propagation and calculate the loss function;

[0018] 2.4 The weights and biases are further determined using backpropagation and gradient descent.

[0019] 2.5 Repeat steps 2.2 to 2.4 until the ECS-LSTM network obtains the correct output.

[0020] Furthermore, step S3 includes: acquiring 3D point cloud data using a 360° scanning laser ranging radar; acquiring depth image data with depth information using a depth camera; acquiring the current instantaneous roll angle, yaw angle, and pitch angle data of the hull using a gyroscope; acquiring the acceleration components of the port and starboard sides of the bow and stern in the X-axis, Y-axis, and Z-axis directions using a shipborne accelerometer; obtaining the current GPS positioning of the ship using a Beidou satellite positioning chip; and obtaining motor parameters using a servo motor.

[0021] Furthermore, step S4 specifically involves: calibrating and associating 3D point cloud data and depth image data, and fusing them using a Kalman filter algorithm in conjunction with a Hungarian matching algorithm to form the current instantaneous environmental situation image data; fusing the roll angle, yaw angle, pitch angle, and the current acceleration components of the bow and stern to obtain the motion parameters of a large inertia object.

[0022] Furthermore, in step S5, the current environmental situation image data is fed into the EC-LSTM network to obtain the current environmental parameter vector x. e ; Set the current environment parameter vector x e With motion parameter vector x s The data is fed into an S-LSTM network to obtain a new motion parameter vector x. ns The new motion parameter vector x ns Structure and x s same.

[0023] Furthermore, the environmental parameter vector

[0024]

[0025] Environmental parameter vector x eIt is the output vector of EC-LSTM, where f ha f hal f har Let θ represent the magnitudes of the forces acting directly in front of the bow, at 45° to the left of the bow, and at 45° to the right of the bow. ha θ hal θ har f ha f hal f har The angles of deflection of the three forces in the three directions of force application need to be specified, θ. ha It is usually 0, while θ hal θ har ∈(-45°, +45°); f hl f hr θ represents the magnitude of the forces received on the port and starboard sides of the bow. hl θ hr It is the direction of the resultant force acting on the port and starboard sides of the bow, and its value ranges from (0° to 180°); f ta f tal f tar Let θ represent the magnitudes of the forces acting directly behind the stern, 45° to the left of the stern, and 45° to the right of the stern. ta θ tal θ tar The angles of deflection of the three forces in the three directions of force application need to be specified (θ). ta It is usually 0, while θ tal θ tar ∈(-45°, +45°); f tl f tr θ represents the magnitude of the forces acting on the port and starboard sides of the stern. tl θ tr It is the direction of the resultant force on the port and starboard sides of the bow, and its value ranges from 0° to 180°.

[0026] Furthermore, the motion parameter vector

[0027] x s =(f hl1 ,f hl2 ,f hr1 ,f hr2 ,f tl1 ,f tl2 ,f tr1 ,f tr2 ,f t1 ,f t2 ,θ r ,θ p ,θ y ,v x ,v y ,vz ) T

[0028] Motion parameter vector x s These are the input parameters of the S-LSTM, where f hl1 ,f hl2 Let f be the thrust of the two propellers on the port side of the bow at time t. hr1 ,f hr2 Let f be the thrust of the two propellers on the starboard side of the bow at time t. tl2 ,f tr1 Let f be the thrust of the two propellers on the port side of the stern at time t. tr1 ,f tr2 Let f be the thrust of the two propellers on the right side of the stern at time t. t1 ,f t2 Let θ be the thrust of the two propellers directly behind the stern at time t. r ,θ p ,θ y These are the current ship's roll angle, pitch angle, and yaw angle, v. x ,v y ,v z These are the current velocity components of the ship along the x, y, and z axes, respectively.

[0029] Furthermore, step S6 specifically involves the following steps:

[0030] 6.1 x ns The force required to be output by each propeller in the vector group is converted into the rotational speed of each propeller. The formulas for thrust and rotational speed are as follows:

[0031]

[0032] Where: K p K is the thrust coefficient. m Where is the torque coefficient, D is the propeller diameter, and n is the rotational speed. s p is the density of water;

[0033] 6.2 Motion control is achieved through a PID control algorithm. The PID control formula is as follows:

[0034]

[0035] In the formula: C is the actual input rotational speed parameter, e is the error term, i.e., the error between the target output and the actual output, P is the proportional coefficient, and T is the proportional coefficient. i It is the integral time coefficient, T d It is the differential time coefficient;

[0036] The final result is a vector composed of the motor's rotational speed, which is distributed to the servo motor by the control system. After receiving the rotational speed command, the servo motor completes the rotation command for the current cycle, thereby completing the control of the motion of the large inertia object.

[0037] Beneficial effects: By adopting active motion control methods, the motion posture of objects with large inertia can be adjusted in advance, or the original motion posture can be temporarily changed to cope with the impact of environmental factors. When the environmental factors actually act on the moving object, the moving object can make secondary adjustments through the pre-adjusted posture and its own sensing system, so that the original motion posture is maintained and the motion is easier to control. Attached Figure Description

[0038] Figure 1 The flowchart shows the control method for large inertia object motion based on ECS-LSTM network.

[0039] Figure 2 This is a schematic diagram of the EC-LSTM network structure;

[0040] Figure 3 The sequential logic diagram of S-LTSM;

[0041] Figure 4 Diagram of LSTM cell structure;

[0042] Figure 5 This is a schematic diagram of PID control;

[0043] Figure 6 This is a schematic diagram showing the distribution of the ship's motion mechanisms;

[0044] Figure 7 A schematic diagram showing the distribution of accelerometers and gyroscopes on the ship's hull;

[0045] Figure 8 This is a schematic diagram showing the distribution of other sensors on the ship's hull. Detailed Implementation

[0046] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0047] like Figure 1 As shown, a control method for the motion of a large inertia object based on an ECS-LSTM network includes the following steps:

[0048] Step 1: Construct an ECS-LSTM network model. This model contains two LSTM networks: the first is an EC-LSTM network, and the second is an S-LSTM network.

[0049] Furthermore, the ECS-LSTM network is composed of a Convolutional Neural Network (CNN) and a Long Short-Term Memory (LSTM) network. A CNN typically consists of an input layer, convolutional layers, pooling layers, fully connected layers, and an output layer. The convolutional and pooling layers can have multiple layers. Unlike fully connected feedforward neural networks, CNNs use convolutions instead of full connections, retaining full connections only in the last few layers. This reduces the number of parameters that need to be trained, effectively accelerating model training. The biggest differences between convolutional and fully connected layers are twofold: local connectivity and weight sharing. Local connectivity means that each neuron in a convolutional layer connects only to neurons in a specific region of the previous layer, learning local features. This local perceptual structure is inspired by the concept of receptive fields, and the size of this local connection region is equal to the size of the convolutional kernel. Compared to fully connected layers, the number of connections between a convolutional layer and the previous layer is greatly reduced; therefore, local connectivity is also called sparse connectivity. Weight sharing means that the convolutional kernels of a convolutional layer share the same parameters, including the weight matrix and bias terms. While local connectivity reduces the number of parameters, the number of parameters that need to be trained is still quite large, hindering efficient network training. Using weight sharing can significantly reduce the number of parameters, allowing only one set of parameters to be learned during convolution.

[0050] The primary function of a convolutional layer is to extract local features. It is the most important layer in a convolutional neural network (CNN), analyzing each local region of the network in greater depth to obtain more abstract features. Convolutional layers used in image processing are typically three-dimensional node matrices, with a size of M × N × D, containing D feature maps of size M × N. Each feature map represents a portion of the abstract features learned by the neural network after the convolution operation. By using multiple different convolutional kernels, multiple different feature maps are obtained to better represent the image. A convolutional layer mainly consists of an input feature map set X∈R. M*N*D Output feature mapping set Y∈R M*N*r Convolution kernel W∈R M*N*D*P The convolutional layer maps input feature set X to output feature set Y. p The intermediate calculation process is as follows:

[0051]

[0052] Among them W p Let X be the convolution kernel, and b be the input feature vector. p The bias value is used, and the activation function f is typically the ReLU function, whose formula is as follows:

[0053]

[0054] Pooling layers, also known as subsampling layers, are typically located between convolutional layers. Their function is to effectively reduce the size of the node matrix, decrease the feature dimension, compress the amount of data and parameters, speed up network computation, and prevent overfitting. For example, when the input is an image, the main function of the pooling layer is to compress the image. This is based on the theory of image feature invariance. When compressing image data, the pooling layer filters out redundant and irrelevant information, retaining only the most important feature information.

[0055] This algorithm employs max pooling, which refers to finding the maximum value of the matrix after the current convolution. The pooling function formula is as follows:

[0056]

[0057] in For the pooling result of layer d with M rows and N columns, x i This refers to the i-th convolution result.

[0058] This patent utilizes a convolutional neural network to extract the spatial situational features of the current instantaneous environment. After obtaining these features, it is necessary to combine them with a long short-term memory network (LSTM) to extract features of the current situation across the entire time and space, so as to construct a complete environmental situation. This allows large inertial objects to obtain the environmental situation of the current time period before control.

[0059] Long Short-Term Memory (LSTM) networks are recurrent neural networks with a special structure that enhances memory capabilities. The main function of LSTM networks is to effectively control the information transfer of the control state c by introducing a gating mechanism. This gating mechanism consists of three gates: the input gate, the forget gate, and the output gate. These three gates have the same structure, primarily composed of the sigmoid function and dot product operations, utilizing the properties of the sigmoid function to control information transfer. The time-expanded structure of LSTM is shown below. Figure 3 The structure of unit state C can be seen below. Figure 4 Its activation function, the sigmoid function, is formulated as follows:

[0060]

[0061] This function is a monotonically increasing function, and its value range is (0,1).

[0062] In LSTM networks, in addition to using the sigmoid function as the activation function, the tanh function is also needed. The formula for the tanh function is as follows:

[0063]

[0064] This function is also a monotonically increasing function, and its range of values ​​is (-1, 1).

[0065] Depend on Figure 4 The detailed structure of an LSTM recurrent cell is as follows: `it` represents the activation value of the input gate, which determines how much input information needs to be stored in the memory cell at the current time step; `Ft` represents the activation value of the forget gate, which determines how much information from the previous cell state needs to be forgotten; and `ot` represents the activation value of the output gate, which determines how much information from the current cell state `ct` needs to be output. The entire cell's computation process is as follows:

[0066] c t =Ft⊙c t-1 +it⊙cwt

[0067] h t =ot⊙tanh(c t )

[0068] Where ⊙ represents the element-wise product of vectors, c t-1 cwt represents the cell state at the previous time step.

[0069] cwt=tanh(W c x t +U c h t-1 +b c )

[0070] Where x t W represents the input at time t. c U c b represents the weight vector of the gate corresponding to the memory cell. c This represents the bias term for the corresponding gate.

[0071] it=σ(W i x t +U i h t-1 +b i )

[0072] Ft=σ(W f x t +U f h t-1 +b f )

[0073] ot=σ(W o x t +U o h t-1 +b o )

[0074] Where σ(·) represents the sigmoid function, h t-1 This represents the hidden layer state at the previous time step.

[0075] The computation steps of an LSTM network mainly include three steps.

[0076] Input x at the current moment t and the hidden layer h at the previous time step t-1 The activation values ​​it, Ft, ot of the three gates and the candidate state cwt are calculated.

[0077] Based on the previous cell state c t-1 The forget gate activation function ft takes the input gate activation value it and candidate states cwt to update the current cell state c. t .

[0078] The output gate activation value ot and the current cell state c are used. t To calculate the hidden layer h at the current time. t .

[0079] The entire LSTM loops through these three steps to build long-distance temporal dependencies.

[0080] like Figure 2 As shown, the EC-LSTM network is a standard CNN-LSTM network (Convolutional Neural Network-Long Short-Term Memory network). It first uses a convolutional neural network to obtain the spatial features of environmental perception data obtained from sensors, and then feeds the spatial features into an LSTM network. Utilizing the ability of LSTM to record temporal states, it obtains the temporal features of the current environmental situation, thus obtaining complete spatiotemporal features about the environmental situation. Finally, this spatiotemporal feature is fed into the second layer of the LSTM network, namely S-LSTM. This long short-term memory layer uses the spatiotemporal feature data of the environmental situation obtained from the previous layer and combines it with the ship motion data obtained from the ship motion sensor, along with the previous temporal recording of the ship's motion state, to complete the judgment of the current instantaneous motion situation and the input of motion parameters.

[0081] Furthermore, as can be seen from step 1, in order to fully construct the entire ECS-LSTM network, in addition to constructing the network structure, it is also necessary to obtain the weight vectors W and U and the bias vector b at each position. To obtain the specific parameters of these vectors, it is necessary to train the entire ECS-LSTM network, which is step 2.

[0082] Step 2: Train the ECS-LSTM network model.

[0083] Training an ECS-LSTM network requires the following steps:

[0084] 2.1 Use a random function to initialize the weights and pre-set values ​​of the entire network.

[0085] 2.2 Extract a valid environmental situation response dataset from the historical database as a training sample and feed it into the ECS-LSTM network.

[0086] 2.3 Obtain the result set through forward propagation and calculate the loss function.

[0087] 2.4 Backpropagation and gradient descent are used to further determine the weights and biases.

[0088] 2.5 Repeat the above steps until the ECS-LSTM network obtains the correct output.

[0089] Furthermore, in neural network learning, gradient descent is primarily used to learn the parameters. The basic idea of ​​the gradient descent algorithm is to update the model parameters along the inverse direction of the gradient of the loss function L(y,f(x)), and the step size of each parameter update is determined by setting the learning rate η, gradually reducing the loss until the loss converges. During training, backpropagation is typically used to calculate the loss function.

[0090] The training process of the backpropagation algorithm is as follows:

[0091] Calculate the input and activation values ​​for each layer.

[0092] The error value of each layer is calculated by backpropagation.

[0093] Calculate the partial derivatives of the parameters for each layer and update them.

[0094] In this patent, the loss function L(y,f(x)) used is the mean square error function (MSE), and its formula is as follows:

[0095]

[0096] Where y i To represent the actual results, this patent refers to the response strategies given in the historical database for different environmental situations, f(x) i The predicted value is the result obtained after the environmental situation parameters are fed into the ECS-LSTM network. The advantage of choosing this loss function is that it is continuous at all points, which makes it easy to differentiate and has a relatively stable solution.

[0097] Furthermore, after training the ECS-LSTM network using historical datasets, the network can be applied to the motion control of objects with large inertia. The acquisition of parameters is step 3.

[0098] Step 3: As Figure 7-8As shown, various specialized sensors mounted on large inertial objects acquire data.

[0099] 1. A 360° scanning laser ranging radar acquires 3D point cloud data. Its scanning frequency is 10 Hz, the angular resolution is 0.08°, and the number of point clouds in each frame is 360° / 0.08° = 4500. With 10 frames per second, the total number of point clouds is 45000, which means the sampling frequency is 45 kHz. Its communication method is serial, and the communication interface is USB 1.0. The data it obtains is a point cloud matrix composed of 4500 points.

[0100] 2. Use a depth camera to acquire depth image data with depth information. The resolution is 640×480, the frame rate is 30fps, the communication method is serial, and the communication interface is USB1.0. The acquired data is 640×480 resolution graphic matrix data with RGBH channels.

[0101] 3. Use gyroscopes to obtain the ship's instantaneous roll angle, yaw angle, and pitch angle data.

[0102] 4. Use the ship's onboard accelerometer to obtain the acceleration components of the port bow, starboard bow, port stern, and starboard stern in the X-axis, Y-axis, and Z-axis directions, respectively.

[0103] In this patent, both the accelerometer and the gyroscope are 6-axis gyroscopes with an operating voltage between 1.71V and 3.6V. They use SPI serial communication, have a sensitivity error of ±0.5%, a gyroscope noise of 3.8 mdps / √Hz, and an accelerometer noise of 70 μg / √Hz. The data obtained are floating-point parameter vectors.

[0104] 5. Obtain the current GPS positioning of the ship using a Beidou satellite positioning chip. This patent uses the "Hangxin-1" 40nm CMOS BDS / GPS RF baseband integrated SoC Beidou navigation chip. This device uses UART serial communication, with sensitivity of -148dBm for acquisition and -165dBm for tracking, positioning accuracy of 2.5 meters CEP, speed measurement accuracy of 0.1 meters / second CEP, and timing accuracy of 25ns. The data obtained is a parameter vector containing positioning data and timestamps.

[0105] 6. Obtain motor speed data using a servo motor. The servo motor has a working power of 750W, a rated current of 3A, a rated torque of 2.4N·m, a maximum torque of 7.1N·m, a rated speed of 3000rpm, and a rotor inertia of 1.82×10⁻⁶. -4 kg·m 2 It adopts the CANOpen serial communication protocol, and the data it obtains is a parameter vector containing parameters such as the motor speed.

[0106] Furthermore, after obtaining all parameters through the above dedicated sensors, it is necessary to eliminate noise in the parameters, and then fuse and merge the parameters of the depth camera and LiDAR to form a parameter matrix that can be used by the ECS-LSTM network. The data fusion and denoising is step 4.

[0107] Step 4: The collected data is fused and cleaned to form the current instantaneous environmental situation image data and motion parameters of large inertial objects.

[0108] 1. The 3D point cloud and depth image are calibrated and correlated, and then fused using the Kalman filter algorithm in conjunction with the Hungarian matching algorithm to form the current instantaneous environmental situation image data.

[0109] 2. By fusing the roll angle, yaw angle, pitch angle, and the current acceleration components of the bow and stern, the motion parameter data of the current object can be obtained.

[0110] Furthermore, for the fusion of 3D point clouds and depth images, the data is first calibrated by calculating the homography matrix H between the two images using the coordinates of four pairs of corresponding points. Then, a projective transformation function is called to transform one image into the viewpoint of the other. The homography matrix allows the pixel coordinates in the depth image to be transformed to their corresponding positions in the 3D point cloud.

[0111] Homography is a concept in projective geometry, also known as a projective transformation. It maps a point (a three-dimensional homogeneous vector) on one projective plane to another, and maps a straight line to a straight line, preserving the linearity. In general, homography is a linear transformation of a three-dimensional homogeneous vector, which can be represented by a 3x3 nonsingular matrix H. The equation for homogeneous coordinates is as follows:

[0112] x1 = Hx2

[0113] Here, H is a 3×3 homogeneous matrix with 8 unknowns. Assuming homography between two images has been obtained, the homography matrix H can correlate the two images. This has many practical applications, such as image correction, alignment, and estimating motion between two cameras in a system. Here, it is used for joint calibration. First, the radar plane is assumed to be a two-dimensional horizontal plane, ignoring its slope. Thus, point p1(x1,y1) on the radar plane and point p2(x2,y2) on the image are a matched pair, with a homography matrix H. Then, we have...

[0114]

[0115] Further, after calibration, the next step is image fusion. Input fusion mainly uses a combination of Hungarian matching and Kalman filtering. The core idea of ​​Hungarian matching is: initially, the maximum matching is the control, then augmenting paths are continuously searched and expanded. This process is repeated until no augmenting path can be found. If there are n points in the left half U of the bipartite graph, then at most n augmenting paths can be found. If there are m edges in the graph, then each time an augmenting path is found (depth-first search or breadth-first search), all edges are traversed at most once, which takes m time. Therefore, the total time is approximately O(nm).

[0116] Furthermore, after processing the data, it is sent to the ECS-LSTM network to obtain the final result parameters, which is step 5.

[0117] Step 5: The fused environmental situation image data is fed into the first-layer EC-LSTM network, and its output is combined with the object motion parameters and fed into the second-layer S-LSTM network to obtain the key nodes and initial global context information containing environmental situation and motion parameters. The data is then fed into the two networks in the next ECS-LSTM to be fused with the new environmental situation image data and large inertia object motion parameters obtained in the current cycle. The key points and global context information obtained after fusion are cyclically input into the next cycle for continuous iteration until the entire system terminates.

[0118] The current environmental situation image data is fed into the EC-LSTM network to obtain the current environmental parameter vector x. e

[0119] The current environment parameter vector x e With motion parameter vector x s The data is fed into an S-LSTM network to obtain a new motion parameter vector x. ns

[0120] The environmental parameter vector structure is as follows:

[0121]

[0122] Environment vector x e It is the output vector of EC-LSTM, where f ha f hal f har Let θ represent the magnitudes of the forces acting directly in front of the bow, at 45° to the left of the bow, and at 45° to the right of the bow. ha θ hal θ har The angles of deflection of the three forces in the three directions of force application need to be specified (θ). ha It is usually 0, while θ hal θhar ∈(-45°, +45°), f hl f hr θ represents the magnitude of the forces received on the port and starboard sides of the bow. hl θ hr It is the direction of the resultant force acting on the port and starboard sides of the bow, and its value ranges from (0° to 180°), f ta f tar f tal Let θ represent the magnitudes of the forces acting directly behind the stern, 45° to the left of the stern, and 45° to the right of the stern. ta θ tal θ tar The angles of deflection of the three forces in the three directions of force application need to be specified (θ). ta It is usually 0, while θ tal θ tar ∈(-45°, +45°), f tl f tr θ represents the magnitude of the forces acting on the port and starboard sides of the stern. tl θ tr It is the direction of the resultant force on the port and starboard sides of the bow, and its value ranges from 0° to 180°.

[0123] The structure of the motion parameter vector is as follows:

[0124] x s =(f hl1 ,f hl2 ,f hr1 ,f hr2 ,f tl1 ,f tl2 ,f tr1 ,f tr2 ,f t1 ,f t2 ,θ r ,θ p ,θ y ,v x ,v y ,v z ) T

[0125] Motion parameter x s These are the input parameters of the S-LSTM, where f hl1 ,f hl2 Let f be the thrust of the two propellers on the port side of the bow at time t. hr1 ,f hr2 Let f be the thrust of the two propellers on the starboard side of the bow at time t. tl2 ,f tr1 Let f be the thrust of the two propellers on the port side of the stern at time t. tr1 ,ftr2 Let f be the thrust of the two propellers on the right side of the stern at time t. t1 ,f t2 Let θ be the thrust of the two propellers directly behind the stern at time t. r ,θ p ,θ y These are the current ship's roll angle, pitch angle, and yaw angle, v. x ,v y ,v z These are the current velocity components of the ship along the x, y, and z axes, respectively.

[0126] Furthermore, the new motion parameter x is ultimately obtained. ns Its structure is similar to x s The same applies. After obtaining the specific motion parameters, converting these parameters into specific instructions that the mechanism can execute is step 6.

[0127] Step 6: Reassemble the joints output by S-LSTM at each beat to form specific executable control commands, and send them to the motion actuator to complete motion control. Figure 6 This is a schematic diagram of the propeller distribution, which is part of the ship's motion mechanism; x ns The force required to be output by each propeller in the vector group is converted into the rotational speed of each propeller. The formula for rotational speed and thrust is as follows:

[0128]

[0129] Where: K p K is the thrust coefficient. m Let be the torque coefficient, and D be the propeller diameter. The rotational speed is n. s p is the density of water.

[0130] Motion control is achieved through a PID control algorithm mechanism; the entire control process is as follows. Figure 5

[0131] The formula for PID control is as follows:

[0132]

[0133] In the formula, c is the actual input rotational speed parameter, e is the error term, i.e. the error between the target output and the actual output, P is the proportional coefficient, Ti is the integral time coefficient, and Td is the derivative time coefficient.

[0134] The final result is a vector composed of the rotational speeds of the 10 motors, which is distributed to the servo motors by the control system. After receiving the rotational speed command, the servo motors complete the rotation command for the current cycle, thereby controlling the motion of the large inertia object.

[0135] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A control method for motion of a large-inertia object based on an ECS-LSTM network, characterized by, The method comprises the following steps Step S1: constructing an ECS-LSTM network model, which contains two LSTM networks, the first one being an EC-LSTM network and the second one being an S-LSTM network; Step S2: training the ECS-LSTM network model; Step S3: acquiring data by using sensors installed on the large inertia object; Step S4: forming current instantaneous environment situation image data and large inertia object motion parameters by data fusion and cleaning of the acquired data; Step S5: sending the environment situation image data into the first layer EC-LSTM network and sending the output result thereof combined with the large inertia object motion parameters into the second layer S-LSTM network to obtain key node information and global context information containing the environment situation and the motion parameters; The two networks in the ECS-LSTM input into the next beat are fused with new environment situation image data and large inertia object motion parameters acquired in the current beat, and the key node information and global context information obtained after the fusion are input into the next beat for iteration until the whole system is terminated; Step S6: recombining the key node information output by the S-LSTM at each beat to form executable control action instructions and issuing the control action instructions to a motion execution mechanism to complete motion control. 2.The control method of large inertia object motion based on ECS-LSTM network according to claim 1, wherein, The EC-LSTM is a CNN-LSTM network used to construct an environment situation and form an environment situation signal, and the model thereof comprises five parts: the first part is an input layer for inputting a signal data set; the second part is a CNN network used to extract spatial features of the signal data set; the third part is an LSTM network used to extract time sequence features of the signal data set; and the fourth part is a feature fusion layer used to concatenate the spatial features extracted by the CNN network and the time sequence features extracted by the LSTM to obtain new signal features. 3.The control method of large inertia object motion based on ECS-LSTM network according to claim 1, wherein, The S-LSTM is an LSTM network used to comprehensively integrate time sequence features and motion parameters and finally form an action instruction set, has long-term memory, fully considers the time correlation of data, can automatically adjust the contribution of historical information to current prediction in a recurrent neural network, and comprises multiple memory cell layers, wherein a neuron cell comprises a cell state C(t), a forgetting gate Ft, an input gate IT and an output gate OT, the state of the memory cell is maintained in the whole cycle, the three gate structures are updated, the cell state is updated, and the purpose of comprehensively updating the time sequence features and the motion parameters is achieved.

4. The control method for motion of a large-inertia object based on an ECS-LSTM network according to claim 1, characterized in that, The step S2 specifically comprises the following steps: 2.1 initializing the weights and prepositions of the whole network by using a random function; 2.2 extracting effective environment situation response data sets from a historical database as training samples and inputting the training samples into the ECS-LSTM network; 2.3 obtaining a result set by forward propagation and calculating a loss function; 2.4 further determining the weights and biases by using back propagation and gradient descent method; 2.5 repeating steps 2.2 to 2.4 until the ECS-LSTM network obtains correct output.

5. The control method of motion of a large-inertia object based on an ECS-LSTM network according to claim 1, characterized in that, The step S3 comprises: acquiring 3D point cloud data by using a 360° scanning laser ranging radar; acquiring depth image data with depth information by using a depth camera; acquiring roll angle, yaw angle and pitch angle data of the current instantaneous ship body by using a gyroscope; acquiring acceleration components of the left and right sides of the bow and the left and right sides of the stern in the X-axis direction, the Y-axis direction and the Z-axis direction by using a shipboard accelerometer; obtaining the GPS positioning of the current ship by using a Beidou satellite positioning chip; and obtaining motor parameters by using a servo motor.

6. The control method of motion of a large-inertia object based on an ECS-LSTM network according to claim 5, characterized in that, The step S4 is specifically: calibrating and correlating the 3D point cloud data and the depth image data, and fusing them by using a Kalman filtering algorithm and a Hungarian matching algorithm to form current instantaneous environment situation image data; and fusing the roll angle, the yaw angle, the pitch angle and the acceleration components of the current bow and stern to obtain the motion parameters of the large-inertia object.

7. The control method of large inertia object motion based on ECS-LSTM network according to claim 1, characterized in that, The step S5 sends the current environment situation image data into the EC-LSTM network to obtain a current environment parameter vector x e ; the current environment parameter vector x e is sent into the S-LSTM network with a motion parameter vector x s to obtain a new motion parameter vector x ns , and the new motion parameter vector x ns has the same structure as x s .

8. The control method of motion of a large-inertia object based on an ECS-LSTM network according to claim 7, characterized in that, The environment parameter vector Environmental parameter vector x e It is the output vector of EC-LSTM, where f ha f hal f har Let θ represent the magnitudes of the forces acting directly in front of the bow, at 45° to the left of the bow, and at 45° to the right of the bow. ha θ hal θ har f ha f hal f har The angles of deflection of the three forces in the three directions of force application need to be specified, θ. ha It is usually 0, while θ hal θ har ∈(-45°, +45°); f hl f hr θ represents the magnitude of the forces received on the port and starboard sides of the bow. hl θ hr It is the direction of the resultant force acting on the port and starboard sides of the bow, and its value ranges from (0° to 180°); f ta f tal f tar Let θ represent the magnitudes of the forces acting directly behind the stern, 45° to the left of the stern, and 45° to the right of the stern. ta θ tal θ tar The angles of deflection of the three forces in the three directions of force application need to be specified (θ). ta It is usually 0, while θ tal θ tar ∈(-45°, +45°); f tl f tr θ represents the magnitude of the forces acting on the port and starboard sides of the stern. tl θ tr It is the direction of the resultant force on the port and starboard sides of the bow, and its value ranges from 0° to 180°.

9. The control method of motion of a large-inertia object based on an ECS-LSTM network according to claim 7, characterized in that, The motion parameter vector x s = (f hl1 ,f hl2 ,f hr1 ,f hr2 ,f tl1 ,f tl2 ,f tr1 ,f tr2 ,f t1 ,f t2 ,θ r ,θ p ,θ y ,v x ,v y ,v z ) T motion parameter vector x s is the S-LSTM input part parameter, where f hl1 ,f hl2 is the thrust of the left two propellers at the bow of the ship at time t, f hr1 ,f hr2 is the thrust of the right two propellers at the bow of the ship at time t, f tl2 ,f tr1 is the thrust of the left two propellers at the stern of the ship at time t, f tr1 ,f tr2 is the thrust of the right two propellers at the stern of the ship at time t, f t1 ,f t2 is the thrust of the two propellers at the stern of the ship at time t, θ r ,θ p ,θ y are the roll angle, pitch angle, yaw angle of the current ship respectively, v x ,v y ,v z are the velocity components of the current ship in x, y, z axes respectively.

10. The control method of motion of a large-inertia object based on an ECS-LSTM network according to claim 7, characterized in that, The specific steps of the step S6 are, 6.1 x ns The required force on each propeller of the vector group to be delivered is converted into the rotational speed of each propeller, the formula for the thrust and rotational speed being as follows: where: K p is the thrust coefficient, K m is the torque coefficient, D is the diameter of the propeller, n is the rotational speed s p is the density of water; 6.2 The motion control is completed by using a PID control algorithm mechanism, and the formula of the PID control is as follows: where C is the true input speed parameter, e is the error term, i.e., the error between the target output and the actual output, P is a proportional coefficient, T i is an integral time coefficient, and T d is a derivative time coefficient. The finally obtained result is a vector composed of the speed of the motor, which is distributed to the servo motor by the control system. After obtaining the speed instruction, the servo motor completes the rotation instruction of the current beat, and then completes the control of the motion of the large-inertia object.

Images