Intelligent steering based on machine learning

Abstract

The present invention relates to an intelligent steering based on machine learning, and belongs to the field of steering gear control technology. The method comprises the following steps: planning a tracking path for a ship to return to a target path from a current position, selecting a reference point from the tracking path as a reference point for calculating the error of steering control; obtaining the tracking deviation by setting parameters and detecting parameters, and correcting the operating conditions of the tracking deviation based on a machine learning model; converting the corrected tracking deviation into a steering angle of a steering gear, sequentially establishing a steering control parameter table between two reference points, and using a PD controller to control the output with the steering gear as an actuator; the present invention selects reference points for tracking deviation calculation by planning the tracking path, and compensates for the deviation effect caused by interference factors based on machine learning, thereby improving the accuracy and efficiency of path tracking.

Description

Technical Field

[0001] The present invention belongs to the technical field of steering gear control, and in particular relates to an intelligent steering system based on machine learning. Background Art

[0002] Intelligent steering is a steering gear control method that combines advanced technology and algorithms, aiming to improve the steering accuracy and efficiency during navigation, thereby achieving automatic error correction of the navigation track deviation angle.

[0003] At present, the steering gear system uses PID controller to complete the ship's rudder control. The error calculation process in the PID control process directly selects points on the target path as reference points, which may cause control oscillations in the steering gear. At the same time, only local optimization is considered without considering the adaptation of the global path. In addition, the control of the rudder angle does not take into account the interference of operating conditions. External environmental factors such as wind flow and waves may affect the actual trajectory of the ship, resulting in low accuracy and low efficiency in the automatic correction process.

[0004] Therefore, it is urgent to propose an intelligent steering method that considers the path regression process from a global perspective and achieves higher accuracy and efficiency by planning and tracking the path, so as to better cope with environmental interference and dynamic changes, reduce oscillations, and optimize the adaptability of the target path. Summary of the invention

[0005] In order to solve the above problems existing in the prior art, the present invention provides an intelligent steering system based on machine learning, which calculates the tracking deviation by planning the tracking path and selecting reference points. At the same time, it compensates for the deviation effect caused by interference factors based on machine learning, thereby improving the accuracy and efficiency of path tracking.

[0006] The purpose of the present invention can be achieved through the following technical solutions:

[0007] The present disclosure provides an intelligent steering method based on machine learning, comprising the following steps:

[0008] S1. Select reference point: plan the tracking path of the ship from the current position to the target path, and select the reference point from the tracking path as the reference point for error calculation of steering control;

[0009] S2. Obtaining rudder angle parameters: Obtaining tracking deviation by setting parameters and detecting parameters, and correcting the tracking deviation according to operating conditions based on the machine learning model;

[0010] S3, target path tracking: according to the corrected tracking deviation, it is converted into the steering angle of the servo, and the steering control parameter table between two reference points is established in sequence, and the servo is used as the actuator to use the PD controller for control output;

[0011] The step of selecting a reference point further comprises the following sub-steps:

[0012] S11, generating a tracking path: generating a tracking path according to the current position of the ship and the target path;

[0013] S12, obtaining reference points: setting the reference point selection interval according to the control fineness requirement on the generated optimal tracking path, and selecting a group of rudder angle control reference points starting from the starting point of the tracking path;

[0014] The step of generating a tracking path further comprises the following sub-steps:

[0015] S111, selecting a target point: setting a threshold value of the direction difference and the curvature difference, sampling a set of candidate target points from the target path, traversing the candidate target points, screening out target points whose direction difference and curvature difference are within the threshold range, and selecting the point closest to the current position as the final target point;

[0016] S112, generating an initial tracking path: according to the current position of the ship and the selected target point, the initial tracking path of the ship is randomly generated using B-spline control points;

[0017] S113, search for control points: encode the control points of the B-spline curve, search for the best control point configuration through the evolution process of the genetic algorithm, and achieve the optimal tracking path planning goal.

[0018] Furthermore, the generating of the initial tracking path further comprises the following sub-steps:

[0019] Assume the time domain is [ t s , t f ], the number of control points is n The B-spline curve trajectory is:

[0020] ;

[0021] In the formula, is the spline curve trajectory, t ∈[ t s , t f ]; are the control points of the B-spline curve, For the i indivual k B-spline basis function, basis function It is composed of the DeBoer-Cox recursion formula, expressed as:

[0022] ;

[0023] In the formula, U 1. U2 is the basis function The coefficient of

[0024] By introducing the B-spline curve into the ship path representation, a set of control point coordinates can be expressed as:

[0025] ;

[0026] The path expression generated by this group of control points is:

[0027] ;

[0028] In the formula, X ( t ) indicates that the point on the path is x The position change on the axis, Y ( t ) indicates that the point on the path is y Changes in position on the axis.

[0029] Furthermore, the control point search further includes the following sub-steps:

[0030] S1131, encode the control point and solve the control point ( , )coordinate:

[0031] ;

[0032] In the formula, ( , ) is the starting point of the path s and end point f Connection sf The coordinates of the equal division points, Starting point s and end point f The distance between rand is a random number and rand ∈[-0.5,0.5], For straight line sf The slope of

[0033] S1132, establishing a fitness function: setting constraints according to actual requirements in the error correction process to establish a fitness function;

[0034] S1133, Genetic Iterative Optimization: Minimize the fitness function through iterative optimization and obtain the optimal path.

[0035] Furthermore, the step of establishing the fitness function further includes the following sub-steps:

[0036] Define path distance: Calculate the path distance based on the position of the points on the trace path :

[0037] ;

[0038] In the formula, Indicates the path z Previous i discrete points, h is the number of discrete points;

[0039] Defining path smoothness: Calculating path smoothness by curvature :

[0040] ;

[0041] In the formula, Indicates the path i The direction angle of the point, which passes through the i Point and i +1 points are calculated based on their relative positions. Indicates the curvature change between three adjacent path points;

[0042] Define path direction: Define the path direction to fit the objective function , which is used to measure the consistency between the direction of the path and the direction of the target path:

[0043] ;

[0044] In the formula, The first i The direction angle of the point, The first i The direction angle of the point;

[0045] Establish a fitness function: assign weights to the established indicators and perform weighted summation to obtain the fitness function:

[0046] ;

[0047] In the formula, , , is the weight coefficient, which is used to adjust the importance of each indicator.

[0048] Furthermore, the genetic iterative optimization further includes the following sub-steps:

[0049] Randomly generate control point coordinates, determine the crossover probability, mutation probability and number of iterations, and use the B-spline interpolation algorithm to generate the tracking path based on the coordinates of the starting and target points.

[0050] Calculate the fitness function value of each path and record it as the fitness of each individual; perform roulette selection operation according to the fitness value of each individual to obtain the offspring, and the individual with the highest fitness among the individuals does not participate in the selection and directly enters the offspring;

[0051] Perform crossover and mutation operations to obtain n +1 generation individual; calculation n +1 generation individual fitness; judgment n Is it greater than the set number of iterations? If it is greater than the maximum number of iterations, proceed to the next step, otherwise return to the above operation; based on the obtained optimal individual, perform B-spline interpolation to obtain the optimal tracking path.

[0052] Furthermore, the step of obtaining the rudder angle parameter further includes the following sub-steps:

[0053] S21, obtaining tracking deviation: according to the selected reference point as the setting parameter, the feedback coordinate point of the GPS and inertial integrated navigation system is used as the detection parameter, and the tracking deviation between the setting parameter and the detection parameter is obtained;

[0054] S22. Tracking deviation correction: By collecting ship operating parameters, a machine learning model is used to correct the operating interference of the tracking deviation and compensate for the tracking deviation.

[0055] Furthermore, the obtaining of tracking deviation further includes the following sub-steps:

[0056] Use GPS and inertial combined navigation system to obtain real-time feedback coordinates, time-align the data, and convert the geographic coordinates provided by GPS into a coordinate system consistent with the inertial navigation system;

[0057] The Kalman filter algorithm is used to fuse the GPS and inertial combined navigation system data to obtain the optimal detection parameters, and the direction deviation between the set parameters and the detection parameters is calculated as the tracking deviation.

[0058] Furthermore, the tracking deviation correction further includes the following sub-steps:

[0059] S221. Establishing a machine learning model: Establishing a data set by collecting historical navigation data, constructing a machine learning model for training, and integrating the model into the ship's steering gear control system;

[0060] S222, collecting ship working condition data: setting corresponding sensors according to the prediction features selected by model training, collecting working condition data in real time, and sending the collected data to the steering gear system;

[0061] S223, correcting tracking deviation: predicting the output rudder angle compensation deviation by inputting the acquired real-time operating condition data into the machine learning model.

[0062] Furthermore, the establishment of the machine learning model further includes the following sub-steps:

[0063] Establishing data set: Establishing data set through historical navigation data. The data types include current position, target position, rudder angle, speed, heading, wind speed, wind direction, and ocean current information.

[0064] Clean the collected raw data, extract the features for prediction from the raw data by using the correlation coefficient method, and combine the features by using polynomial feature enhancement;

[0065] Construct BP neural network model: set the number of nodes in the input layer according to the selected prediction features, set the output layer to compensate for the deviation, set the number of layers and the number of nodes in each layer of the hidden layer according to the empirical formula, and select the activation function to complete the model construction;

[0066] Model training: The collected data is allocated as training set, validation set and test set in the ratio of 70%, 15% and 15% respectively; the data is passed to the hidden layer through the input layer, and then reaches the output layer through the activation function to calculate the output prediction value; the gradient of the loss function is calculated, and the error is back-propagated to each layer to adjust the weight and bias of each neuron; the weight of the model is updated through multiple iterations until the model converges or the set stop condition is reached;

[0067] The performance of the model is evaluated using validation set data and test set data.

[0068] Furthermore, the target path tracking further includes the following sub-steps:

[0069] Establish the control output of the PD controller: Combine the proportional and differential control outputs to get the final control output u(t):

[0070] ;

[0071] Where, e(t) is the tracking error at time t, K p is the proportional gain, K d is the differential gain;

[0072] Obtain the transfer function of the PD controller: In the continuous time domain, the transfer function of the PD controller is expressed as the relationship from error to control output. In the Laplace transform domain, the transfer function of the PD controller is G PD ( s ) is expressed as:

[0073] ;

[0074] in, s is the complex frequency variable in the Laplace transform;

[0075] According to the output formula of the PD controller, the control signal is calculated online, the control signal is limited, and the control signal calculated by the PD algorithm is converted into a control instruction of the servo, which drives the servo to perform the corresponding angle rotation;

[0076] For the parameters of the PD controller, a deep deterministic policy gradient algorithm is used for optimization:

[0077] Based on the reward function, the learning parameters of the Actor-Critic network are updated, and the effect of the control system under the current PD parameters is evaluated by the reward function. The input value of the reward function is the tracking deviation obtained after the control system is running and the output result of the PD algorithm. After the reward function is calculated, its output value is compared with the value estimate output by the Critic network through a comparator, and the obtained error value is sent to the Actor-Critic network to update the learning parameters of the Actor-Critic network.

[0078] Based on the Actor-Critic network training of PD parameters, the actor adjusts the PD parameters according to the deviation of the control system input, and the evaluator estimates the value of the adjustment effect based on the deviation obtained from the control system operation after the PD parameters are adjusted and the adjusted PD parameters. When the training results meet the set requirements, the training is stopped and the obtained PD parameters are output as the optimal value.

[0079] The beneficial effects of the present invention are:

[0080] The present invention firstly establishes a tracking path for correcting the deviation of the ship's navigation trajectory, selects a reference point from the tracking path as the basis for calculating the deviation of the steering control, refines the stage of the steering control, and thus improves the stability of the ship's operation and the smoothness of the regulation; then, based on the calculated deviation, the rudder angle control under the current ship's operating conditions is predicted through a machine learning model to compensate for the deviation, reduce the influence of interference factors on the actual trajectory of the ship, and improve the accuracy of path tracking; finally, the deviation is input into the PD controller to complete the steering control during navigation on the tracking path, quickly respond to input changes, and reduce overshoot and oscillation.

[0081] In the process of tracking path planning, the fitness function is established by setting constraints according to actual needs, so as to generate an optimal path that is smooth, easy to control and adapts to the target path. By flexibly selecting reference points, the controllability of the steering adjustment is achieved and the control accuracy is improved. In the process of deviation compensation, the machine learning algorithm is used to compensate for the influence of the rudder angle caused by the interference factors of the operating conditions, so that the deviation obtained by the PD controller is more accurate, the system responds faster, and the efficiency of path tracking is improved. In the steering control process of the PD controller, the deep deterministic policy gradient algorithm is used to optimize the parameters of the PD controller to adapt to different operating conditions and enhance the robustness of the control system. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] In order to facilitate understanding by those skilled in the art, the present invention is further described below with reference to the accompanying drawings.

[0083] Figure 1 A schematic diagram of steps of intelligent steering based on machine learning provided by an embodiment of the present invention;

[0084] Figure 2 A schematic diagram of the steps of selecting a reference point provided by an embodiment of the present invention;

[0085] Figure 3 A schematic diagram of the steps of generating a tracking path provided by an embodiment of the present invention;

[0086] Figure 4 A schematic diagram of the steps for obtaining rudder angle parameters provided by an embodiment of the present invention;

[0087] Figure 5 A schematic diagram of the steps of tracking deviation correction provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0088] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the specific implementation methods, structures, features and effects of the present invention are described in detail below in conjunction with the accompanying drawings and preferred embodiments.

[0089] Intelligent steering based on machine learning uses a data-driven approach to optimize steering gear control and collects historical navigation data from sensors and ship systems, including rudder angle, ship position, speed, environmental conditions, etc.; a machine learning algorithm is used to train the model to learn the steering compensation data of the ship under different conditions, and the compensation data is added to the deviation angle adjusted by the steering gear, thereby reducing the impact of operating conditions on path tracking. In this application, in order to consider the control oscillation in the process of regressing to the target path and the adaptability of the trajectory deviation to the target path after correction, the ship tracking path in the regression process is planned.

[0090] Specifically, the steps of the intelligent steering based on machine learning of the present invention are described in detail through the following contents:

[0091] This embodiment provides an intelligent steering based on machine learning, such as Figure 1 As shown, the following steps are included:

[0092] S1. Select reference point: Plan the tracking path of the ship from the current position to the target path, and select the reference point from the tracking path as the reference point for error calculation of steering control, such as Figure 2 As shown, the following steps are included:

[0093] S11, generate tracking path: generate tracking path according to the current position of the ship and the target path, such as Figure 3 As shown, the following steps are included:

[0094] S111, selecting target points: setting thresholds of direction difference and curvature difference, sampling a set of candidate target points from the target path, and these points can be sampled evenly or distributed as needed according to the geometric characteristics of the path;

[0095] Traverse the candidate target points, filter out the target points whose direction difference and curvature difference are within the threshold range, and select the point closest to the current position as the final target point among the target points that meet the conditions.

[0096] It can be understood that this embodiment generates a tracking path to enable the ship to return to the target path from the current position. When generating the tracking path, it is necessary to determine a point on the target path as the end point of the tracking path. However, the selection of the target point requires that the path from the current position of the ship to the target point achieves the best match with the target path in terms of curvature and direction to ensure the fit of the path direction and the feasibility of rudder angle adjustment.

[0097] It should be noted that if no target point that meets the conditions is found among the sampling points, the following measures can be taken: Relax the threshold: gradually increase the direction difference threshold and the curvature difference threshold until a suitable target point is found; Select the suboptimal point: in the absence of a suitable point, select the point with the smallest direction or curvature difference as the target point, although it may not fully meet the threshold conditions. The threshold can be set through experiments or experience to achieve better results. By directly screening points that meet the threshold conditions and giving priority to the nearest points, the computational complexity is significantly reduced. At the same time, the target point selection is achieved directly through simple screening, and the real-time performance is good.

[0098] S112, generating an initial tracking path: according to the current position of the ship and the selected target point, using B-spline control points to randomly generate an initial tracking path for the ship, including the following steps:

[0099] Assume the time domain is [ ts , t f ], the number of control points is n The B-spline curve trajectory is:

[0100] ;

[0101] In the formula, is the spline curve trajectory, t ∈[ t s , t f ]; are the control points of the B-spline curve, For the i indivual k B-spline basis function, basis function It is composed of the DeBoer-Cox recursion formula, expressed as:

[0102] ;

[0103] In the formula, U 1. U 2 is the basis function The coefficient of .

[0104] By introducing the B-spline curve into the ship path representation, a set of control point coordinates can be expressed as:

[0105] ;

[0106] The path expression generated by this group of control points is:

[0107] ;

[0108] In the formula, X ( t ) indicates that the point on the path is x The position change on the axis, Y ( t ) indicates that the point on the path is y Changes in position on the axis.

[0109] It should be noted that, in this embodiment, the shape and direction of the spline curve are adjusted by using the control points of the B-spline curve, and the first control point and the last control point are the starting point and the end point of the spline curve trajectory, respectively. , The B-spline interpolation optimization method can interpolate path points at a certain distance. After interpolation, the curve path is smoother and the curvature change is more continuous, which improves the path quality and characteristics, and then optimizes and smoothes the values ​​of lateral deviation and heading deviation, preventing sudden error changes that lead to control oscillation and other problems, making the transition of the ship's tracking path more natural and continuous.

[0110] S113, search for control points: encode the control points of the B-spline curve, search for the best control point configuration through the evolution process of the genetic algorithm, and achieve the optimal tracking path planning goal, including the following steps:

[0111] S1131, encode the control point and solve the control point ( , )coordinate:

[0112] ;

[0113] In the formula, ( , ) is the starting point of the path s and end point f Connection sf The coordinates of the equal division points, Starting point s and end point f The distance between rand is a random number and rand ∈[-0.5,0.5], For straight line sf The slope of .

[0114] Understandably, the starting point s and end point f The distance between the starting point s and end point f The longitude and latitude coordinates of are solved using the distance formula. Through the above representation, the control points of the B-spline curve are represented as individuals of the genetic algorithm, that is, a group of control points represents a feasible path, and a tracking path represents a genetic individual to complete the subsequent optimization.

[0115] S1132, establishing a fitness function: setting constraints according to actual requirements in the error correction process to establish a fitness function, including the following steps:

[0116] (1) Define path distance: Calculate the path distance based on the position of the points on the tracking path :

[0117] ;

[0118] In the formula, Indicates the path z Previousi discrete points, h is the number of discrete points.

[0119] This restriction is used to constrain the path distance so that the traced path minimizes the total distance.

[0120] (2) Defining path smoothness: Calculating path smoothness through curvature :

[0121] ;

[0122] In the formula, Indicates the path i The direction angle of the point, which passes through the i Point and i +1 points are calculated based on their relative positions. Represents the curvature change between three adjacent path points.

[0123] In order to make the path as smooth as possible, the curvature should be as small as possible.

[0124] (3) Define the path direction: Define the path direction to fit the objective function , which is used to measure the consistency between the direction of the path and the direction of the target path:

[0125] ;

[0126] In the formula, The first i The direction angle of the point, The first i The direction angle of the point.

[0127] By minimizing , the direction of the actual planned tracking path can be made to match the direction of the target path as much as possible, avoiding deviation from the expected direction of the target path.

[0128] (4) Establishing a fitness function: Assign weights to the established indicators and perform weighted summation to obtain the fitness function:

[0129] ;

[0130] In the formula, , , is the weight coefficient, which is used to adjust the importance of each indicator.

[0131] Since the tracking path planned in this embodiment is used to correct the errors that occur in the target path tracking of the ship, and the correction is performed in conjunction with the rudder system, in this process, since it is an error correction, the ship is within the correction error range of the target path, and there is no need to consider factors such as obstacles. Therefore, the problem actually solved in this process should meet the conditions: the restricted path is as short as possible, the restricted path is as smooth as possible, and the restricted path is as close to the target path as possible.

[0132] S1133, genetic iterative optimization: minimize the fitness function through iterative optimization and obtain the optimal path, including the following steps:

[0133] Randomly generate control point coordinates, determine parameters such as crossover probability, mutation probability, and number of iterations, and use the B-spline interpolation algorithm to generate a tracking path based on the coordinates of the starting and target points.

[0134] Calculate the fitness function value of each path and record it as the fitness of each individual; perform roulette selection operation according to the fitness value of each individual to obtain the offspring. At the same time, the individual with the highest fitness among the individuals does not participate in the selection and directly enters the offspring;

[0135] Perform crossover and mutation operations to obtain n +1 generation individual; calculation n +1 generation individual fitness; judgment n Is it greater than the set number of iterations? If it is greater than the maximum number of iterations, proceed to the next step, otherwise return to the above operation;

[0136] According to the obtained optimal individual, B-spline interpolation is performed to obtain the optimal tracking path.

[0137] It should be noted that the above method randomly generates tracking paths through B-spline control points, and calculates the comprehensive cost function of each path in the dynamic flow field taking into account path distance, path smoothness and fit to the target path. The genetic algorithm is used to iteratively search for the optimal control point combination to provide refined data for the subsequent rudder system to control the rudder angle, thereby ensuring that the ship accurately follows the target path while reducing rudder angle fluctuations and improving the stability of the ship's navigation.

[0138] S12, obtaining reference points: setting the reference point selection interval based on the control fineness requirement on the generated optimal tracking path, and selecting a group of rudder angle control reference points starting from the starting point of the tracking path.

[0139] It can be understood that an optimal tracking path for the ship to return to the target path is generated by the above method. By using the points on the tracking path as reference points, the error between the current position of the ship and the reference point can be solved and converted into a control error of the rudder angle. The path tracking of the ship can be completed under the segmented control adjustment. By planning the tracking path, the accuracy of path tracking and the stability of control are improved. At the same time, the position of the ship is corrected through the tracking path, so that after the ship returns to the target point, the speed and direction of the ship fit the target path, reducing the risk of yaw after reaching the target point.

[0140] S2. Obtaining rudder angle parameters: Obtain tracking deviation by setting parameters and detecting parameters, and correct the tracking deviation based on the machine learning model. Figure 4 As shown, the following steps are included:

[0141] S21, obtaining tracking deviation: according to the selected reference point as the setting parameter, the feedback coordinate point of the GPS and inertial combined navigation system is used as the detection parameter, and the tracking deviation between the setting parameter and the detection parameter is obtained, including the following steps:

[0142] Use GPS and inertial combined navigation system to obtain real-time feedback coordinates. The inertial navigation system estimates the current position by measuring acceleration and angular velocity, while GPS provides global positioning coordinates.

[0143] The sampling frequencies of GPS and inertial integrated navigation system data are usually different, so the data needs to be time-aligned to ensure that the timestamps of the two are consistent. Synchronization can be achieved through interpolation or data alignment;

[0144] Convert the geographic coordinates (such as latitude and longitude) provided by GPS into a coordinate system consistent with the inertial navigation system (such as the ENU coordinate system or the ECEF coordinate system). This ensures that the two data are compared in the same reference frame.

[0145] The GPS and inertial combined navigation system data are fused using algorithms such as Kalman filtering to obtain the optimal detection parameters and reduce errors or drifts when the GPS signal is lost.

[0146] The directional deviation between the set parameters and the detected parameters is calculated as the tracking deviation.

[0147] Among them, GPS and inertial combined navigation system, GPS has positioning function with high positioning accuracy, and the accuracy will not change with the change of positioning time, while the inertial navigation system can provide extremely accurate motion displacement. Through data fusion technology, the advantages of both can be combined, using the global positioning capability of GPS and the high-frequency data of the inertial combined navigation system to improve the performance and accuracy of the overall system.

[0148] In this embodiment, the tracking deviation of the setting parameters and the detection parameters is mainly used to calculate the deflection angle of the steering gear to achieve intelligent steering to complete the tracking path navigation.

[0149] S22, Tracking deviation correction: By collecting ship operating parameters, the machine learning model is used to correct the tracking deviation and compensate for the tracking deviation. Figure 5 As shown, the following steps are included:

[0150] S221. Establishing a machine learning model: Establishing a data set by collecting historical navigation data, constructing a machine learning model for training, and integrating the model into the ship's steering gear control system, including the following steps:

[0151] Establish a data set: Establish a data set through historical navigation data. The data types include but are not limited to current position, target position, rudder angle, speed, heading, wind speed, wind direction, and ocean current information; clean the collected raw data, remove outliers and noise data, process missing data, and perform normalization or standardization; extract features for prediction from the raw data through the correlation coefficient method, and combine the features through polynomial feature enhancement.

[0152] Construct BP neural network model: set the number of nodes in the input layer according to the selected prediction features, set the output layer to compensate for the deviation, set the number of layers and the number of nodes in each layer of the hidden layer according to the empirical formula, and select the activation function to complete the model construction;

[0153] The empirical formula is:

[0154] ;

[0155] In the formula, h , m , n are the number of neurons in the hidden layer, input layer, and output layer, respectively. a is a constant ranging from 1 to 10.

[0156] Model training: The collected data is divided into training set, validation set and test set in the ratio of 70%, 15% and 15%;

[0157] The data is passed from the input layer to the hidden layer, and then to the output layer through the activation function to calculate the output prediction value; the gradient of the loss function is calculated, and the error is back-propagated to each layer to adjust the weight and bias of each neuron; the weight of the model is updated through multiple iterations until the model converges or reaches the set stopping condition (such as the loss function no longer decreases significantly).

[0158] Use validation set data to evaluate the performance of the model. Common evaluation indicators include mean square error (MSE), mean absolute error (MAE), etc. Validation sets are used to avoid overfitting of the model.

[0159] Use the test set data to evaluate the model to ensure the model's generalization ability on unseen data.

[0160] S222, collecting ship working condition data: setting corresponding sensors according to the prediction features selected by model training, collecting working condition data in real time, and sending the collected data to the steering gear system;

[0161] S223, correcting tracking deviation: predicting the output rudder angle compensation deviation by inputting the acquired real-time operating condition data into the machine learning model.

[0162] In this process, the tracking deviation is first obtained through the deviation between the current position and the reference point. Based on the tracking deviation, the deviation angle that the ship needs to compensate under real-time conditions is added to make the actual trajectory of the ship during navigation on the tracking path more accurate, so that the ship can complete the regression of the target path in the planned tracking path.

[0163] S3, target path tracking: according to the corrected tracking deviation, it is converted into the steering angle of the steering gear, and the steering control parameter table between two reference points is established in sequence, and the steering gear is used as the actuator to use the PD controller for control output, including the following steps:

[0164] Proportional control (P): The proportional control part directly adjusts the output based on the current error.

[0165] ;

[0166] Among them, K p is the proportional gain, and e(t) is the tracking error at time t.

[0167] Derivative control (D): The derivative control part reacts to the rate of change of the error and is used to predict the trend of the error and suppress overshoot caused by rapid changes in the error.

[0168] ;

[0169] Among them, K d is the differential gain.

[0170] Control output of PD controller: The final control output u(t) is obtained by combining the proportional and differential control outputs:

[0171] ;

[0172] Transfer function of PD controller: In the continuous time domain, the transfer function of PD controller is expressed as the relationship from error to control output. In the Laplace transform domain, the transfer function G of PD controller is PD ( s ) is expressed as:

[0173] ;

[0174] in, s is the complex frequency variable in the Laplace transform.

[0175] According to the output formula of the PD controller, the control signal is calculated online and the control signal is limited to ensure that the output signal is within the physical range allowed by the servo.

[0176] The control signal calculated by the PD algorithm is converted into a control instruction of the servo (such as a PWM signal) to drive the servo to perform the corresponding angle rotation.

[0177] The parameters of the PD controller are adjusted using DDPG (Deep Deterministic Policy Gradient), including:

[0178] Update the learning parameters of the Actor-Critic network based on the reward function. The reward function is in the form of the sum of all input signals multiplied by the corresponding weights, and is used to evaluate the effect of the control system under the current PD parameters. The input value of the reward function is the tracking deviation obtained after the control system is running and the output result of the PD algorithm. After the reward function is calculated, its output value is compared with the value estimate output by the Critic network through a comparator, and the error value is sent to the Actor-Critic network for updating the learning parameters of the Actor-Critic network.

[0179] The PD parameters are trained based on the Actor-Critic network. The Actor-Critic network is a deep neural network architecture. The actor adjusts the PD parameters according to the deviation of the control system input. The evaluator estimates the value of the adjustment effect based on the deviation obtained by the control system operation after the PD parameters are adjusted and the adjusted PD parameters. When the training results meet the set requirements, the training is stopped and the obtained PD parameters are output as the optimal value.

[0180] It should be noted that the tracking path is divided into several segments by the number of selected reference points. The corresponding rudder angle parameters are obtained through deviation calculation and interference correction at the two end points of each segment. The rudder angle parameters are input into the steering gear system to complete the steering control of the tracking path navigation.

[0181] The present invention firstly establishes a tracking path for correcting the deviation of the ship's navigation trajectory, selects a reference point from the tracking path as the basis for calculating the deviation of the steering control, refines the stage of the steering control, and thus improves the stability of the ship's operation and the smoothness of the regulation; then, based on the calculated deviation, the rudder angle control under the current ship's operating conditions is predicted through a machine learning model to compensate for the deviation, reduce the influence of interference factors on the actual trajectory of the ship, and improve the accuracy of path tracking; finally, the deviation is input into the PD controller to complete the steering control during navigation on the tracking path, quickly respond to input changes, and reduce overshoot and oscillation.

[0182] In the process of tracking path planning, the fitness function is established by setting constraints according to actual needs, so as to generate an optimal path that is smooth, easy to control and adapts to the target path. By flexibly selecting reference points, the controllability of the steering adjustment is achieved and the control accuracy is improved. In the process of deviation compensation, the machine learning algorithm is used to compensate for the influence of the rudder angle caused by the interference factors of the operating conditions, so that the deviation obtained by the PD controller is more accurate, the system responds faster, and the efficiency of path tracking is improved. In the steering control process of the PD controller, the deep deterministic policy gradient algorithm is used to optimize the parameters of the PD controller to adapt to different operating conditions and enhance the robustness of the control system.

[0183] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technical personnel in this field can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any brief modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. An intelligent steering system based on machine learning, characterized in that: The following steps are involved: Select reference point: Plan the tracking path of the ship from the current position to the target path, and select the reference point from the tracking path as the reference point for error calculation of steering control; Obtaining rudder angle parameters: Obtaining tracking deviation by setting parameters and detecting parameters, and correcting the operating conditions of the tracking deviation based on the machine learning model; Target path tracking: According to the corrected tracking deviation, it is converted into the steering angle of the servo, and the steering control parameter table between two reference points is established in sequence. The servo is used as the actuator and the PD controller is used for control output; The step of selecting a reference point further comprises the following sub-steps: Generate tracking path: Generate tracking path according to the current position of the ship and the target path; Get reference points: Set the reference point selection interval based on the control precision requirement on the generated optimal tracking path, and select a set of rudder angle control reference points starting from the starting point of the tracking path; The step of generating a tracking path further comprises the following sub-steps: Select target point: set the thresholds of direction difference and curvature difference, sample a set of candidate target points from the target path, traverse the candidate target points, filter out the target points whose direction difference and curvature difference are within the threshold range, and select the point closest to the current position as the final target point; Generate initial tracking path: According to the current position of the ship and the selected target point, the initial tracking path of the ship is randomly generated using B-spline control points; Search control points: Encode the control points of the B-spline curve and search for the best control point configuration through the evolution process of the genetic algorithm to achieve the optimal tracking path planning goal; The generating of the initial tracking path further comprises the following sub-steps: Assume the time domain is [ ts , tf ], the number of control points is n The B-spline curve trajectory is: ； In the formula, Z ( t ) is the spline curve trajectory, t ∈[ ts , tf ]; are the control points of the B-spline curve, For the i indivual k B-spline basis function, basis function It is composed of the DeBoer-Cox recursion formula, expressed as: ； In the formula, U 1. U 2 is the basis function The coefficient of By introducing the B-spline curve into the ship path representation, a set of control point coordinates can be expressed as: ； The path expression generated by this group of control points is: ； In the formula, X ( t ) indicates that the point on the path is x The position change on the axis, Y ( t ) indicates that the point on the path is y Changes in position on the axis.

2. The intelligent steering system based on machine learning according to claim 1, characterized in that: The search control point further comprises the following sub-steps: Encode the control points and solve the control points ( , )coordinate: ； In the formula, ( , ) is the starting point of the path s and end point f Connection sf The coordinates of the equal division points, d sf Starting point s and end point f The distance between rand is a random number and rand ∈[-0.5,0.5], k sf For straight line sf The slope of Establishing fitness function: Setting constraints according to actual requirements in the error correction process to establish the fitness function; Genetic iterative optimization: Minimize the fitness function through iterative optimization and obtain the optimal path.

3. The intelligent steering system based on machine learning according to claim 2, characterized in that: The establishment of the fitness function further comprises the following sub-steps: Define path distance: Calculate the path distance based on the position of the points on the trace path Jd : ； In the formula, Indicates the path z Previous i discrete points, h is the number of discrete points; Defining path smoothness: Calculating path smoothness by curvature : ； In the formula, Indicates the path i The direction angle of the point, which passes through the i Point and i +1 points are calculated based on their relative positions. Indicates the curvature change between three adjacent path points; Define path direction: Define the path direction to fit the objective function , which is used to measure the consistency between the direction of the path and the direction of the target path: ； In the formula, The first i The direction angle of the point, The first i The direction angle of the point; Establish a fitness function: assign weights to the established indicators and perform weighted summation to obtain the fitness function: ； In the formula, , , is the weight coefficient, which is used to adjust the importance of each indicator.

4. The intelligent steering system based on machine learning according to claim 3, characterized in that: The genetic iterative optimization further includes the following sub-steps: Randomly generate control point coordinates, determine the crossover probability, mutation probability and number of iterations, and use the B-spline interpolation algorithm to generate the tracking path based on the coordinates of the starting and target points. Calculate the fitness function value of each path and record it as the fitness of each individual; perform roulette selection operation according to the fitness value of each individual to obtain the offspring, and the individual with the highest fitness among the individuals does not participate in the selection and directly enters the offspring; Perform crossover and mutation operations to obtain n +1 generation individual; calculate n +1 generation individual fitness; judgment n Is it greater than the set number of iterations? If it is greater than the maximum number of iterations, proceed to the next step, otherwise return to the above operation; based on the obtained optimal individual, perform B-spline interpolation to obtain the optimal tracking path.

5. The intelligent steering system based on machine learning according to claim 1, characterized in that: The step of obtaining the rudder angle parameter further comprises the following sub-steps: Obtain tracking deviation: Based on the selected reference point as the setting parameter and the coordinate point feedback from the GPS and inertial integrated navigation system as the detection parameter, obtain the tracking deviation between the setting parameter and the detection parameter; Tracking deviation correction: By collecting ship operating parameters, a machine learning model is used to correct the operating interference of the tracking deviation and compensate for the tracking deviation.

6. The intelligent steering system based on machine learning according to claim 5, characterized in that: The obtaining of tracking deviation further includes the following sub-steps: Use GPS and inertial combined navigation system to obtain real-time feedback coordinates, time-align the data, and convert the geographic coordinates provided by GPS into a coordinate system consistent with the inertial navigation system; The Kalman filter algorithm is used to fuse the GPS and inertial combined navigation system data to obtain the optimal detection parameters, and the direction deviation between the set parameters and the detection parameters is calculated as the tracking deviation.

7. The intelligent steering system based on machine learning according to claim 6, characterized in that: The tracking deviation correction further includes the following sub-steps: Establishing a machine learning model: By collecting historical navigation data to establish a data set, constructing a machine learning model for training, and integrating the model into the ship's steering gear control system; Collect ship operating data: Set up corresponding sensors based on the prediction features selected by model training, collect operating data in real time, and send the collected data to the steering gear system; Correct tracking deviation: The real-time operating data acquired is input into the machine learning model to predict the output rudder angle compensation deviation.

8. The intelligent steering system based on machine learning according to claim 7, characterized in that: The establishment of the machine learning model further includes the following sub-steps: Establishing data set: Establishing data set through historical navigation data. The data types include current position, target position, rudder angle, speed, heading, wind speed, wind direction, and ocean current information. Clean the collected raw data, extract the features for prediction from the raw data by using the correlation coefficient method, and combine the features by using polynomial feature enhancement; Construct BP neural network model: set the number of nodes in the input layer according to the selected prediction features, set the output layer to compensate for the deviation, set the number of layers and the number of nodes in each layer of the hidden layer according to the empirical formula, and select the activation function to complete the model construction; Model training: The collected data is allocated as training set, validation set and test set in the ratio of 70%, 15% and 15% respectively; the data is passed to the hidden layer through the input layer, and then reaches the output layer through the activation function to calculate the output prediction value; the gradient of the loss function is calculated, and the error is back-propagated to each layer to adjust the weight and bias of each neuron; the weight of the model is updated through multiple iterations until the model converges or the set stop condition is reached; The performance of the model is evaluated using validation set data and test set data.

9. The intelligent steering system based on machine learning according to claim 1, characterized in that: The target path tracking further includes the following sub-steps: Establish the control output of the PD controller: Combine the proportional and differential control outputs to get the final control output u(t): ； Where, e(t) is the tracking error at time t, K p is the proportional gain, K d is the differential gain; Obtain the transfer function of the PD controller: In the continuous time domain, the transfer function of the PD controller is expressed as the relationship from error to control output. In the Laplace transform domain, the transfer function of the PD controller is GPD ( S ) is expressed as: ； in, S is the complex frequency variable in the Laplace transform; According to the output formula of the PD controller, the control signal is calculated online, the control signal is limited, and the control signal calculated by the PD algorithm is converted into a control instruction of the servo, which drives the servo to perform the corresponding angle rotation; For the parameters of the PD controller, a deep deterministic policy gradient algorithm is used for optimization: Based on the reward function, the learning parameters of the Actor-Critic network are updated, and the effect of the control system under the current PD parameters is evaluated by the reward function. The input value of the reward function is the tracking deviation obtained after the control system is running and the output result of the PD algorithm. After the reward function is calculated, its output value is compared with the value estimate output by the Critic network through a comparator, and the obtained error value is sent to the Actor-Critic network to update the learning parameters of the Actor-Critic network. Based on the Actor-Critic network training of PD parameters, the actor adjusts the PD parameters according to the deviation of the control system input, and the evaluator estimates the value of the adjustment effect based on the deviation obtained from the control system operation after the PD parameters are adjusted and the adjusted PD parameters. When the training results meet the set requirements, the training is stopped and the obtained PD parameters are output as the optimal value.

Images