Controller optimization method and device, water area movable equipment and storage medium

Abstract

The embodiment of the invention provides a controller optimization method and device, a water area mobile device and a storage medium. The method comprises the following steps: acquiring actual driving data of the water area mobile equipment; according to the actual driving data, training to obtain a probability dynamical model of the water area movable equipment; optimizing controller parameters of the controller by using a particle swarm algorithm and a probability dynamical model; controlling the operation of the water area movable equipment by using the current optimized controller parameters, and acquiring actual driving data generated by the water area movable equipment under the control of the optimized controller; when the optimization process of the controller meets a first preset condition, determining the current optimized controller parameter as the controller parameter of the target controller; and when the optimization process of the controller does not meet the first predetermined condition, returning to the step of training according to the actual driving data to obtain the probability dynamical model. According to the technical scheme provided by the embodiment of the invention, the labor cost can be reduced.

Description

Technical Field

[0001] The present application relates to the field of control technology, and in particular to a controller optimization method, device, water area mobile device and storage medium. Background Art

[0002] In intelligent ship motion control, the controller completes a series of autonomous driving tasks based on the ship's navigation data, such as heading keeping and path tracking. The control effect is directly related to the controller parameters, so controller parameter tuning is an indispensable process in industry.

[0003] However, in practical applications, this work usually requires engineers to make repeated adjustments through trial and error based on the theoretical knowledge and expert experience of the controller. This is a tedious task for engineers and requires a lot of time and effort. In addition, parameter adjustments based on human intuition may not produce optimal parameter values. Summary of the invention

[0004] In view of the above problems, the present application is proposed to provide a controller optimization method, device, water area mobile device and storage medium that solve the above problems or at least partially solve the above problems.

[0005] In a first aspect of the present application, a method for optimizing a controller is provided, wherein the controller is used to control an assisted driving function of a movable device in water areas, and the method comprises: Acquire actual driving data of the movable device in the water area, where the actual driving data is generated by the movable device in the water area moving under the control of the controller; According to the actual driving data, a probabilistic dynamics model of the movable device in the water area is obtained by training; Optimizing controller parameters of the controller using a particle swarm algorithm and the probabilistic dynamics model; Using the currently optimized controller parameters to control the operation of the mobile device in the water area, and obtaining actual driving data generated by the mobile device in the water area under the control of the optimized controller; When the optimization process of the controller satisfies a first predetermined condition, determining the currently optimized controller parameters as controller parameters of the target controller; When the optimization process of the controller does not satisfy the first predetermined condition, the process returns to the step of training and obtaining a probabilistic dynamic model of the movable device in the water area according to the actual driving data.

[0006] In a second aspect of the present application, an electronic device is provided. The electronic device comprises: a memory and a processor, wherein: The memory is used to store programs; The processor is coupled to the memory, and is used to execute the program stored in the memory to implement the method described above.

[0007] According to a third aspect of the present application, a movable device in water area is provided, which includes the above-mentioned electronic device.

[0008] According to a fourth aspect of the present application, a computer-readable storage medium storing a computer program is provided, wherein the computer program can implement any of the above-described methods when executed by a computer.

[0009] In the technical solution provided in the embodiment of the present application, only a small amount of actual driving data of the movable device in water area needs to be collected for training the probabilistic dynamics model. In the subsequent process of optimizing the controller parameters using the particle swarm algorithm, only the probabilistic dynamics model needs to be used to simulate the movement of the movable device in water area without interacting with the actual movable device in water area. It can be seen that the technical solution provided in the embodiment of the present application can effectively reduce the number of interactions with the actual movable device in water area, and can realize the rapid adjustment of the assisted driving function controller of the movable device in water area under the condition of very few interactions. It can be seen that the technical solution provided in the embodiment of the present application can not only reduce labor costs, but also quickly obtain controller parameters that meet the actual assisted driving function requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0011] Figure 1 A schematic diagram of a flow chart of an optimization method provided in one embodiment of the present application; Figure 2 A structural example diagram of a probabilistic dynamics model provided in one embodiment of the present application; Figure 3 A structural block diagram of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0012] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below according to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0013] In addition, some of the processes described in the specification, claims and the above-mentioned figures of the present application include multiple operations that appear in a specific order, and these operations may not be executed in the order in which they appear in this article or may be executed in parallel. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish between different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., do not represent the order of precedence, and do not limit the "first" and "second" to be different types.

[0014] Figure 1 A flow chart of a controller optimization method provided for an embodiment of the present application. The executor of the method may be an electronic device, which may be a central controller of a water thruster (a controller integrated into a water thruster) or a cloud server. The controller is used to control the assisted driving function of a movable device in water areas, that is, the controller is a controller that completes the assisted driving function, and the controller may be a proportional, integral and differential (PID) controller, a neural network controller, and the like. For a PID controller, the controller parameters are a proportional coefficient (Kp), an integral coefficient (Ki), and a differential coefficient (Kd). For a neural network controller, the controller parameters are weights and biases of the neural network, and the like. The assisted driving functions may include heading keeping, position keeping, speed keeping, path following, trajectory tracking, and the like.

[0015] like Figure 1 As shown, the optimization method of the controller may include the following steps: 101. Obtain actual driving data of the movable device in the water area.

[0016] Among them, movable devices in water areas can be ships, underwater robots, submarines, etc.

[0017] The actual driving data is generated by the movement of the movable device in the water area under the control of the controller.

[0018] In some embodiments, the actual driving data in step 101 may be generated by the movement of the movable device in the water area under the control of the initial controller. The controller parameters of the initial controller are initial controller parameters. That is, the actual driving data in step 101 is generated by the initial controller controlling the movement of the movable device in the water area according to the initial controller parameters. The initial controller parameters may be set based on actual experience or randomly initialized.

[0019] Exemplarily, the actual driving data includes historical motion state quantities and historical motion control quantities of the movable device for water areas at multiple historical moments, wherein the historical motion control quantity at any historical moment is determined by the controller based on the controller parameters it has at that historical moment and the historical motion state quantity of the movable device for water areas at that historical moment.

[0020] The motion control quantity refers to the parameter used to control the motion state of the movable device in water area, which is transmitted to the actuator through the controller to realize the control of the motion state of the movable device in water area. Taking the movable device in water area as a ship as an example, the actuator can be a propeller and a rudder, and the motion control quantity can include propeller power data, rudder angle data for controlling the rudder angle steering, or steering wheel angle data.

[0021] The motion state quantity may include at least one of the position and velocity of the movable device in the water area in the three degrees of freedom of sway, sway and yaw. The position and velocity of the movable device in the water area in the three degrees of freedom of sway, sway and yaw include: the position in sway, the linear velocity in sway, the position in sway, the linear velocity in sway, the position in yaw and the angular velocity in yaw. Exemplarily, the motion state quantity may be the motion state quantity in the global coordinate system (i.e., the earth coordinate system) or the local coordinate system of the movable device in the water area.

[0022] Different assisted driving functions focus on different driving data (i.e., data that needs to be collected). For example, the driving data that heading maintenance focuses on are mainly: angular velocity of yaw, position of yaw, linear velocity of sway, and linear velocity of pitch; the driving data that position maintenance focuses on are mainly: position of sway, linear velocity of sway, position of pitch, and linear velocity of pitch; the driving data that speed maintenance focuses on are mainly: linear velocity of pitch. Of course, in actual applications, in order to make the probabilistic neural network model more closely fit the real ship, the six types of data mentioned above can be used for each assisted driving function.

[0023] In practical applications, the actual driving data of the mobile device in the water area within a preset time period can be obtained. The length of the preset time period can be set according to actual needs, and the embodiment of the present application does not specifically limit this, for example: 20 seconds. In some embodiments, a data acquisition device can be provided on the mobile device in the water area, and the data acquisition device collects the actual driving data of the mobile device in the water area within the preset time period, and sends the collected data to the electronic device, so that the electronic device can execute Figure 1 The optimization method shown.

[0024] 102. According to the actual driving data, a probabilistic dynamic model of the movable device in the water area is obtained by training.

[0025] Among them, the probabilistic dynamics model of mobile equipment in waters is a mathematical framework used to describe and predict the motion behavior of mobile equipment in waters in uncertain environments. This type of model combines classical dynamics equations with probability theory to deal with the randomness and uncertainty factors inherent in the marine environment, such as the impact of waves, currents, wind, etc. on the motion of mobile equipment in waters. Moreover, as a prediction model built based on probability theory and mathematical statistics, the probabilistic dynamics model can also show good accuracy and reliability when faced with small samples or data containing random errors.

[0026] The probabilistic dynamics model of movable equipment in water areas trained based on actual driving data can realistically simulate the movement process of movable equipment in water areas to a certain extent.

[0027] In some embodiments, the probabilistic dynamics model is used to predict the predicted state change of the movable device in the water area, and the predicted state change is expressed in the form of a probability distribution. Exemplarily, the predicted state change obeys a normal distribution, and the probabilistic dynamics model outputs the variance and mean of the normal distribution obeyed by the predicted state change.

[0028] Exemplarily, the probabilistic dynamic model is used to output a predicted state change expressed in the form of a probability distribution based on the motion state quantity and motion control quantity of the movable device in the water area at the previous moment. In this way, based on the motion state quantity of the movable device in the water area at the previous moment and the predicted state change expressed in the form of a probability distribution output by the probabilistic dynamic model, the predicted motion state quantity of the movable device in the water area at the next moment expressed in the form of a probability distribution can be obtained. The probabilistic dynamic model outputs the variance and mean of the normal distribution obeyed by the predicted state change quantity. The motion state quantity of the movable device in the water area at the previous moment is added to the mean output by the probabilistic dynamic model to obtain the target mean. The predicted motion state quantity expressed in the form of a probability distribution is determined according to the target mean and the variance output by the probabilistic dynamic model. The predicted motion state quantity obeys the normal distribution determined by the target mean and the variance output by the probabilistic dynamic model.

[0029] In other embodiments, the probabilistic dynamics model is used to predict the predicted motion state quantity of the movable device in the water area, and the predicted motion state quantity is expressed in the form of a probability distribution. Exemplarily, the predicted state change quantity obeys a normal distribution. The probabilistic dynamics model outputs the variance and mean of the normal distribution obeyed by the predicted motion state quantity.

[0030] 103. Optimize controller parameters of the controller using a particle swarm algorithm and the probabilistic dynamics model.

[0031] In the particle swarm algorithm, the position of a particle is used to represent a controller parameter of a controller, which can be regarded as a candidate controller parameter. The fitness of the particle is determined based on the state difference between the predicted operating state and the reference motion state, and the predicted operating state is predicted using the probabilistic dynamics model.

[0032] The reference motion state can be understood as a desired motion state, and the reference motion state can be set according to the preset assisted driving function that the controller needs to implement. It should be noted that when the preset assisted driving function that the controller needs to implement is different, the reference motion state will also be different.

[0033] Among them, optimizing the controller parameters of the controller is equivalent to optimizing the controller.

[0034] 104. Use the currently optimized controller parameters to control the operation of the movable device in the water area, and obtain actual driving data generated by the movable device in the water area under the control of the optimized controller.

[0035] The currently optimized controller parameters refer to the controller parameters optimized in step 103 , and the controller parameters of the optimized controller are the currently optimized controller parameters, wherein the optimized controller can be considered as the controller parameters optimized in step 103 .

[0036] Taking the movable equipment in water area as a ship as an example, the optimized controller parameters can be fed back to the ship control module, and the control module uses the optimized controller to control the intelligent assisted driving function.

[0037] 105. When the optimization process of the controller satisfies a first predetermined condition, determine the currently optimized controller parameters as controller parameters of a target controller.

[0038] 106. When the optimization process of the controller does not satisfy the first predetermined condition, return to the step of training and obtaining a probabilistic dynamic model of the movable device in the water area according to the actual driving data.

[0039] When the optimization process of the controller does not meet the first predetermined condition, new actual driving data is determined based on the actual driving data generated by the movable device for water areas under the control of the optimized controller, and the probabilistic dynamic model of the movable device for water areas is trained and returned based on the new actual driving data.

[0040] When the optimization process of the controller does not meet the first predetermined condition, the i-th iteration is started. In the i-th iteration, the actual driving data generated by controlling the movement of the movable device in the water area using the controller obtained in the i-1-th iteration is obtained. Based on the actual driving data, the probabilistic dynamic model of the movable device in the water area is trained, and the controller parameters of the controller are optimized using the particle swarm algorithm and the probabilistic dynamic model. Wherein, i is an integer greater than or equal to 2. It should be noted that the actual driving data used in the first iteration can be generated by controlling the movable device in the water area using the initial controller.

[0041] In an optional manner, in the i-th iteration, not only the actual driving data generated by controlling the movement of the movable device in the water area by the controller obtained in the i-1th iteration can be used to train the model, but also the actual driving data used to train the model in the i-1th iteration can be used to train the model. Wherein, i is an integer greater than or equal to 2. In other words, therefore, the total data set used in the i-th iteration is the set of actual driving data collected during the i-th iteration. For example, 200 pairs of data are collected in the first iteration, and 200 pairs of data are collected in the second iteration. Then the total data set used in the second iteration is 200+200=400 pairs, and so on.

[0042] In some embodiments, the first predetermined condition may include: the number of iterations of the optimization process of the controller reaches a sixth preset number, and / or, in the seventh consecutive preset number of iterations of the optimization process of the controller, the difference in actual cumulative costs corresponding to the controller parameters obtained from any two iterations is less than a second preset value, and the actual cumulative cost corresponding to the controller parameters obtained from any one iteration is calculated based on actual driving data generated by controlling the movement of the movable device in the water area using the controller obtained in that iteration and a preset cost function.

[0043] Exemplarily, the preset cost function is: (1) In the formula, are the historical motion state and reference state at time t respectively, and L is a pre-specified weighting coefficient matrix that determines the cost corresponding to different state dimensions.

[0044] In practical applications, the actual driving data generated by controlling the movement of the movable equipment in the water area using the controller obtained in this optimization process includes the historical movement state at multiple historical moments, where the historical movement state is represented by the movement state quantity. For the historical movement state at each moment, the cost corresponding to the moment can be calculated according to the above formula (1), and the costs corresponding to multiple historical moments can be accumulated to obtain the actual accumulated cost corresponding to the controller parameters obtained in this optimization process.

[0045] The sixth preset number may be a preset maximum number of iterations, that is, the maximum number of executions of the optimization process. In practical applications, the maximum number of iterations may be set according to actual needs, and the present application embodiment does not specifically limit this. For example, the maximum number of iterations is 5.

[0046] Among them, the seventh preset number of times can be set according to actual needs, and the embodiment of the present application does not make specific limitations on this. For example: the seventh preset number of times is 2 times.

[0047] The sixth preset number of times may be greater than the seventh preset number of times.

[0048] Exemplarily, when the number of executions of the optimization process reaches a maximum number of iterations or the actual cumulative costs corresponding to the controller parameters obtained from multiple consecutive optimization processes tend to be stable, the iteration is stopped.

[0049] In some other embodiments, the first preset condition may include: the optimization process of the controller reaches a sixth preset number of times, and / or when the actual cumulative cost corresponding to the controller parameters obtained in a certain optimization process is less than a third preset value. Exemplarily, in the embodiment of the present application, when the number of executions of the optimization process reaches the maximum number of iterations or the actual cumulative cost corresponding to the controller parameters obtained in a certain optimization process is less than the third preset value, the iteration is stopped.

[0050] In the technical solution provided in the embodiment of the present application, only a small amount of actual driving data of the movable device in water area needs to be collected for training the probabilistic dynamics model. In the subsequent process of optimizing the controller parameters using the particle swarm algorithm, only the probabilistic dynamics model needs to be used to simulate the movement of the movable device in water area without interacting with the actual movable device in water area. It can be seen that the technical solution provided in the embodiment of the present application can effectively reduce the number of interactions with the actual movable device in water area, and can realize the rapid adjustment of the assisted driving function controller of the movable device in water area under the condition of very few interactions. It can be seen that the technical solution provided in the embodiment of the present application can not only reduce labor costs, but also quickly obtain controller parameters that meet the actual assisted driving function requirements.

[0051] In some embodiments, the actual driving data includes historical motion state quantities and historical motion control quantities of the movable device for water area at multiple historical moments. In the above step 102, "according to the actual driving data, the probabilistic dynamics model of the movable device for water area is trained" can be implemented by the following steps: 1021. According to the historical motion state quantity and the historical motion control quantity of the first historical moment among the multiple historical moments, the probabilistic dynamics model is used to obtain a first predicted state change quantity expressed in the form of probability distribution.

[0052] Exemplarily, the probabilistic dynamics model outputs the mean and variance of the normal distribution obeyed by the first predicted state change.

[0053] In some embodiments, the historical motion state quantity and the historical motion control quantity of the first historical moment among the multiple historical moments are input into the probabilistic dynamics model to obtain a first predicted state change quantity represented by a probability distribution form.

[0054] 1022. Calculate the state change difference between the first predicted state change amount and the expected state change amount by using a preset loss function.

[0055] The state change difference refers to the difference between the first predicted state change amount and the expected state change amount.

[0056] Among them, the expected state change is the state change between the historical motion state of the second historical moment among the multiple historical moments and the historical motion state of the first historical moment, the first historical moment and the second historical moment are any two adjacent moments among the multiple historical moments, and the first historical moment is earlier than the second historical moment.

[0057] Among them, the state change between the historical motion state at the second moment and the historical motion state at the first historical moment may include at least one of the position change in the three degrees of freedom of lateral swing, longitudinal swing and yaw, and the speed change in the three degrees of freedom of lateral swing, longitudinal swing and yaw.

[0058] In an optional implementation, the preset loss function is a negative log-likelihood function, and the function value of the preset loss function is a state change difference.

[0059] In practical applications, given a training data set of size N, denoted as ,in [ ] is the training input set, [ is the corresponding training target, ,in, is the position and velocity of the three degrees of freedom of sway, swing and pitch at time n, is the propeller power and rudder angle at time n, and the training target is the state change at adjacent moments ,in, They are respectively the position changes in the three degrees of freedom of lateral swing, longitudinal swing and yaw between time n+1 and time n, and the velocity changes in the three degrees of freedom of lateral swing, longitudinal swing and yaw between time n+1 and time n.

[0060] The probabilistic dynamics model is trained by minimizing the negative log-likelihood function: (2) In the formula, , The input is The mean and variance of the model predictions / outputs at .

[0061] Among them, the negative log-likelihood function incorporates the uncertainty of the output, which helps to improve the prediction accuracy of the model.

[0062] 1023. Optimize model parameters of the probabilistic dynamics model according to the state change difference.

[0063] According to the state change difference, the model parameters of the probabilistic dynamic model are optimized by using an optimizer, wherein the optimizer can optimize the model parameters of the probabilistic dynamic model based on an optimization algorithm, such as a gradient descent method, a stochastic gradient descent method, a momentum method, etc.

[0064] In some embodiments, Figure 2 As shown, the probabilistic dynamics model may include multiple sub-models 200 (B sub-models are shown in the figure). Exemplarily, the probabilistic dynamics model may be established using a probabilistic ensemble (PE) method, which includes a probabilistic neural network model and an idol ensemble.

[0065] In the above step 103, each sub-model is used to predict the state change amount, and the output of the probabilistic dynamics model is obtained based on the outputs of multiple sub-models. That is, the optimized probabilistic dynamics model is used to predict the state change amount between the second moment and the first moment based on the motion state amount and motion control amount at the first moment, wherein the motion state amount and motion control amount at the first moment are used as the input of each sub-model, and the state change amount is obtained based on the outputs of the multiple sub-models.

[0066] The mean and variance of the probability distribution of the probabilistic dynamics model output are: (3) (4) In the formula, are the mean and variance of the prediction / output of the b-th sub-model, are the mean and variance of the predictions / outputs of the probabilistic dynamics model, respectively.

[0067] Among them, the sub-model can be a probabilistic neural network.

[0068] Given the state and control variables at time t, the predicted state distribution at time t+1 can be expressed as: (5) in, (6) (7) In some embodiments, the initial model parameters of each sub-model are obtained by randomization; the training sample set of each sub-model is obtained by randomly sampling the actual driving data for a first preset number of times. For each sub-model, the training sample set corresponding to the sub-model includes a training sample, and the training sample includes: the historical motion state quantity and the historical motion control quantity at the first historical moment, and the state change quantity between the historical motion state quantity at the second historical moment and the historical motion state quantity at the first historical moment. In the training sample, the historical motion state quantity and the historical motion control quantity at the first historical moment are used as the input of the sub-model, and the state change quantity between the historical motion state quantity at the second historical moment and the historical motion state quantity at the first historical moment is used as the expected state change quantity (i.e., the training label) of the sub-model. In this way, during training, for each sub-model, the historical motion state quantity and the historical motion control quantity of the training sample at the first historical moment in the training sample set of the sub-model can be used as the input of the sub-model to obtain the first predicted state change quantity output by the sub-model, and then the preset loss function is used to calculate the state change difference between the first predicted state change quantity and the expected state change quantity in the training sample, and the model parameters of the sub-model are optimized based on the state change difference. In an embodiment of the present application, different sub-models correspond to different training sample sets (or training data sets), that is, different sub-models are trained using different training sample sets.

[0069] In other embodiments, during training, the historical motion state quantity and the historical motion control quantity at the first moment among multiple historical moments are used as inputs of each of the sub-models, and the first predicted state change quantity is obtained based on the outputs of the multiple sub-models. The first predicted state change quantity is obtained based on the outputs of the multiple sub-models, which can be specifically implemented by the above formulas (3) and (4), and then the state change difference between the first predicted state change quantity and the expected state change quantity is calculated using a preset loss function, and the model parameters of the multiple sub-models are optimized based on the state change difference.

[0070] Each time a training sample is obtained by random sampling, the first preset number of times can be designed according to actual needs, and the embodiment of the present application does not specifically limit this, for example: 200 times.

[0071] Optionally, in the i-th iteration of the optimization process of the controller, the actual driving data generated by the controller controlling the movement of the movable device in the water area obtained in the i-1th iteration is sampled a first preset number of times to obtain a newly added training sample; the newly added training sample and the training sample used in the i-1th iteration are used as the training samples required for the i-th iteration, and the probabilistic dynamic model of the movable device in the water area is obtained by training according to the training samples required for the i-th iteration. Wherein, i is an integer greater than or equal to 2.

[0072] Each sub-model may specifically be a bootstrap model, for example, a probabilistic neural network model.

[0073] The model parameters of each sub-model are randomly initialized (that is, the parameters of each probabilistic neural network model may be different), and each sub-model has its own independent training data set, which is obtained by sampling the total data set previously collected from the real environment N times (data is repeatable), where N is the size of the total data set. Specifically, assuming that 200 pairs of data are collected within 20 seconds, each pair of data includes the motion state quantity at time t as the model input and the change between the motion state quantity at time t+1 and the motion state quantity at time t as the training label, then each model will sample 200 times in these 200 pairs of data to obtain a training data set unique to the model, in which the sampled data may be the same data, that is, the data is repeatable. Using different training data sets for different sub-models helps capture cognitive uncertainty (currently there are only 200 pairs of data, the data samples are relatively small, and the model's lack of cognition of the real environment leads to cognitive uncertainty), making the model more consistent with the real ship. It should be pointed out that since the controller parameter optimization requires multiple iterations, the total data set used in the i-th iteration is the set of all actual driving data collected during the i-th iteration. For example, 200 pairs of data are collected in the first iteration, and another 200 pairs of data are collected at the beginning of the second iteration. Then the total data set used in the second iteration is 200+200=400 pairs, and so on.

[0074] In some embodiments, in the above step 103, “optimizing the controller parameters of the controller by using the particle swarm algorithm and the probabilistic dynamics model” can be implemented by the following steps: 1031. Based on the multiple particles in the particle swarm algorithm and the probabilistic dynamics model, predict the predicted motion state of the movable device in the water area.

[0075] Among the multiple particles, the position of each particle serves as a controller parameter of the controller. It can be considered that the position of each particle serves as a candidate controller parameter of the controller.

[0076] The velocity of each particle indicates the direction and speed of position update.

[0077] For each particle, a probabilistic dynamics model and controller parameters represented by the particle can be used to predict the predicted motion state of the movable device in the water area.

[0078] In an optional embodiment, the predicted motion state is characterized by a predicted motion state quantity. In the above step 1031, "predicting the predicted motion state of the movable device in the water area based on the multiple particles in the particle swarm algorithm and the probabilistic dynamics model" can be implemented by the following steps: S11. For each of the particles, calculate the motion control amount at the first moment based on the motion state amount at the first moment and the particle.

[0079] Among them, the motion state quantity at the first moment can be a randomly generated motion state quantity, or a historical motion state quantity of a movable device in water area at a certain historical moment (the motion state quantity is the motion state quantity actually collected), or a motion state quantity predicted by a probabilistic dynamics model.

[0080] S12. Input the motion state quantity and the motion control quantity at the first moment into the probabilistic dynamics model to obtain a second predicted state change quantity expressed in the form of probability distribution.

[0081] Exemplarily, the motion state quantity at the first moment and the motion control quantity at the first moment are input into the probabilistic dynamics model, and the probabilistic dynamics model outputs the mean and variance of the normal distribution obeyed by the second predicted state change quantity.

[0082] S13. Extract a predicted state change amount from the second predicted state change amount as a target predicted state change amount.

[0083] Sampling from the normal distribution obeyed by the second predicted state change amount to obtain a predicted state change amount, and using the predicted state change amount as the target predicted state change amount. Exemplarily, a Monte Carlo method can be used to sample from the normal distribution obeyed by the second predicted state change amount to obtain a predicted state change amount.

[0084] S14. Calculate the predicted motion state quantity at the second moment according to the motion state quantity at the first moment and the target predicted state change quantity.

[0085] When the target predicted state change and the motion state at the first moment are expressed based on the same coordinate system (eg, the global coordinate system), the motion state at the first moment can be added to the target predicted state change to obtain the predicted motion state at the second moment.

[0086] When the target predicted state change is expressed based on a first coordinate system (for example, the local coordinate system of a movable device in water areas) and the motion state at the first moment is expressed based on a second coordinate system (for example, the global coordinate system), the target predicted state change can be converted to the second coordinate system and then added to the motion state at the first moment to obtain the predicted motion state at the second moment.

[0087] 1032. Determine the fitness of the plurality of particles based on a state difference between the predicted motion state and a reference motion state.

[0088] For each particle, the fitness of the particle may be determined according to the state difference between the predicted motion state quantity calculated for the example and the reference motion state quantity.

[0089] 1033. When the fitness does not satisfy the second predetermined condition, update the positions and velocities of the multiple particles, and return to the step of predicting the motion state of the movable device in the water area based on the multiple particles in the particle swarm algorithm and the probabilistic dynamics model.

[0090] In some embodiments, the second predetermined condition includes at least one of the following: the number of update iterations of the fitness of the multiple particles reaches a fourth preset number; the fitness of any particle is updated for a fifth preset number of consecutive times, and the difference between the fitness obtained by any two update iterations is less than the first preset value.

[0091] The fourth preset number refers to the maximum number of iterations, which can be set according to actual needs and is not specifically limited in the present embodiment. It should be noted that in this embodiment, the maximum number of iterations may be different from the maximum number of iterations involved in the first predetermined condition in the above embodiment.

[0092] The size of the first preset value can be set according to actual needs, and the embodiment of the present application does not specifically limit this. The first preset value in the embodiment of the present application can be different from the second preset value in the above embodiment.

[0093] In the fifth preset number of consecutive update iterations of the fitness of any particle, the difference between the fitness obtained in any two update iterations is less than the first preset value, indicating that the fitness of any particle tends to be stable.

[0094] In the process of searching for the optimal solution, the particle records the personal best position and the global best position, and updates the position and velocity of the particle based on these two positions. Assume that the position and velocity of particle j are and .make (k) represents the best position of the particle after the kth iteration, and let represents the current global optimal position after the kth iteration. The position and velocity of the particle are updated according to the following formula: (8) (9) In the formula, represents the number of iterations, , is the learning factor, usually set to , and are two random numbers between 0 and 1, is the inertia factor, reflecting the ability of the particle to maintain its previous velocity. As an option, a linearly decreasing inertia factor is used, and its value at the kth iteration is determined by the following formula: (10) In the formula, and They represent the initial inertia factor and the inertia factor at the maximum number of iterations S respectively.

[0095] 1034. When the fitness satisfies the second predetermined condition, select a position corresponding to a particle with the smallest fitness from the multiple particles as the optimized controller parameter.

[0096] In some embodiments, the position of the particle with the smallest fitness in the last iteration process can be used as the optimized controller parameter.

[0097] In other embodiments, the position of the particle with the smallest fitness during multiple iterations of the fitness of multiple particles may be used as the optimized controller parameter.

[0098] In an optional implementation, in order to improve the calculation accuracy of the fitness of each particle, the step of "determining the fitness of the plurality of particles based on the state difference between the predicted motion state and the reference state" in the above step 1032 can be implemented by the following steps: S21. For each of the particles, use the predicted motion state quantity and the reference motion state quantity as inputs of a preset cost function to calculate the instantaneous cost at the first moment.

[0099] The instantaneous cost is used to measure the state difference between the predicted motion state quantity and the reference motion state quantity.

[0100] Exemplarily, the preset cost function is: (11) In the formula, are the predicted motion state and reference state at time t respectively, and L is a pre-specified weighting coefficient matrix that determines the cost corresponding to different state dimensions.

[0101] S22: Taking the second moment as the updated first moment, and taking the predicted motion state quantity as the updated motion state quantity at the first moment.

[0102] S23, returning to the step of calculating the motion control amount at the first moment based on the motion state amount at the first moment and the particle, until a second preset number of iterations is completed.

[0103] That is, after taking the second moment as the updated first moment and taking the predicted motion state quantity as the updated motion state quantity of the first moment, return to execute the step of calculating the motion control quantity of the first moment based on the motion state quantity of the first moment and the particle, until the second preset number of iterations is completed.

[0104] Through the second preset number of iterations, the predicted motion state quantities at multiple moments can be obtained, and the predicted motion state quantities at the multiple moments constitute a predicted / simulated motion trajectory.

[0105] S24: Calculate the cumulative cost based on all the instantaneous costs during the second preset number of iterations.

[0106] Exemplarily, the sum of all the instantaneous costs during the second preset number of iterations may be taken as the accumulated cost.

[0107] S25. Determine the fitness according to the accumulated cost.

[0108] In an optional implementation, the accumulated cost may be used as the fitness of the particle.

[0109] In practical applications, in order to further improve the calculation accuracy of fitness, multiple motion state quantity samples can be set for each particle, and the multiple motion state quantity samples are sampled from a normal distribution with a preset motion state quantity as the mean. Among them, the preset motion state quantity can be a randomly generated motion state quantity, or a historical motion state quantity of a movable device in water area at a certain historical moment (the motion state quantity is the actual collected motion state quantity). In the above step S24, "calculating the cumulative cost based on all the instantaneous costs in the second preset number of iterations" may include: S241. For each motion state quantity sample, calculate the cumulative cost corresponding to the motion state quantity sample based on all the instantaneous costs in the second preset number of iterations.

[0110] Among them, for each motion state quantity sample, the motion state quantity at the first moment in the first iteration process during the second preset number of iterations is the motion state quantity sample.

[0111] That is to say, for each motion state quantity sample, the second preset number of iterations is performed once.

[0112] In the above step S25, "determining the fitness according to the accumulated cost" can be implemented by the following steps: S251. Determine the fitness according to the accumulated costs corresponding to each of the plurality of motion state quantity samples.

[0113] The average value of the accumulated costs corresponding to multiple motion state samples can be used as the fitness.

[0114] (12) in, is the cumulative cost corresponding to the mth motion state quantity sample. Wherein, T is the second preset number of times, which can also be called the set prediction step number.

[0115] In the formula, represents the controller parameter to be updated. The smaller the cumulative cost, the closer the predicted running trajectory is to the reference trajectory, and thus the better the current controller parameters are. The reference trajectory includes the reference motion state quantities at the multiple moments.

[0116] Taking the movable equipment in the water area as a ship as an example, the technical solution provided by the embodiment of the present application realizes the automatic adjustment of the intelligent assisted driving function controller by using the historical driving data of the ship, minimizes the number of times the engineer interacts with the actual ship environment, and can complete the rapid adjustment of the intelligent assisted driving function controller for ships of different sizes and types under the condition of very few interactions. Specifically, this solution adopts a model-based reinforcement learning method, in which the historical driving data of the ship is first used to train the model so that the model is close to the real ship, so that the model is used to replace the real ship, and then the controller parameters are optimized by the model to obtain the controller parameters that meet the real ship. Compared with the method of the background technology, this solution only needs to collect a small amount of ship driving data, that is, the number of interactions with the actual ship environment is small, so that the rapid adjustment of the intelligent assisted driving function controller for ships of different sizes and types can be completed under the condition of very few interactions. In addition, this solution is to adjust the controller parameters after several iteration cycles in the initial operation stage of the ship, and the controller parameters do not need to be adjusted during the subsequent operation process.

[0117] Figure 3 FIG. 1 is a schematic diagram showing the structure of an electronic device provided by an embodiment of the present application. Figure 3 As shown, the electronic device includes a memory 1101 and a processor 1102. The memory 1101 can be configured to store various other data to support operations on the electronic device. Examples of these data include instructions for any application or method for operating on the electronic device. The memory 1101 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, SRAM), electrically erasable programmable read only memory (Electrically Erasable Programmable read only memory, EEPROM), erasable programmable read only memory (Electrical Programmable Read Only Memory, EPROM), programmable read only memory (Programmable Read Only Memory, PROM), read only memory (Read Only Memory, ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0118] The memory 1101 is used to store programs; The processor 1102 is coupled to the memory 1101 and is used to execute the program stored in the memory 1101 to implement the methods provided by the above-mentioned method embodiments.

[0119] Further, if Figure 3 As shown, the electronic device also includes: a communication component 1103, a display 1104, a power component 1105, an audio component 1106 and other components. Figure 3 Only some components are shown schematically, which does not mean that the electronic device only includes Figure 3 Components shown.

[0120] Accordingly, an embodiment of the present application further provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a computer, the steps or functions of the methods provided in the above-mentioned method embodiments can be implemented.

[0121] An embodiment of the present application also provides a movable device in water area, comprising the above-mentioned electronic device.

[0122] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0123] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM (Read Only Memory) / RAM (Random Access Memory), a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0124] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A controller optimization method, characterized in that: The controller is used to control the auxiliary driving function of a mobile device in water area, and the method includes: Acquire actual driving data of the movable device in the water area, where the actual driving data is generated by the movable device in the water area moving under the control of the controller; According to the actual driving data, a probabilistic dynamics model of the movable device in the water area is obtained by training; Optimizing controller parameters of the controller using a particle swarm algorithm and the probabilistic dynamics model; Using the currently optimized controller parameters to control the operation of the mobile device in the water area, and obtaining actual driving data generated by the mobile device in the water area under the control of the optimized controller; When the optimization process of the controller satisfies a first predetermined condition, determining the currently optimized controller parameters as controller parameters of the target controller; When the optimization process of the controller does not satisfy the first predetermined condition, the process returns to the step of training and obtaining a probabilistic dynamic model of the movable device in the water area according to the actual driving data.

2. The method according to claim 1, characterized in that The actual driving data includes historical motion state quantities and historical motion control quantities of the movable device in the water area at multiple historical moments; the probabilistic dynamics model of the movable device in the water area is trained based on the actual driving data, including: According to the historical motion state quantity and the historical motion control quantity of the first historical moment among the multiple historical moments, using the probabilistic dynamics model, a first predicted state change quantity represented in the form of probability distribution is obtained; Using a preset loss function, calculating a state change difference between the first predicted state change amount and an expected state change amount, wherein the expected state change amount is a state change amount between a historical motion state amount at a second historical moment among the multiple historical moments and a historical motion state amount at the first historical moment, wherein the first historical moment and the second historical moment are any two adjacent moments among the multiple historical moments, and the first historical moment is earlier than the second historical moment; The model parameters of the probabilistic dynamics model are optimized according to the state change difference.

3. The method according to claim 2, characterized in that The probabilistic dynamics model includes a plurality of sub-models, and the initial model parameters of each sub-model are obtained by randomization; The training sample set of each sub-model is obtained by randomly sampling the actual driving data a first preset number of times, and the training sample sets of different sub-models are different. Each sub-model is trained based on the training sample set of the sub-model.

4. The method according to any one of claims 1 to 3, characterized in that The method of optimizing the controller parameters of the controller by using the particle swarm algorithm and the probabilistic dynamics model includes: Based on the plurality of particles in the particle swarm algorithm and the probabilistic dynamics model, predicting the predicted motion state of the movable device in the water area, wherein the position of each particle in the plurality of particles is used as a candidate controller parameter of the controller; determining the fitness of the plurality of particles based on a state difference between the predicted motion state and a reference motion state; When the fitness does not satisfy the second predetermined condition, updating the positions and velocities of the plurality of particles, and returning to the step of predicting the motion state of the movable device in the water area based on the plurality of particles in the particle swarm algorithm and the probabilistic dynamics model; When the fitness satisfies the second predetermined condition, a position corresponding to a particle with the smallest fitness is selected from the multiple particles as the optimized controller parameter.

5. The method according to claim 4, characterized in that The predicted motion state is characterized by a predicted motion state quantity; the predicted motion state of the movable device in the water area is predicted based on the plurality of particles in the particle swarm algorithm and the probabilistic dynamics model, including: For each of the particles, calculating a motion control amount at the first moment based on the motion state amount at the first moment and the particle; Inputting the motion state quantity and the motion control quantity at the first moment into the probabilistic dynamics model to obtain a second predicted state change quantity represented in the form of probability distribution; Extracting a predicted state change amount from the second predicted state change amount as a target predicted state change amount; The predicted motion state quantity at the second moment is calculated based on the motion state quantity at the first moment and the target predicted state change quantity.

6. The method according to claim 5, characterized in that The reference motion state is represented by a reference motion state quantity, and determining the fitness of the plurality of particles based on the state difference between the predicted motion state and the reference state includes: For each of the particles, the predicted motion state quantity and the reference motion state quantity are used as inputs of a preset cost function to calculate an instantaneous cost at the first moment, wherein the instantaneous cost is used to measure the state difference between the predicted motion state quantity and the reference motion state quantity; Taking the second moment as the updated first moment, and taking the predicted motion state quantity as the updated motion state quantity at the first moment; Returning to the step of calculating the motion control amount at the first moment based on the motion state amount at the first moment and the particle, until a second preset number of iterations is completed; Calculate the cumulative cost based on all the instantaneous costs during the second preset number of iterations; The fitness is determined according to the accumulated cost.

7. The method according to claim 6, characterized in that Each of the particles corresponds to a plurality of motion state quantity samples, and the plurality of motion state quantity samples are sampled from a normal distribution with a preset motion state quantity as a mean; Calculating the cumulative cost based on all the instantaneous costs during the second preset number of iterations includes: For each motion state quantity sample, the cumulative cost corresponding to the motion state quantity sample is calculated based on all the instantaneous costs in the second preset number of iterations, and for each motion state quantity sample, the motion state quantity at the first moment in the first iteration in the second preset number of iterations is the motion state quantity sample; Determining the fitness according to the accumulated cost includes: The fitness is determined according to the accumulated costs corresponding to each of the plurality of motion state quantity samples.

8. The method according to claim 6, characterized in that The second predetermined condition includes at least one of the following: The number of update iterations of the fitness of the plurality of particles reaches a fourth preset number; In the fifth preset number of consecutive update iterations of the fitness of any particle, the difference between the fitness obtained by any two update iterations is less than the first preset value.

9. The method according to claim 6, characterized in that The first predetermined condition includes at least one of the following: The number of iterations of the optimization process of the controller reaches a sixth preset number; In the seventh consecutive preset number of iterations in the optimization process of the controller, the difference between the actual cumulative costs corresponding to the controller parameters obtained in any two iterations is less than the second preset value, and the actual cumulative cost corresponding to the controller parameters obtained in any one iteration is calculated based on the actual driving data generated by controlling the movement of the movable equipment in the water area using the controller obtained in this iteration and the preset cost function.

10. An electronic device, characterized in that: include: A memory and a processor, wherein The memory is used to store programs; The processor is coupled to the memory, and is used to execute the program stored in the memory to implement the controller optimization method according to any one of claims 1 to 9.

11. A movable device in water area, characterized in that: The electronic device comprising claim 10.

12. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a computer, the controller optimization method according to any one of claims 1 to 9 can be implemented.

Images