Hovercraft hovering control method, device and equipment and storage medium

Abstract

The invention provides a hovercraft hovering control method, device and equipment and a storage medium, and the method comprises the steps: obtaining the current actual attribute data of a hovercraft, and setting the expected attribute data of the hovercraft based on the actual attribute data; determining at least one sample data of the hovercraft based on a first strategy neural network, the expected attribute data and the actual attribute data; inputting the at least one sample data into a first value network for training to obtain a first value function, and updating the first strategy neural network and the first value network based on the first value function to obtain a second value network and a second strategy neural network; and calculating a first control parameter based on the second value network and the second strategy neural network, and controlling the hovercraft to operate based on the first control parameter.

Description

Technical Field

[0001] The present disclosure relates to the field of data control, and in particular to a method, device, equipment and storage medium for controlling the lift of a hovercraft. Background Art

[0002] A fully-lifted hovercraft is an amphibious vessel that can achieve autonomous lift in various terrains such as land, shoals and water. The lift system of a hovercraft is a key element in achieving the unique amphibious characteristics of the hovercraft, and is also the premise and basis for the hovercraft to achieve autonomous navigation. However, the air cushion generated by the lift system is easily affected by the terrain environment. In severe sea conditions, the wave pump air movement and skirt leakage flow caused by waves and undulating water surfaces will significantly affect the cushion pressure. This effect causes the hovercraft to suffer violent heave and sink motions, increasing the instability of the ship. Summary of the invention

[0003] The present disclosure provides a method, device, equipment and storage medium for controlling the lift of a hovercraft, so as to at least solve the above technical problems existing in the prior art.

[0004] According to a first aspect of the present disclosure, a method for controlling the lift of a hovercraft is provided, comprising: Acquiring current actual attribute data of the hovercraft, and setting expected attribute data of the hovercraft based on the actual attribute data; Determine at least one sample data of the hovercraft based on the first strategy neural network, the expected attribute data and the actual attribute data; Inputting the at least one sample data into a first value network for training to obtain a first value function, and updating the first strategy neural network and the first value network based on the first value function to obtain a second value network and a second strategy neural network; A first control parameter is calculated based on the second value network and the second strategy neural network, and the operation of the hovercraft is controlled based on the first control parameter.

[0005] In one embodiment, the determining at least one sample data of the hovercraft based on the first strategy neural network, the expected attribute data and the actual attribute data comprises: Calculate error data of the hovercraft at a current moment based on the expected attribute data and the actual attribute data, and determine current initialization state data of the hovercraft based on the error data; Determining first motion data of the hovercraft based on the first strategy neural network, inputting the first motion data into a first control algorithm, and determining state data of the hovercraft at a next moment; A reward function of the hovercraft is constructed based on the error data, and first sample data is determined based on the reward function, the initialization state data, the next moment state data and the first action data.

[0006] In one possible implementation manner, the calculating error data of the hovercraft at a current moment based on the expected attribute data and the actual attribute data, and determining current initialization state data of the hovercraft based on the error data, comprises: Acquire actual attribute data at a previous moment, and calculate error change rate data of the hovercraft based on the actual attribute data at the previous moment and the current actual attribute data; calculating error vector data of the hovercraft based on the expected attribute data and the actual attribute data; The current initialization state data of the hovercraft is determined based on the error vector data and the error change rate data.

[0007] In one possible implementation, updating the first policy neural network and the first value network based on the first value function includes: Respectively obtaining initialization network parameters of the first strategy neural network and the first value network; Based on the first value function, the initialized network parameters are updated to obtain current network parameters corresponding to the first strategy neural network and the first value network; Determine value network parameters based on the initialization network parameters and the current network parameters, and update the first policy neural network and the first value network based on the value network parameters.

[0008] In one possible implementation manner, constructing a reward function of the hovercraft based on the error data comprises: determining an error reward function based on the error vector data, and determining an error change rate reward function based on the error change rate data; A first weight coefficient and a second weight coefficient are set, and a reward function of the hovercraft is constructed based on the first weight coefficient, the second weight coefficient, the error reward function, and the error change rate reward function.

[0009] According to a second aspect of the present disclosure, there is provided a hovercraft lift control device, the device comprising: a data acquisition unit, configured to acquire current actual attribute data of the hovercraft, and set expected attribute data of the hovercraft based on the actual attribute data; a sample data calculation unit, configured to determine at least one sample data of the hovercraft based on the first strategy neural network, the expected attribute data and the actual attribute data; A neural network updating unit, configured to input the at least one sample data into a first value network for training to obtain a first value function, and update the first strategy neural network and the first value network based on the first value function to obtain a second value network and a second strategy neural network; A control parameter calculation unit is used to calculate a first control parameter based on the second value network and the second strategy neural network, and control the operation of the hovercraft based on the first control parameter.

[0010] In one possible implementation manner, the sample data calculation unit is further used to calculate error data of the hovercraft at a current moment based on the expected attribute data and the actual attribute data, and determine current initialization state data of the hovercraft based on the error data; determine first action data of the hovercraft based on the first strategy neural network, input the first action data into a first control algorithm, and determine state data of the hovercraft at a next moment; construct a reward function of the hovercraft based on the error data, and determine first sample data based on the reward function, the initialization state data, the state data at the next moment, and the first action data; The neural network updating unit is also used to respectively obtain the initialization network parameters of the first strategy neural network and the first value network; update the initialization network parameters based on the first value function to obtain the current network parameters corresponding to the first strategy neural network and the first value network; determine the value network parameters based on the initialization network parameters and the current network parameters, and update the first strategy neural network and the first value network based on the value network parameters.

[0011] In one possible implementation manner, the sample data calculation unit is further used to obtain actual attribute data at a previous moment, calculate error change rate data of the hovercraft based on the actual attribute data at the previous moment and the current actual attribute data; calculate error vector data of the hovercraft based on the expected attribute data and the actual attribute data; determine current initialization state data of the hovercraft based on the error vector data and the error change rate data; An error reward function is determined based on the error vector data, and an error change rate reward function is determined based on the error change rate data; a first weight coefficient and a second weight coefficient are set, and a reward function of the hovercraft is constructed based on the first weight coefficient, the second weight coefficient, the error reward function and the error change rate reward function.

[0012] According to a third aspect of the present disclosure, there is provided an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the present disclosure.

[0013] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute the method described in the present disclosure.

[0014] The hovercraft lift control method, device, equipment and storage medium disclosed in the present invention construct sample data corresponding to the hovercraft through the actual attribute data and expected attribute data of the hovercraft, use the sample data as training samples of the first strategy neural network and the first value network, update the network parameters, and calculate the control parameters of the hovercraft based on the neural network after the network parameters are updated, which can improve the intelligence and autonomy of the hovercraft and enable the hovercraft to better adapt to various environments and mission requirements.

[0015] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present disclosure are shown in an exemplary and non-limiting manner, in which: In the drawings, the same or corresponding reference numerals represent the same or corresponding parts.

[0017] Figure 1 The schematic diagram shows the implementation process of a hovercraft lift control method according to an embodiment of the present disclosure. Figure 1 ; Figure 2 The schematic diagram shows the implementation process of a hovercraft lift control method according to an embodiment of the present disclosure. Figure 2 ; Figure 3 A schematic diagram of a hovercraft lift control device according to an embodiment of the present disclosure is shown; Figure 4 A schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0018] In order to make the purpose, features, and advantages of the present disclosure more obvious and easy to understand, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.

[0019] Figure 1 The schematic diagram shows the implementation process of a hovercraft lift control method according to an embodiment of the present disclosure. Figure 1 ,like Figure 1 As shown, a method for controlling the lift of a hovercraft according to an embodiment of the present disclosure includes the following steps: Step 101, obtaining current actual attribute data of the hovercraft, and setting expected attribute data of the hovercraft based on the actual attribute data.

[0020] In the disclosed embodiment, the attribute data of the hovercraft includes: heave height, heave speed and air chamber pressure. The corresponding actual attribute data is the attribute data of the hovercraft at the current moment, and the expected attribute data of the hovercraft is set according to the current actual attribute data and the target lift position.

[0021] Step 102: determining at least one sample data of the hovercraft based on the first strategy neural network, the expected attribute data and the actual attribute data.

[0022] In the disclosed embodiment, the error data of the hovercraft at the current moment is calculated based on the expected attribute data and the actual attribute data, and the current initialization state data of the hovercraft is determined based on the error data; wherein, by acquiring the actual attribute data at the previous moment, the error change rate data of the hovercraft is calculated based on the actual attribute data at the previous moment and the current actual attribute data; the error vector data of the hovercraft is calculated based on the expected attribute data and the actual attribute data; the current initialization state data of the hovercraft is determined based on the error vector data and the error change rate data, and the initialization state data consists of the heave height error, heave speed error and air chamber pressure error of the hovercraft at the current moment. The initialization state data is input into the first policy neural network, and the first action data of the hovercraft is determined based on the first policy neural network, wherein the first policy neural network is a policy neural network (Actor) of Deep Deterministic Policy Gradient (DDPG), and the first action data is input into the first control algorithm to determine the next moment state data of the hovercraft, wherein the first control algorithm is the current control algorithm of the DDPG neural network.

[0023] In the disclosed embodiment, a reward function of the hovercraft is constructed based on the error data, wherein an error reward function is determined based on the error vector data, and an error change rate reward function is determined based on the error change rate data; a first weight coefficient and a second weight coefficient are set, and a reward function of the hovercraft is constructed based on the first weight coefficient, the second weight coefficient, the error reward function, and the error change rate reward function. First sample data is determined based on the reward function, the initialization state data, the next moment state data, and the first action data. The first sample data is stored in an experience pool as a training data set, and at the same time, the experience pool is also used to store and replay historical experience data.

[0024] Step 103: input the at least one sample data into the first value network for training to obtain a first value function, and update the first strategy neural network and the first value network based on the first value function to obtain a second value network and a second strategy neural network.

[0025] In the disclosed embodiment, at least one sample data is selected from the experience pool and input into the first value network for model training, wherein the first value network is the value network (Critic) in the DDPG neural network, and the generated first value function is the value function of the state-action pair, which includes the state and action of the hovercraft at the current moment.

[0026] In the disclosed embodiment, the initialization network parameters of the first strategy neural network and the first value network are respectively obtained; the initialization network parameters are updated based on the first value function, specifically: the current target value is calculated according to the first value function, the initialization network parameters of the Critic and the initialization network parameters of the Actor are updated, and the current network parameters corresponding to the first strategy neural network and the first value network are obtained; the learning rate parameters are set, and the value network parameters are determined based on the learning rate parameters, the initialization network parameters and the current network parameters, and the first strategy neural network and the first value network are updated based on the value network parameters to obtain the updated second value network and second strategy neural network.

[0027] In the disclosed embodiment, the initial round iteration enters the loop, and the maximum number of rounds is set to M. The maximum number of steps per round is set to N. When the maximum number of steps N is reached, the round of iterative training ends. At this time, the number of iterations is increased by 1, and the operations of steps 101-103 are repeated, and so on, until the preset maximum number of iterations M is reached. After the maximum number of iterations is reached, the updated second value network and the second strategy neural network are obtained.

[0028] Step 104: Calculate a first control parameter based on the second value network and the second strategy neural network, and control the operation of the hovercraft based on the first control parameter.

[0029] In the disclosed embodiment, the process of determining the first action data of the hovercraft to the updated second value network and the second strategy neural network is set as one round, and the maximum number of rounds M is set. The maximum number of steps in each round is N. When the maximum number of steps N is reached, the round of iterative training ends. At this time, the number of iterations is increased by 1, and the above steps are repeated, and so on, until the preset maximum number of iterations M is reached.

[0030] In the disclosed embodiment, a linear sliding mode switching function is designed according to the error change rate data, the cushioning motion model of the hovercraft is substituted into the linear sliding mode switching function, and iterative training is performed in combination with the second value network and the second strategy neural network, specifically: the control rate parameter is calculated based on the maximum number of steps N in each round, and the set of control rate parameters obtained in each round is iteratively trained based on the maximum number of rounds M to obtain a first control rate parameter, that is, the control rate parameter after iterative optimization.

[0031] Figure 2 The schematic diagram shows the implementation process of a hovercraft lift control method according to an embodiment of the present disclosure. Figure 2 ,like Figure 2 As shown, a method for controlling the lift of a hovercraft according to an embodiment of the present disclosure includes the following steps: Step 201, obtaining attribute data of the hovercraft.

[0032] In the embodiment of the present disclosure, firstly, the Actor network and the Critic value network are constructed, and their parameters are respectively and Initialize. At the same time, initialize the experience pool for storing and replaying experience data. At the same time, the initial round iteration (Episode=1) enters the loop, and sets the maximum number of rounds to M. Set the maximum number of steps per round to N. The attribute data of the hovercraft includes heave height, heave speed, and air chamber pressure, which correspond to actual values ​​and expected values. According to the acquired attribute data, enter the expected attribute data, that is, enter the expected heave height , Heave speed , Air chamber pressure .

[0033] Step 202, constructing sample data of hovercraft.

[0034] In the disclosed embodiment, assuming that the pitch angle has a small variation range, the heave motion model of the hovercraft is simplified to:

[0035] in, represents the first derivative of the heave position, represents the heave speed, represents the mass of the hovercraft, represents the first derivative of the heave velocity, represents the longitudinal velocity, represents the lateral velocity, represents the rolling angular velocity, is the pitch angular velocity, Indicates air chamber The air chamber pressure, Indicates air chamber The air chamber area, represents the acceleration due to gravity, represents the air density, Relative wind speed, represents the aerodynamic (moment) coefficient, Represents the horizontal projection area of ​​the hull.

[0036] In the embodiment of the present disclosure, the error at this moment is calculated respectively Error change rate , get the initialization state , specifically: The error vector is expressed as:

[0037] in, represents the heave position error, represents the heave speed error, Indicates the chamber pressure error, Indicates the actual heave position, Indicates the expected heave position in meters. Indicates the actual heave speed, It represents the expected heave speed in m / s. P represents the air chamber pressure of the hovercraft. Indicates the desired air chamber pressure.

[0038] The state vector of the state space is:

[0039] in, represents the heave position error at time t, represents the heave speed error at time t, Represents the chamber pressure error at time t.

[0040] In the embodiment of the present disclosure, based on the Actor strategy network, the state vector is input into the Actor strategy network, and an action is selected according to the current strategy. , the action space expression is:

[0041] in, Parameters representing heave position error, Parameter representing the heave velocity error.

[0042] In the embodiment of the present disclosure, the action Input the neural network DDPG to get the state at the next moment , and calculate the instant reward function, where, in the control system of the hovercraft cushion lift, the design of the reward function should consider the error between the expected value and the actual value, as well as the impact of the error change rate on the control performance of the system, and reflect the stability and response speed of the system according to the error change rate. Smaller errors and stable error change rates should receive higher rewards to encourage the system to maintain the desired height and correct errors in time. The reward function is used to qualitatively measure the environment's evaluation of the performance of the control system to guide the decision-making of the intelligent agent to take actions under different environmental conditions. Therefore, its error change rate can be defined as:

[0043] in, represents the heave position error, represents the heave speed error, Represents the pressure error of the air chamber. Design reward function The expression is:

[0044]

[0045]

[0046] in, and is the weight coefficient. , They are the reward function of the error and the reward function of the error change rate respectively. and are the minimum error tolerance and the maximum error tolerance respectively. and When the hovercraft is at the desired lift altitude.

[0047] In the embodiment of the present disclosure, based on the above calculation, the corresponding sample data is obtained , store the sample data in the experience pool as a training data set.

[0048] Step 203: Neural network update.

[0049] In the embodiment of the present disclosure, N groups of sample data are extracted from the experience pool. Start training. With the current state and current status As the input of the Critic current network, the value function of the state-action pair is obtained . Calculate the current target value according to the formula and update the Critic network parameters Actor Network Parameters , specifically update the current network parameters and The expressions are:

[0050]

[0051] Where N is the sampling volume, is the sample weight, is the target action value, is the discount factor. Update the target parameters of the Critic network according to the formula Actor Network Target Parameters , specific updated value network parameters and policy network parameters The expression is:

[0052] in, is the learning rate.

[0053] Step 204, calculating control parameters, and controlling the hovercraft to rise based on the control parameters.

[0054] In the disclosed embodiment, since the air chamber pressure error is determined by the speed of the cushion lift fan, the controller is designed to mainly control the heave position error and the heave speed error. According to the error change rate in step 202, it can be obtained:

[0055] In the formula, the actual heave position is defined as , Heave speed The error between the expected value and the expected value. Taking the first-order derivative of the above formula, we can get:

[0056] Design the linear sliding mode switching function as:

[0057] In the formula, , , is the sliding mode parameter, , , . Taking the first-order derivative of the above formula, we can get:

[0058] Substituting the formula of the hovercraft heave motion model into the above formula, we can get:

[0059] According to the stability condition, , then the control parameters of the design are derived as:

[0060] In the formula, is the sliding mode parameter. is a symbolic function.

[0061] In the disclosed embodiment, when the maximum number of steps N is reached, the round of iterative training ends, and the sliding mode control parameters at this moment can be obtained according to the formula, otherwise, return to step 201, and the number of iterations Episode is increased by 1, and the operation is continued according to steps 201 to 203, and so on, until the preset maximum number of iterations M is reached, and the iteratively optimized sliding mode control parameters are obtained. Because the DDPG algorithm based on reinforcement learning needs to obtain a good control strategy through continuous learning and training. This learning process requires a large amount of sample data and iterations, so the time cost is relatively high. Therefore, when the hovercraft is autonomously lifted on land, the traditional control algorithm has a better control effect, does not require a large amount of training data and iterations, and can quickly achieve a stable control effect. For other environments, the DDPG algorithm is superior to other algorithms in terms of steady-state convergence time and steady-state error, and achieves the best control effect. Its performance is also significantly better than the fuzzy adaptive control method. In contrast, the traditional PID control method has the worst effect.

[0062] In the disclosed embodiment, the designed controller combines the learning ability of reinforcement learning and the stability of the sliding mode controller, and adjusts the parameters of the sliding mode controller in real time according to different terrain environments. This controller makes up for the shortcomings of the traditional controller's insufficient adaptive ability, and does not need to rely on manual tuning and parameter adjustment. Through the DDPG algorithm, the controller can autonomously learn and optimize parameters according to real-time feedback information and environmental conditions to achieve better control effects. Compared with traditional control methods, this algorithm has higher stability, enabling the controller to better adapt to complex and changing environments and achieve more precise control effects.

[0063] Figure 3 FIG. 1 shows a schematic diagram of a hovercraft lift control device according to an embodiment of the present disclosure. Figure 3 As shown, a hovercraft control device according to an embodiment of the present disclosure includes: A data acquisition unit 301 is used to acquire current actual attribute data of the hovercraft, and set expected attribute data of the hovercraft based on the actual attribute data; The sample data calculation unit 302 is used to determine at least one sample data of the hovercraft based on the first strategy neural network, the expected attribute data and the actual attribute data.

[0064] The sample data calculation unit 302 is also used to calculate the error data of the hovercraft at a current moment based on the expected attribute data and the actual attribute data, and determine the current initialization state data of the hovercraft based on the error data; determine the first action data of the hovercraft based on the first strategy neural network, input the first action data into the first control algorithm, and determine the state data of the hovercraft at the next moment; construct the reward function of the hovercraft based on the error data, and determine the first sample data based on the reward function, the initialization state data, the state data at the next moment and the first action data.

[0065] The sample data calculation unit 302 is also used to obtain the actual attribute data at the previous moment, calculate the error change rate data of the hovercraft based on the actual attribute data at the previous moment and the current actual attribute data; calculate the error vector data of the hovercraft based on the expected attribute data and the actual attribute data; determine the current initialization state data of the hovercraft based on the error vector data and the error change rate data; determine the error reward function based on the error vector data, and determine the error change rate reward function based on the error change rate data; set a first weight coefficient and a second weight coefficient, and construct the reward function of the hovercraft based on the first weight coefficient, the second weight coefficient, the error reward function and the error change rate reward function.

[0066] The neural network updating unit 303 is used to input the at least one sample data into the first value network for training to obtain a first value function, and update the first strategy neural network and the first value network based on the first value function to obtain a second value network and a second strategy neural network.

[0067] The neural network updating unit 303 is also used to respectively obtain the initialization network parameters of the first strategy neural network and the first value network; update the initialization network parameters based on the first value function to obtain the current network parameters corresponding to the first strategy neural network and the first value network; determine the value network parameters based on the initialization network parameters and the current network parameters, and update the first strategy neural network and the first value network based on the value network parameters.

[0068] The control parameter calculation unit 304 is used to calculate a first control parameter based on the second value network and the second strategy neural network, and control the operation of the hovercraft based on the first control parameter.

[0069] In an exemplary embodiment, the data acquisition unit 301, the sample data calculation unit 302, the neural network update unit 303 and the control parameter calculation unit 304 can be implemented by one or more central processing units (CPU, Central Processing Unit), graphics processing units (GPU, Graphics Processing Unit), application specific integrated circuits (ASIC, Application Specific Integrated Circuit), DSP, programmable logic devices (PLD, Programmable Logic Device), complex programmable logic devices (CPLD, Complex Programmable Logic Device), field programmable gate arrays (FPGA, Field-Programmable Gate Array), general processors, controllers, microcontrollers (MCU, Micro Controller Unit), microprocessors (Microprocessors), or other electronic components.

[0070] Regarding the device in the above embodiment, the specific manner in which each module and unit performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0071] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device and a readable storage medium.

[0072] Figure 4 A schematic block diagram of an example electronic device 800 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0073] like Figure 4As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 to a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0074] A number of components in the device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0075] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as a hovercraft cushion lift control method. For example, in some embodiments, a hovercraft cushion lift control method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of a hovercraft cushion lift control method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to execute a hovercraft lift control method in any other appropriate manner (for example, by means of firmware).

[0076] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0077] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0078] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0079] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0080] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0081] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0082] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0083] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of the present disclosure, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0084] The above is only a specific embodiment of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present disclosure, which should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. A method for controlling the lift of a hovercraft, characterized in that: The method comprises: Acquiring current actual attribute data of the hovercraft, and setting expected attribute data of the hovercraft based on the actual attribute data; Determine at least one sample data of the hovercraft based on the first strategy neural network, the expected attribute data and the actual attribute data; Inputting the at least one sample data into a first value network for training to obtain a first value function, and updating the first strategy neural network and the first value network based on the first value function to obtain a second value network and a second strategy neural network; A first control parameter is calculated based on the second value network and the second strategy neural network, and the operation of the hovercraft is controlled based on the first control parameter.

2. The method according to claim 1, characterized in that The determining of at least one sample data of the hovercraft based on the first strategy neural network, the expected attribute data and the actual attribute data comprises: Calculate error data of the hovercraft at a current moment based on the expected attribute data and the actual attribute data, and determine current initialization state data of the hovercraft based on the error data; Determining first motion data of the hovercraft based on the first strategy neural network, inputting the first motion data into a first control algorithm, and determining state data of the hovercraft at a next moment; A reward function of the hovercraft is constructed based on the error data, and first sample data is determined based on the reward function, the initialization state data, the next moment state data and the first action data.

3. The method according to claim 2, characterized in that The step of calculating error data of the hovercraft at a current moment based on the expected attribute data and the actual attribute data, and determining current initialization state data of the hovercraft based on the error data, comprises: Acquire actual attribute data at a previous moment, and calculate error change rate data of the hovercraft based on the actual attribute data at the previous moment and the current actual attribute data; calculating error vector data of the hovercraft based on the expected attribute data and the actual attribute data; The current initialization state data of the hovercraft is determined based on the error vector data and the error change rate data.

4. The method according to claim 2, characterized in that: The updating of the first policy neural network and the first value network based on the first value function includes: Respectively obtaining initialization network parameters of the first strategy neural network and the first value network; Based on the first value function, the initialized network parameters are updated to obtain current network parameters corresponding to the first strategy neural network and the first value network; Determine value network parameters based on the initialization network parameters and the current network parameters, and update the first policy neural network and the first value network based on the value network parameters.

5. The method according to claim 3, characterized in that: The step of constructing a reward function of the hovercraft based on the error data comprises: determining an error reward function based on the error vector data, and determining an error change rate reward function based on the error change rate data; A first weight coefficient and a second weight coefficient are set, and a reward function of the hovercraft is constructed based on the first weight coefficient, the second weight coefficient, the error reward function, and the error change rate reward function.

6. A hovercraft lift control device, characterized in that: The device comprises: a data acquisition unit, configured to acquire current actual attribute data of the hovercraft, and set expected attribute data of the hovercraft based on the actual attribute data; a sample data calculation unit, configured to determine at least one sample data of the hovercraft based on the first strategy neural network, the expected attribute data and the actual attribute data; A neural network updating unit, configured to input the at least one sample data into a first value network for training to obtain a first value function, and update the first strategy neural network and the first value network based on the first value function to obtain a second value network and a second strategy neural network; A control parameter calculation unit is used to calculate a first control parameter based on the second value network and the second strategy neural network, and control the operation of the hovercraft based on the first control parameter.

7. The device according to claim 6, characterized in that The sample data calculation unit is further used to calculate error data of the hovercraft at a current moment based on the expected attribute data and the actual attribute data, and determine current initialization state data of the hovercraft based on the error data; determine first action data of the hovercraft based on the first strategy neural network, input the first action data into a first control algorithm, and determine state data of the hovercraft at a next moment; Constructing a reward function of the hovercraft based on the error data, and determining first sample data based on the reward function, the initialization state data, the next moment state data and the first action data; The neural network updating unit is also used to respectively obtain the initialization network parameters of the first strategy neural network and the first value network; update the initialization network parameters based on the first value function to obtain the current network parameters corresponding to the first strategy neural network and the first value network; determine the value network parameters based on the initialization network parameters and the current network parameters, and update the first strategy neural network and the first value network based on the value network parameters.

8. The device according to claim 7, characterized in that The sample data calculation unit is further used to obtain the actual attribute data at the previous moment, and calculate the error change rate data of the hovercraft based on the actual attribute data at the previous moment and the current actual attribute data; calculating error vector data of the hovercraft based on the expected attribute data and the actual attribute data; Determine current initialization state data of the hovercraft based on the error vector data and the error change rate data; determining an error reward function based on the error vector data, and determining an error change rate reward function based on the error change rate data; A first weight coefficient and a second weight coefficient are set, and a reward function of the hovercraft is constructed based on the first weight coefficient, the second weight coefficient, the error reward function, and the error change rate reward function.

9. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 5.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to make a computer execute the method according to any one of claims 1-5.

Images