Unmanned aerial vehicle flight control method and system of nonlinear prediction model based on multi-constraint optimization

Abstract

The invention provides an unmanned aerial vehicle flight control method and system based on a multi-constraint optimization nonlinear prediction model, and the method is characterized in that the method comprises the steps: obtaining multi-mode unmanned aerial vehicle state data and multi-mode flight environment monitoring data which are collected in real time through a multi-mode sensor; acquiring control input including pitching, rolling and thrust motion for controlling the unmanned aerial vehicle; taking the multi-modal unmanned aerial vehicle state data, the multi-modal flight environment monitoring data and the control input as the input of a pre-trained nonlinear prediction model, outputting a yaw control quantity for adjusting the control input, and generating a control instruction for controlling the flight of the unmanned aerial vehicle by using the yaw control quantity and the control input; wherein the nonlinear prediction model takes multiple constraints including a dynamic model constraint, a reference control input constraint and a state constraint as a training target in a training process. According to the invention, high-precision stable control of the unmanned aerial vehicle can be realized.

Description

Technical Field

[0001] The present application belongs to the field of UAV flight control technology, and in particular, relates to a UAV flight control method and system based on a nonlinear prediction model of multi-constraint optimization. Background Art

[0002] Sichuan's unique geographical and climatic features make it a region prone to frequent geological disasters, known as the "geological disaster museum". Due to the large number of mountainous areas and dense vegetation, Sichuan's geological disasters are highly concealed, sudden and dynamic, which poses significant challenges to emergency repair work.

[0003] The transportation of emergency repair materials for power supply is crucial in disaster relief work. The mountainous terrain of Sichuan is complex, and the repair of power facilities in the mountainous areas faces road congestion. The last five kilometers of complex mountain roads restrict the transportation of disaster relief materials by land. Multi-rotor drones are suitable for the transportation of emergency repair materials for power supply in disaster areas due to their mobility, vertical take-off and landing, and low cost.

[0004] However, there is still much room for improvement in the flight control of UAVs under load conditions, especially the robust flight control under extreme conditions such as variable load, crosswind, and low battery.

[0005] In the prior art, most UAV flight controllers have relatively simple settings for environmental factors, and only consider the impact of environmental factors, without considering the impact of the hanging load on the UAV's posture.

[0006] Therefore, how to provide a UAV flight control method that comprehensively considers complex environmental factors and suspended loads is a technical problem that needs to be solved urgently. Summary of the invention

[0007] The present application provides a UAV flight control method based on a nonlinear prediction model of multi-constraint optimization, which is used for UAV flight control that comprehensively considers complex environmental factors and suspended loads.

[0008] In a first aspect, an embodiment of the present application provides a UAV flight control method based on a nonlinear prediction model of multi-constraint optimization, the method comprising: obtaining multimodal UAV state data and multimodal flight environment monitoring data collected in real time by a multimodal sensor; obtaining a control input comprising pitch, roll and thrust motion for controlling the UAV; using the multimodal UAV state data, multimodal flight environment monitoring data and control input as inputs of a pre-trained nonlinear prediction model, outputting a yaw control amount for adjusting the control input, and generating a control instruction for controlling the flight of the UAV using the yaw control amount and the control input; wherein the training set of the nonlinear prediction model comprises multimodal UAV state data, multimodal flight environment monitoring data, control input and annotated yaw control amount labels, and the nonlinear prediction model takes multiple constraints comprising dynamic model constraints, reference control input constraints and state constraints as training targets during the training process.

[0009] In some embodiments of the present invention, the multimodal UAV status data is collected by a gyroscope and an accelerometer, and the multimodal flight environment monitoring data is collected by GPS, a lidar, an infrared sensor, an ultrasonic sensor, a barometer, a magnetometer and a thermometer. The multimodal UAV status data includes the angular velocity, rotation direction, rotation angle, flight direction and flight acceleration of the UAV flight, and the multimodal flight environment monitoring data includes satellite positioning, air pressure, obstacles, magnetic field parameters and ambient temperature.

[0010] In some embodiments of the present invention, the control input is generated based on a PID controller, and the PID controller is further used to obtain a posture control instruction input by a user in real time, and to generate a control input after parsing the posture control instruction.

[0011] In some embodiments of the present invention, the dynamic model constraints include the load tension due to the suspension of the UAV; the reference control input constraints include thrust constraints; and the state constraints include the angular velocity constraints of the UAV flight, the flight acceleration constraints, and the energy consumption of the UAV.

[0012] In some embodiments of the present invention, the training of the nonlinear prediction model also includes using an adaptive neural network to adjust the yaw control amount during the online adaptive control stage so that the flight state of the drone based on the control instruction fits the pitch, roll and thrust motion state of the drone expected by the control input.

[0013] In some embodiments of the present invention, the cost function used to train the nonlinear prediction model is expressed as:

[0014] f total =λ1f s +λ2f c+λ3(f v +f a )+λ4f b +λ5f o +λ6f p ;

[0015] Among them, f c and f s Represent the collision cost and smoothness cost respectively, f a and f v Represent the acceleration cost and speed cost respectively, f b represents the energy consumption cost of the drone, f o and f p They represent the anti-collision cost between UAVs and the tension constraint cost of the payload respectively. λ1, λ2, λ3, λ4, λ5, and λ6 represent the weight ratios of smoothness, obstacle avoidance safety, dynamic feasibility, energy consumption, anti-collision safety between UAVs, and load tension constraint respectively.

[0016] In a second aspect, an embodiment of the present application provides an electronic device, including a processor, a memory, and a computer program / instructions stored in the memory, wherein the processor is used to execute the computer program / instructions, and when the computer program / instructions are executed, the electronic device implements the steps of the method described in the above embodiment.

[0017] In the third aspect, an embodiment of the present application provides a UAV flight control system based on a nonlinear prediction model of multi-constraint optimization, characterized in that the system includes: electronic equipment; multimodal UAV state acquisition equipment, including a gyroscope and an accelerometer, for collecting the angular velocity, rotation direction, rotation angle, flight direction and flight acceleration of the UAV flight; multimodal flight environment monitoring equipment, including GPS, lidar, infrared sensor, ultrasonic sensor, barometer, magnetometer and thermometer, for collecting satellite positioning, air pressure, obstacles, magnetic field parameters and ambient temperature.

[0018] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program / instruction stored thereon, which, when executed by a processor, implements the steps of the method described in the above embodiment.

[0019] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the method described in the above embodiment.

[0020] The UAV flight control method and system based on the nonlinear prediction model of multi-constraint optimization proposed in the present invention can add a variety of dynamic and kinematic constraints, comprehensively consider the external force interference and UAV load in complex environments, and thus adjust the flight control instructions of the UAV, so that the UAV flight can stably complete the transportation task in complex environments, overcome the serious interference of the geographical and weather conditions in complex mountainous areas to the stable flight of the UAV, utilize the universal approximation characteristics of neural networks, realize real-time and high-precision estimation of interference, embed the neural network method into the model predictive control algorithm, and realize high-precision and stable control of the UAV.

[0021] Additional advantages, purposes, and features of the present invention will be described in part in the following description, and will become apparent to those skilled in the art after studying the following, or may be learned from the practice of the present invention. The purposes and other advantages of the present invention can be specifically pointed out and obtained in the specification and the accompanying drawings.

[0022] Those skilled in the art will appreciate that the objectives and advantages that can be achieved with the present invention are not limited to the above specific description, and the above and other objectives that can be achieved by the present invention will be more clearly understood from the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The accompanying drawings are used to provide further understanding of the present disclosure and constitute a part of the specification. They are used to explain the present disclosure together with the following detailed description, but do not constitute a limitation of the present disclosure.

[0024] In the attached picture:

[0025] Figure 1 This is a flow chart of a UAV flight control method based on a nonlinear prediction model of multi-constraint optimization in one embodiment of the present invention.

[0026] Figure 2 This is a schematic diagram of the control principle of a drone in one embodiment of the present invention.

[0027] Figure 3 Schematic diagram of the control architecture of a drone in one embodiment of the present invention.

[0028] Figure 4 FIG. 4 is a schematic diagram of a neural network model structure in one embodiment of the present invention.

[0029] Figure 5 It is a schematic diagram of the structure of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0030] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0031] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0032] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "comprises" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0033] In recent years, multi-rotor UAVs, as simple and flexible small-sized aerial devices, have incomparable advantages in disaster investigation and other emergency rescue operations in mountainous areas. They have been widely used in the investigation and design of power emergency repair, but have not yet been widely used in the field of emergency repair operations. The complex environment of mountainous areas poses many challenges to the flight control of single UAVs, and it is necessary to overcome a variety of influencing factors. The existence of random and unpredictable time-varying disturbances in the external environment poses a severe challenge to the problem of strong robust flight control of UAVs, requiring the designed control system to be highly robust.

[0034] In order to overcome the problems existing in the prior art, the present invention proposes a UAV flight control method and system based on a nonlinear prediction model with multi-constraint optimization. The pre-trained nonlinear prediction model is used to overcome various influencing factors and the random and unpredictable time-varying disturbances in the external environment, thereby realizing strong robust flight control of the UAV.

[0035] Figure 1 This is a flow chart of a UAV flight control method based on a nonlinear prediction model of multi-constraint optimization in one embodiment of the present invention.

[0036] like Figure 1 As shown, the embodiment of the present application provides a UAV flight control method based on a nonlinear prediction model of multi-constraint optimization, the method comprising:

[0037] Step S110: Acquire multimodal UAV status data and multimodal flight environment monitoring data collected in real time by multimodal sensors.

[0038] Step S120: Obtain control input including pitch, roll and thrust motions for controlling the drone.

[0039] Step S130: using the multimodal UAV state data, multimodal flight environment monitoring data and control input as inputs of a pre-trained nonlinear prediction model, outputting a yaw control amount for adjusting the control input, and using the yaw control amount and the control input to generate a control instruction for controlling the flight of the UAV.

[0040] Among them, the training set of the nonlinear prediction model includes multimodal UAV state data, multimodal flight environment monitoring data, control input and annotated yaw control quantity labels. During the training process, the nonlinear prediction model takes multiple constraints including dynamic model constraints, reference control input constraints and state constraints as training targets.

[0041] The UAV flight control method and system using the nonlinear prediction model based on multi-constraint optimization proposed in the present invention can add multiple dynamic and kinematic constraints, comprehensively consider external force interference and UAV load in complex environments, and thus adjust the flight control instructions of the UAV, so that the UAV flight can stably complete the transportation task in complex environments, overcome the serious interference of geographical and weather conditions in complex mountainous areas to the stable flight of the UAV, utilize the universal approximation characteristics of neural networks, realize real-time and high-precision estimation of interference, embed the neural network method into the model predictive control algorithm, and realize high-precision and stable control of the UAV.

[0042] In some embodiments of the present application, the multimodal UAV status data is collected through a gyroscope and an accelerometer, and the multimodal flight environment monitoring data is collected through GPS, a lidar, an infrared sensor, an ultrasonic sensor, a barometer, a magnetometer, and a thermometer. The multimodal UAV status data includes the angular velocity, rotation direction, rotation angle, flight direction, and flight acceleration of the UAV flight, and the multimodal flight environment monitoring data includes satellite positioning, air pressure, obstacles, magnetic field parameters, and ambient temperature.

[0043] By adopting this implementation, the state of the UAV can be evaluated using multimodal data, which can be used to train a model to achieve flight control of the UAV in a complex environment.

[0044] In some embodiments of the present application, the control input is generated based on a PID controller, and the PID controller is also used to obtain a posture control instruction input by a user in real time, and generate a control input after parsing the posture control instruction.

[0045] By adopting this implementation mode, a hierarchical control method is adopted, which can efficiently control the flight of the UAV on the one hand, and accurately control the flight status of the UAV on the other hand.

[0046] In some embodiments of the present application, the dynamic model constraint includes the load tension due to the suspension of the drone; the reference control input constraint includes the thrust constraint; the state constraint includes the angular velocity constraint of the drone flight, the flight acceleration constraint, and the energy consumption of the drone. The present invention is not limited to this, and the above constraint types are only examples.

[0047] By adopting this implementation, nonlinear prediction model training can be performed through multiple constraints, thereby achieving precise control of UAV flight.

[0048] In some embodiments of the present application, the training of the nonlinear prediction model also includes using an adaptive neural network to adjust the yaw control amount during the online adaptive control stage so that the flight state of the drone based on the control instruction fits the pitch, roll and thrust motion state of the drone expected by the control input.

[0049] By adopting this implementation, it is possible to combine offline training and online training to perform UAV flight control, so that the actual flight attitude of the UAV fits the expected flight state.

[0050] In some implementations of the present application, the cost function used to train the nonlinear prediction model is expressed as follows:

[0051] f total =λ1f s +λ2f c +λ3(f v +f a )+λ4f b +λ5f o +λ6f p ;

[0052] Among them, f c and f s Represent the collision cost and smoothness cost respectively, f a and f v Represent the acceleration cost and speed cost respectively, f b represents the energy consumption cost of the drone, f o and f p They represent the anti-collision cost between UAVs and the tension constraint cost of the payload respectively. λ1, λ2, λ3, λ4, λ5, and λ6 represent the weight ratios of smoothness, obstacle avoidance safety, dynamic feasibility, energy consumption, anti-collision safety between UAVs, and load tension constraint respectively.

[0053] By adopting this implementation mode, an example is given of the cost function used to train the nonlinear prediction model to implement the technical solution proposed in the present application.

[0054] Figure 2 This is a schematic diagram of the control principle of a drone in one embodiment of the present invention.

[0055] In the research of autonomous flight of UAVs, UAV perception refers to the ability of UAVs to recognize and understand their own status and surrounding environment during flight, such as considering the spatial limitations of the surrounding environment and effectively detecting the boundaries of obstacles. Autonomous perception is the first link in the perception and avoidance system of UAVs, which requires the detection and acquisition of obstacle information to provide support for the decision-making system. According to different working modes, the perception system can be divided into cooperative and non-cooperative types. For cooperative perception systems, it is required that all aerial vehicles carry cooperative sensors to complete target detection, such as the Traffic Alert and Collision Avoidance System (TCAS) and the Automatic Dependent Surveillance-Broadcast (ADS-B). For non-cooperative perception systems, they are often used in situations where information cannot be exchanged between aircraft and obstacles. At this time, autonomous detection is required through non-cooperative perception systems, which usually use radar, laser and photoelectric sensors. Among them, because the image obtained by a single photoelectric sensor loses depth information and cannot directly obtain the distance information of the target, it is necessary to use multiple sensors to use stereo vision to obtain accurate obstacle information, or to use them in conjunction with laser and radar sensors.

[0056] The motion stability of UAVs is affected by many factors such as sensor measurement accuracy and state estimation algorithm. When the state of the UAV system can be accurately obtained, its flight stability will depend on the performance of the designed controller. Due to the particularity of UAV motion, the UAV system needs to be decomposed into two subsystems before the controller is designed. The position controller and attitude controller are designed by control methods to achieve stable control of the UAV. The control algorithm can be roughly divided into linear control algorithm and nonlinear control algorithm, including PID control, model predictive control, anti-disturbance control, sliding mode control, intelligent control and robust control.

[0057] During the technical solution design stage of this application, key technology research includes research on the flight control characteristics of UAVs in complex environments and research on strong and robust flight control technology for single UAVs that are adapted to complex mountainous environments.

[0058] Research on the flight control characteristics of UAVs in complex environments. Meteorological factors such as wind speed, wind direction, atmospheric turbulence, temperature and humidity have a significant impact on the stability and control performance of UAVs. By combining simulation and experiment, the influence mechanism of meteorological factors on the flight control characteristics of UAVs can be analyzed, and the structure and control algorithm design of UAVs can be improved in reverse. The height changes of mountains, valleys and terrains in mountainous areas affect the flight altitude and path planning of UAVs, and also cause serious interference to GPS signals, resulting in unstable flight. In mountainous environments, dense fog or frequent rainfall will reduce the performance of visual sensors, and laser radars will be affected by the reflection of mountainous environments, resulting in large data errors. By analyzing the impact of different environmental factors on the flight characteristics of UAVs, the performance requirements of control algorithms for adapting to different environmental factors are inverted. Analyze the qualitative and quantitative influence of hanging loads on system performance. Under the same configuration conditions, the heavier the hanging load, the greater the lateral pull on the UAV, and the easier it is to cause the UAV to roll over and become unstable. Analyze the relationship between hanging load weight, swing characteristics and the stability domain of UAVs.

[0059] Research on the robust flight control technology of single UAVs adapted to the complex environment of mountainous areas. The environmental uncertainty in mountainous areas is large, and the UAV itself also has serious uncertainty due to the effect of the hanging load, which causes significant difficulties for the stable control of the UAV. In addition, the complex environment in mountainous areas leads to intermittent GPS signals, and the GPS navigation and positioning accuracy is poor. Optical sensors such as vision are affected by the weather. It is difficult for UAVs to navigate and locate in mountainous areas, which further increases the difficulty of stable control of UAVs. Therefore, the UAV flight control algorithm is required to be highly robust. This project is committed to studying the robust flight control method of single UAVs in complex mountainous environments, and realizing the stable flight of UAVs under the conditions of external wind field interference and positioning error. At the same time, an evaluation mechanism for the relationship between external interference, control strategy parameters and flight performance is established to provide a theoretical basis for UAVs to meet flight requirements. The operating environment of UAVs in mountainous areas is complex, and it is inevitable to encounter extreme flight conditions. Therefore, UAVs should have the ability to diagnose and isolate faults in real time and reconstruct the system according to the fault conditions. UAV fault diagnosis and self-repair reconstruction are the guarantee for its autonomous control, which can improve the survivability and flight safety of UAVs. Fault tolerance and reconfiguration control require research on: how to model major faults and structural damage in different situations without making the model too large; how to integrate different online fault detection and identification and adaptive reconfigurable control algorithms to cover different types of errors, including sensor failures, control effector failures, and structural damage to collaborative transportation systems; and how to change the control allocation algorithm when a failure occurs to ensure the safety of the drone in harsh environments and high-risk situations.

[0060] Figure 3 Schematic diagram of the control architecture of a drone in one embodiment of the present invention.

[0061] The scheme proposed in this application inherits the high robustness and accuracy of the model predictive control algorithm and the characteristics of easy engineering practice, and can reduce state jitter, have a certain generalization ability, and ensure the stable flight of the drone. Unlike the traditional PID-controlled quadcopter drone, the drone control architecture of this scheme is logically divided into two layers. The bottom layer is a fast and lightweight PID control that runs on the lower computer to achieve high-frequency attitude control. The upper layer adopts a model predictive control algorithm based on meta-learning. Model predictive control uses the pitch, roll and thrust of the drone as control inputs, and the yaw control quantity is calculated separately, using the yaw speed as input. Since the model between the control input quantity and the actual pitch, roll, and thrust control effects is unknown, this subject assumes it as a second-order system, and uses system identification to identify the parameters of the system. Using nonlinear model predictive control methods, constrained optimization problems are constructed, and dynamic model constraints, reference inputs, and state constraints are added. Due to the nonlinear characteristics of the dynamic model, the constrained optimization problem constructed is a nonlinear programming problem. The acados solver based on C language that directly generates a static link library can be used. It can not only directly solve the nonlinear model predictive control problem, but also has a greatly improved operating efficiency compared to the casadi solver commonly used in model predictive control.

[0062] For the load hanging under the drone, its impact on the drone can be regarded as a force disturbance in three-dimensional space. This force disturbance will cause a large deviation between the control effect produced by the control input calculated in the ideal disturbance-free dynamics model and the observed effect of the state estimator. This topic will use meta-learning to predict the disturbance of the load external force, and use an offline pre-trained model and a small online neural network to re-model the system. The controlled object can be regarded as the sum of two partial dynamics:

[0063]

[0064] where f F (x,u) is a model based on physical rules, f D (x,u,w) is the unmodeled dynamics including the hoisted load and tether. w is an unknown hidden state representing the potential time-varying underlying environmental conditions.

[0065] Figure 4 FIG. 4 is a schematic diagram of a neural network model structure in one embodiment of the present invention.

[0066] For the unmodeled dynamics of the hoisting load and the tether, they are linearized by a neural model based on the Koopman operator, which greatly improves the speed of solving the model predictive control problem. Based on the original dynamic model of the drone, the offline neural network also adds the force on the rope and other interferences, predicts the force and other interference on the rope through the neural network, and compares them with the magnitude of the force calculated by the dynamic equation. The DNN-based meta-learning algorithm is used to learn as an invariant representation of the unmodeled dynamics, and the error between the two is also added to the training for the second time, so as to better estimate and compensate for the force on the rope; in the online adaptive control stage, an adaptive neural network is used to adjust the linear coefficients and model predictive control parameters. The online adaptive control algorithm we designed is a composite adaptation that can achieve fast adaptation while maintaining the global exponential stability and robustness of the closed-loop system.

[0067] This control algorithm inherits the high robustness and accuracy of the traditional model predictive control algorithm, and is easy to implement in engineering, and has a certain degree of adaptability. When the drone carries the hoisting load with a rope, the tension of the rope acts on the drone. By designing a method that combines meta-learning to observe the interference of the hoisting load on the drone, the system is modeled by combining a neural network with a nominal dynamics model to offset the interference caused by the load traction. The drone will be able to handle the load more smoothly, and the observer and the robust controller are effectively combined to ensure that the drone can effectively and stably track the predetermined trajectory while transporting materials.

[0068] Furthermore, in some embodiments of the present application, an evaluation mechanism for the relationship between control strategy and flight performance is introduced. The evaluation mechanism between the control strategy and flight performance of the UAV is to quantitatively evaluate the impact of different control strategies on the flight performance of the UAV, so as to select and optimize the strategy that best suits the specific mission and environmental conditions. First, the performance indicators used to evaluate flight performance should be clearly defined, including flight efficiency, flight stability, mission completion time, energy consumption and other aspects.

[0069] In some embodiments of the present invention, a soft constraint method can be used to perform nonlinear optimization on the generated trajectory at the back end to achieve better results. On the basis of ensuring the smoothness, safety and dynamic feasibility of the quadcopter trajectory, thrust constraints are added to make the constraints more reasonable, reduce the energy consumption of the drone, and make the formation of the drone more compact, achieving better results and ensuring that the drone can perform real-time repair operations in complex environments. The entire cost function can be expressed by the following formula:

[0070] f total =λ1f s +λ2f c +λ3(fv +f a )+λ4f b +λ5f o +λ6f p ;

[0071] where f c and f s Represent the collision cost and smoothness cost respectively, f a and f v Represent the acceleration cost and speed cost respectively, f b represents the energy consumption cost of the drone, f o and f p They represent the anti-collision cost between UAVs and the tension constraint cost of the payload respectively. λ1, λ2, λ3, λ4, λ5, and λ6 represent the weight ratios of smoothness, obstacle avoidance safety, dynamic feasibility, energy consumption, anti-collision safety between UAVs, and load tension constraint respectively.

[0072] The UAV flight control method and system using the nonlinear prediction model based on multi-constraint optimization proposed in the present invention can add multiple dynamic and kinematic constraints, comprehensively consider external force interference and UAV load in complex environments, and thus adjust the flight control instructions of the UAV, so that the UAV flight can stably complete the transportation task in complex environments, overcome the serious interference of geographical and weather conditions in complex mountainous areas to the stable flight of the UAV, utilize the universal approximation characteristics of neural networks, realize real-time and high-precision estimation of interference, embed the neural network method into the model predictive control algorithm, and realize high-precision and stable control of the UAV.

[0073] In a second aspect, an embodiment of the present application provides an electronic device, which includes a computer device, the computer device including a processor and a memory, the memory storing computer instructions, the processor being used to execute the computer instructions stored in the memory, and when the computer instructions are executed by the processor, the device implements the steps implemented by the electronic device.

[0074] In the third aspect, an embodiment of the present application provides a UAV flight control system based on a nonlinear prediction model of multi-constraint optimization, characterized in that the system includes: the electronic device; a multimodal UAV state acquisition device, including a gyroscope and an accelerometer, for collecting the angular velocity, rotation direction, rotation angle, flight direction and flight acceleration of the UAV flight; a multimodal flight environment monitoring device, including GPS, a lidar, an infrared sensor, an ultrasonic sensor, a barometer, a magnetometer and a thermometer, for collecting satellite positioning, air pressure, obstacles, magnetic field parameters and ambient temperature.

[0075] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having computer program instructions stored thereon, and when the computer program instructions are executed by a processor, the above-mentioned offshore platform safety warning system is implemented.

[0076] Figure 5 It is a schematic diagram of the structure of an electronic device provided in one embodiment of the present application.

[0077] like Figure 5 As shown, an embodiment of the present application provides an electronic device, the device comprising: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the above-mentioned offshore platform safety warning system is implemented.

[0078] The electronic device may include a processor 1201 and a memory 1202 storing computer program instructions.

[0079] Specifically, the processor 1201 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0080] The memory 1202 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 1202 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 1202 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 1202 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 1202 is a non-volatile solid-state memory.

[0081] The memory may include read-only memory (ROM), random access memory (RAM), magnetic disk storage media devices, optical storage media devices, flash memory devices, electrical, optical or other physical / tangible memory storage devices. Thus, typically, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to an aspect of the present disclosure.

[0082] The processor 1201 implements any one of the battery thermal runaway parameter determination methods in the above embodiments by reading and executing computer program instructions stored in the memory 1202 .

[0083] In one example, the electronic device may further include a communication interface 1203 and a bus 1210. Figure 3 As shown, the processor 1201, the memory 1202, and the communication interface 1203 are connected via a bus 1210 and communicate with each other.

[0084] The communication interface 1203 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0085] Bus 1210 includes hardware, software or both, and the parts of electronic equipment are coupled to each other.For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front side bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 1210 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the application considers any suitable bus or interconnection.

[0086] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.

[0087] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0088] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.

[0089] Aspects of the present disclosure are described above with reference to the flowchart and / or block diagram of the method, device (system) and computer program product according to the embodiment of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the function / action specified in one or more boxes of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field programmable logic circuit. It can also be understood that each box in the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can also be implemented by dedicated hardware that performs a specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0090] The above is only a specific implementation of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.

Claims

1. A UAV flight control method based on a nonlinear prediction model with multi-constraint optimization, characterized in that: The method comprises: Obtain multimodal drone status data and multimodal flight environment monitoring data collected in real time by multimodal sensors; Get control inputs including pitch, roll, and thrust movements for controlling the drone; Using the multimodal UAV state data, the multimodal flight environment monitoring data and the control input as inputs of a pre-trained nonlinear prediction model, outputting a yaw control amount for adjusting the control input, and using the yaw control amount and the control input to generate a control instruction for controlling the flight of the UAV; Among them, the training set of the nonlinear prediction model includes multimodal UAV state data, multimodal flight environment monitoring data, control input and annotated yaw control quantity labels. During the training process, the nonlinear prediction model takes multiple constraints including dynamic model constraints, reference control input constraints and state constraints as training targets.

2. The method according to claim 1, characterized in that The multimodal UAV status data is collected through a gyroscope and an accelerometer, and the multimodal flight environment monitoring data is collected through GPS, a lidar, an infrared sensor, an ultrasonic sensor, a barometer, a magnetometer and a thermometer. The multimodal UAV status data includes the angular velocity, rotation direction, rotation angle, flight direction and flight acceleration of the UAV flight, and the multimodal flight environment monitoring data includes satellite positioning, air pressure, obstacles, magnetic field parameters and ambient temperature.

3. The method according to claim 1, characterized in that: The control input is generated based on a PID controller, and the PID controller is also used to obtain a posture control instruction input by a user in real time, and generate a control input after parsing the posture control instruction.

4. The method according to claim 1, characterized in that: The dynamic model constraints include load tension due to the suspension of the drone; The reference control input constraint includes a thrust constraint; The state constraints include the angular velocity constraints of the UAV flight, the flight acceleration constraints, and the energy consumption of the UAV.

5. The method according to claim 1, characterized in that The training of the nonlinear prediction model also includes using an adaptive neural network to adjust the yaw control amount during the online adaptive control stage so that the flight state of the drone based on the control instruction fits the pitch, roll and thrust motion states of the drone expected by the control input.

6. The method according to claim 1, characterized in that The cost function used to train the nonlinear prediction model is expressed as: f total =λ1f s +λ2f c +λ3(f v +f a )+λ4f b +λ5f o +λ6f p ; Among them, f c and f s Represent the collision cost and smoothness cost respectively, f a and f v Represent the acceleration cost and speed cost respectively, f b represents the energy consumption cost of the drone, f o and f p They represent the anti-collision cost between UAVs and the tension constraint cost of the payload respectively. λ1, λ2, λ3, λ4, λ5, and λ6 represent the weight ratios of smoothness, obstacle avoidance safety, dynamic feasibility, energy consumption, anti-collision safety between UAVs, and load tension constraint respectively.

7. An electronic device comprising a processor, a memory and a computer program / instruction stored in the memory, characterized in that: The processor is used to execute the computer program / instructions. When the computer program / instructions are executed, the electronic device implements the steps of the method according to any one of claims 1 to 7.

8. A UAV flight control system based on a nonlinear prediction model with multi-constraint optimization, characterized in that: The system comprises: The electronic device according to claim 7; Multimodal drone status acquisition equipment, including gyroscopes and accelerometers, used to collect the drone's angular velocity, rotation direction, rotation angle, flight direction, and flight acceleration; Multimodal flight environment monitoring equipment, including GPS, lidar, infrared sensor, ultrasonic sensor, barometer, magnetometer and thermometer, is used to collect satellite positioning, air pressure, obstacles, magnetic field parameters and ambient temperature.

9. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the steps of the method as claimed in any one of claims 1 to 7 are implemented.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images