Unmanned aerial vehicle flight control method and system

Abstract

The embodiment of the invention provides an unmanned aerial vehicle flight control method and system, and relates to the technical field of data processing, the method is applied to an embedded calculation device on an unmanned aerial vehicle in an unmanned aerial vehicle flight control system, the unmanned aerial vehicle flight control system further comprises a control end, and the method comprises the steps that a first correction parameter at the current moment is acquired; the first correction parameter is sent to the control end, so that the control end determines a target correction parameter at the current moment according to the first correction parameter and a second correction parameter, and the second correction parameter is the target correction parameter at the previous moment; a flight control instruction is generated according to the target correction parameter at the current moment, the flight control instruction is sent to the unmanned aerial vehicle to control the unmanned aerial vehicle to fly, the first correction parameter and the target correction parameter are used for indicating the unmanned aerial vehicle to fly to a target position, and the target position is the position where the unmanned aerial vehicle should be located at the next moment. By adopting the technical scheme provided by the embodiment of the invention, the accuracy of flight control of the unmanned aerial vehicle can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a flight control method and system for an unmanned aerial vehicle. Background Art

[0002] Drone technology has been widely used and developed rapidly in many industries. For example, drones are used to inspect roads and deliver express in cities. However, these drone applications require drones to fly to the target point along a predetermined path, that is, they need to be controlled to fly along a predetermined route.

[0003] In the related technology, during the flight of a drone, when the drone deviates from a predetermined route, the embedded computing device in the drone flight control system calculates the yaw correction parameters of the drone and sends the yaw correction parameters to the control end. The control end receives the yaw correction parameters and generates a control instruction based on the yaw correction parameters and sends it to the drone to control the drone to correct its flight direction based on the yaw correction parameters.

[0004] However, when the flight speed of the UAV increases, the calculation and transmission process of the UAV's yaw correction parameters will be affected by factors such as environmental noise (such as instantaneous fluctuations in wind speed and air pressure) and visual noise (such as non-target objects being detected as targets), resulting in excessive correction of the UAV, causing the UAV to deviate further from the established route, making it difficult to accurately control the UAV to fly along the established route. Summary of the invention

[0005] The purpose of the embodiment of the present invention is to provide a UAV flight control method and system to achieve precise control of the UAV. The specific technical solution is as follows:

[0006] A first aspect of an embodiment of the present invention provides a method for controlling a drone flight, which is applied to an embedded computing device on a drone in a drone flight control system, wherein the drone flight control system further includes a control terminal, and the method includes:

[0007] Get the first correction parameter at the current moment;

[0008] The first correction parameter is sent to the control end so that the control end determines the target correction parameter at the current moment according to the first correction parameter and the second correction parameter, where the second correction parameter is the target correction parameter at the previous moment; a flight control instruction is generated according to the target correction parameter at the current moment, and the flight control instruction is sent to the UAV to control the UAV to fly, where the first correction parameter and the target correction parameter are used to instruct the UAV to fly to a target position, where the target position is the position where the UAV should be located at the next moment.

[0009] In one embodiment of the present invention, the obtaining of the first correction parameter at the current moment includes:

[0010] Acquire an image to be detected, where the image to be detected is an image collected in real time by the drone during flight;

[0011] Determine a target object from the image to be detected;

[0012] Determining a target position of the drone based on a position of the target object in the image to be detected;

[0013] A first correction parameter at a current moment is calculated according to the actual position of the drone and the target position of the drone.

[0014] In one embodiment of the present invention, determining the target object from the image to be detected includes:

[0015] According to a preset feature extraction model, the target object in the image to be detected is extracted; wherein the preset feature extraction model includes a backbone network, a neck network and an output network connected in sequence, the backbone network includes a plurality of standard convolution CBL units, the CBL unit is composed of a convolution layer, a batch normalization layer and an activation function, a CBL unit in the backbone network is directly connected to the output network, and the CBL unit is added with a residual connection, and other CBL units in the backbone network except the CBL unit directly connected to the output network are connected to the neck network, and the neck network includes a spatial pyramid pooling SPP unit and a cross-region segmentation CSP unit.

[0016] In one embodiment of the present invention, after determining the target position of the drone based on the position of the target object in the image to be detected, the method further includes:

[0017] Acquire position information of a first detection frame surrounding the target object;

[0018] estimating position information of a second detection frame in a next frame image of the image to be detected;

[0019] Matching the position information of the first detection frame with the position information of the second detection frame;

[0020] If the match is successful, determine whether the intersection over union (IOU) of the first detection frame and the second detection frame exceeds a preset threshold;

[0021] If it exceeds the preset threshold, it is determined that the object surrounded by the second detection frame is the updated target object, and the target position of the drone is updated according to the position information of the second detection frame.

[0022] In one embodiment of the present invention, the sending the first correction parameter to the control terminal includes:

[0023] sending the first correction parameter to the control end;

[0024] When no new first correction parameter is sent to the control end within a preset time period after the moment when the first correction parameter is sent, the control end is instructed to control the drone to perform a landing or return operation.

[0025] In one embodiment of the present invention, the method further comprises:

[0026] Acquire battery status information of the drone, wherein the battery status information includes battery charge status;

[0027] Detecting whether the battery status information is abnormal;

[0028] If not, instruct the control terminal to control the drone to continue flying;

[0029] If so, calculating whether the battery power status of the drone is sufficient to return to the take-off point;

[0030] When the battery power of the drone is sufficient to return to the take-off point, instruct the control terminal to control the drone to return from the current position to the take-off point;

[0031] When the battery power status of the drone is not enough to return to the take-off point, the control end is instructed to control the drone to perform a landing operation.

[0032] A second aspect of an embodiment of the present invention provides a drone flight control method, which is applied to a control end in a drone flight control system, wherein the drone flight control system also includes a drone, and the method includes:

[0033] Receiving a first correction parameter at a current moment sent by an embedded computing device on the drone;

[0034] According to the first correction parameter and the second correction parameter, the target correction parameter at the current moment is determined, and the second correction parameter is the target correction parameter at the previous moment; a flight control instruction is generated according to the target correction parameter at the current moment, and the flight control instruction is sent to the UAV to control the UAV to fly, and the first correction parameter and the target correction parameter are used to instruct the UAV to fly to a target position, and the target position is the position where the UAV should be located at the next moment.

[0035] A third aspect of an embodiment of the present invention provides a UAV flight control system, wherein the UAV flight control system includes a control terminal and a UAV.

[0036] The embedded computing device on the drone is used to obtain the first correction parameter at the current moment and send the first correction parameter to the control end;

[0037] The control end is used to receive the first correction parameter, determine the target correction parameter at the current moment according to the first correction parameter and the second correction parameter, where the second correction parameter is the target correction parameter at the previous moment; generate a flight control instruction according to the target correction parameter at the current moment, and send the flight control instruction to the drone to control the drone to fly, where the first correction parameter and the target correction parameter are used to instruct the drone to fly to a target position, where the target position is the position where the drone should be located at the next moment;

[0038] The UAV is used to receive the flight control instruction and fly based on the flight control instruction.

[0039] A fourth aspect of an embodiment of the present invention provides a UAV flight control device, which is applied to an embedded computing device in a UAV flight control system. The UAV flight control system also includes a UAV and a control terminal. The device includes:

[0040] A first correction parameter acquisition module, used to acquire the first correction parameter at the current moment;

[0041] The first correction parameter sending module is used to send the first correction parameter to the control end, so that the control end determines the target correction parameter at the current moment according to the first correction parameter and the second correction parameter, and the second correction parameter is the target correction parameter at the previous moment; a flight control instruction is generated according to the target correction parameter at the current moment, and the flight control instruction is sent to the UAV to control the UAV to fly, and the first correction parameter and the target correction parameter are used to instruct the UAV to fly to a target position, and the target position is the position where the UAV should be located at the next moment.

[0042] According to a fifth aspect of an embodiment of the present invention, there is provided a UAV flight control device, which is applied to a control end in a UAV flight control system, wherein the UAV flight control system further includes a UAV, and the device includes:

[0043] A first correction parameter receiving module, used for receiving a first correction parameter at a current moment sent by an embedded computing device on the drone;

[0044] The target correction parameter determination module is used to determine the target correction parameter at the current moment according to the first correction parameter and the second correction parameter, wherein the second correction parameter is the target correction parameter at the previous moment; generate a flight control instruction according to the target correction parameter at the current moment, and send the flight control instruction to the UAV to control the UAV to fly, wherein the first correction parameter and the target correction parameter are used to instruct the UAV to fly to a target position, wherein the target position is the position where the UAV should be located at the next moment.

[0045] According to a sixth aspect of an embodiment of the present invention, there is provided an electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0046] Memory, used to store computer programs;

[0047] The processor is used to implement any of the above-mentioned UAV flight control methods when executing the program stored in the memory.

[0048] According to a seventh aspect of an embodiment of the present invention, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, any of the above-mentioned UAV flight control methods is implemented.

[0049] An embodiment of the present invention also provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute any of the above-mentioned UAV flight control methods.

[0050] Beneficial effects of the embodiments of the present invention:

[0051] In the technical solution provided by the embodiment of the present invention, during the flight of the drone, the first correction parameter at the current moment and the correction parameter at the previous moment are calculated to obtain the target correction parameter at the current moment. The target correction parameter at the current moment is not only affected by the first correction parameter, but also regulated by the correction parameter at the previous moment. When the drone is affected by instantaneous noise factors such as environmental noise and visual noise, resulting in a large error in the first correction parameter, the target correction parameter at the current moment is effectively regulated by the accurate correction parameter that may not have been affected at the previous moment, thereby avoiding excessive correction amplitude caused by instantaneous noise, thereby accurately controlling the drone to fly according to the established route.

[0052] Of course, it is not necessary to achieve all of the advantages described above at the same time to implement any product or method of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0054] Figure 1 A schematic diagram of the structure of a UAV flight control system provided by an embodiment of the present invention;

[0055] Figure 2 A schematic diagram of a flow chart of a first unmanned aerial vehicle flight control method provided by an embodiment of the present invention;

[0056] Figure 3 A schematic diagram of a flow chart of a second UAV flight control method provided by an embodiment of the present invention;

[0057] Figure 4 A schematic diagram of calculating a first correction parameter under a straight line path provided by an embodiment of the present invention;

[0058] Figure 5 A schematic diagram of calculating a first correction parameter under a curved path provided by an embodiment of the present invention;

[0059] Figure 6 A schematic diagram of a flow chart of a third unmanned aerial vehicle flight control method provided by an embodiment of the present invention;

[0060] Figure 7 A schematic diagram of a fourth UAV flight control method provided by an embodiment of the present invention;

[0061] Figure 8 A schematic diagram of a fifth UAV flight control method provided by an embodiment of the present invention;

[0062] Fig. 9 A schematic diagram of a sixth method for controlling the flight of an unmanned aerial vehicle provided by an embodiment of the present invention;

[0063] Fig.10 A schematic structural diagram of a first UAV flight control device provided by an embodiment of the present invention;

[0064] Fig.11 A schematic diagram of the structure of a second UAV flight control device provided by an embodiment of the present invention;

[0065] Fig.12 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0066] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field based on the present invention belong to the scope of protection of the present invention.

[0067] Drone technology has been widely used and developed rapidly in many industries. For example, drones are used to inspect roads and deliver express in cities. However, these drone applications require drones to fly to the target point along a predetermined path, that is, they need to be controlled to fly along a predetermined route.

[0068] In the related technology, during the flight of a drone, when the drone deviates from a predetermined route, the visual end in the drone flight control system calculates the yaw correction parameters of the drone and sends the yaw correction parameters to the control end. The control end receives the yaw correction parameters and generates a control instruction based on the yaw correction parameters and sends it to the drone to control the drone to correct its flight direction based on the yaw correction parameters.

[0069] The UAV flight control system in the related art includes an embedded computing device, a control terminal and a UAV. The UAV flight control method is implemented by the following steps:

[0070] (1) Receive UAV body information through the embedded computing device deployed on the UAV.

[0071] (2) The control end sends real-time image acquisition instructions to the drone.

[0072] (3) The embedded computing device performs feature recognition detection and tracking on the images collected during the UAV flight, and calculates the UAV yaw correction parameters.

[0073] (4) The embedded computing device sends the correction parameters to the control end through socket communication.

[0074] (5) The control end receives the correction parameters, generates flight control instructions based on the correction parameters and sends them to the UAV.

[0075] (6) The UAV receives flight control instructions and achieves autonomous navigation flight along the recognized image features.

[0076] There are at least the following problems in the UAV flight control method in the related art:

[0077] (1) Due to the limited computing power of embedded computing devices, image feature extraction models with smaller network structures are often used, resulting in insufficient image feature recognition accuracy and failure to meet the demand for accurate recognition of image features. As a result, it is difficult for drones to fly based on given image features and it is difficult to achieve autonomous patrol flight of drones.

[0078] (2) When the existing target tracking algorithm is applied in the UAV flight control process, it is easy to misidentify the tracked target as another object in the image frame, resulting in target tracking errors and causing objects with the same characteristics to have identity (ID) changes in different image frames. This change will insert an erroneous ID between two normal IDs, which in turn will cause the transmission order of flight correction parameters to be disrupted, affecting the accuracy and continuity of the UAV flight control.

[0079] (3) The existing method for processing drone correction parameters is prone to over-correction of the drone, especially when the drone's flight speed increases. The calculation and transmission process of the drone's yaw correction parameters will be affected by factors such as environmental noise (such as instantaneous fluctuations in wind speed and air pressure) and visual noise (such as non-target objects being detected as targets), resulting in over-correction of the drone, causing the drone to easily deviate from the original established route.

[0080] (4) The embedded computing devices in the existing UAV flight control system only receive the UAV's Euler angle (pitch, roll, yaw) information and UAV speed information, which leads to a lack of real-time perception of the aircraft information, and an inability to respond to emergencies such as low battery in a timely manner, making it difficult to achieve effective connection between previous and subsequent flight missions.

[0081] In order to solve at least one of the above problems and achieve precise control of a UAV, an embodiment of the present invention provides a UAV flight control method and system.

[0082] See also Figure 1 , is a schematic diagram of the structure of a UAV flight control system provided in an embodiment of the present invention, wherein the UAV flight control system includes a UAV 101 and a control terminal 102.

[0083] The embedded computing device on the drone 101 is used to obtain the first correction parameter at the current moment and send the first correction parameter to the control terminal 102 .

[0084] The control end 102 is used to receive the first correction parameter, determine the target correction parameter at the current moment according to the first correction parameter and the second correction parameter, where the second correction parameter is the target correction parameter at the previous moment; generate a flight control instruction according to the target correction parameter at the current moment, and send the flight control instruction to the UAV to control the UAV to fly, where the first correction parameter and the target correction parameter are used to instruct the UAV to fly to a target position, where the target position is the position where the UAV should be located at the next moment;

[0085] The drone 101 is used to receive flight control instructions and fly based on the flight control instructions.

[0086] In the technical solution provided by the embodiment of the present invention, during the flight of the drone, the first correction parameter at the current moment and the correction parameter at the previous moment are calculated to obtain the target correction parameter at the current moment. The target correction parameter at the current moment is not only affected by the first correction parameter, but also regulated by the correction parameter at the previous moment. When the drone is affected by instantaneous noise factors such as environmental noise and visual noise, resulting in a large error in the first correction parameter, the target correction parameter at the current moment is effectively regulated by the accurate correction parameter that may not have been affected at the previous moment, thereby avoiding excessive correction amplitude caused by instantaneous noise, thereby accurately controlling the drone to fly according to the established route.

[0087] See also Figure 2 , is a schematic flow chart of a first UAV flight control method provided in an embodiment of the present invention. The method is applied to an embedded computing device in the above-mentioned UAV flight control system. The above-mentioned UAV flight control system also includes a control end. The above-mentioned method includes steps S201-S202.

[0088] S201, obtaining the first correction parameter at the current moment.

[0089] The first correction parameter is used to instruct the drone to fly to the target position, and the first correction parameter may include flight angle and flight distance as parameters. During the flight of the drone, the drone may deviate from the established route. At this time, the embedded computing device will obtain the correction parameter to instruct the drone to fly to the position it should be at the next moment (i.e., the target position).

[0090] Due to the influence of factors such as environmental noise and visual noise, the first correction parameter obtained by the embedded computing device at the current moment may not be accurate and cannot accurately instruct the UAV to fly towards the target position. Therefore, in an embodiment of the present invention, after the embedded computing device obtains the first correction parameter at the current moment and sends the first correction parameter to the control end, the control end does not directly generate a flight control instruction based on the first correction parameter, but generates the target correction parameter at the current moment based on the first correction parameter at the current moment and the correct correction parameter at the previous moment (the second correction parameter), and generates a flight control instruction based on the target correction parameter at the current moment.

[0091] In one embodiment of the present invention, a flight route map of a UAV may be pre-stored in an embedded computing device. The embedded computing device may obtain the current actual position of the UAV through a global positioning system (GPS), and determine the target position where the UAV should be located at the next moment according to the pre-stored flight route map of the UAV. Based on the current actual position of the UAV and the target position of the UAV, the flight angle and / or flight distance of the UAV is determined, that is, the first correction parameter at the current moment is determined.

[0092] In another embodiment of the present invention, the control end sends an image acquisition instruction to the drone, so that the drone acquires images in real time during flight, the embedded computing device acquires the current position of the drone, and determines the target position where the drone should be located at the next moment from the real-time image acquired by the drone, and determines the flight angle and / or flight distance of the drone based on the current actual position of the drone and the target position of the drone, that is, determines the first correction parameter at the current moment. The specific implementation process of this embodiment can be found in the following description.

[0093] The embodiment of the present invention does not specifically limit the method for obtaining the first correction parameter at the current moment.

[0094] S202, sending the first correction parameter to the control end so that the control end determines the target correction parameter at the current moment according to the first correction parameter and the second correction parameter; generating a flight control instruction according to the target correction parameter at the current moment, and sending the flight control instruction to the UAV to control the UAV to fly.

[0095] The second correction parameter is the target correction parameter of the previous moment, that is, the second correction parameter is obtained by integrating the first correction parameter of the previous moment and the correction parameter of the historical moment, and is regulated by the accurate correction parameter of the historical moment, so it is a more accurate correction parameter. And because the second correction parameter is calculated by the control end, the control end can store the second correction parameter for a period of time.

[0096] The above-mentioned target parameter at the current moment is obtained by the control end based on the first correction parameter and the second correction parameter, and the target correction parameter at the current moment is used to instruct the drone to fly to the target position. Since the target correction parameter at the current moment is obtained based on the fusion of the first correction parameter and the second correction parameter, if the drone deviates from the correct route at the current moment, and because of the influence of various noises due to the excessive flight speed, the first correction parameter at the current moment obtained by the embedded computing device is not accurate, the correct correction parameter at the previous moment can effectively suppress the erroneous first correction parameter, and the target correction parameter at the current moment obtained based on the first correction parameter and the second correction parameter can effectively reduce the propagation of errors, thereby controlling the drone to fly along the correct route.

[0097] In one embodiment of the present invention, the embedded computing device can send the first correction parameter to the control end based on the User Datagram Protocol (UDP). In the UAV flight control scenario with high real-time requirements, the low latency feature of the UDP protocol has significant advantages. The low latency feature enables the control end to receive and respond to the correction parameters more quickly, achieve more accurate flight control and real-time adjustment, especially in emergency situations or inspection tasks with rapid environmental changes, which can greatly improve the response speed and stability of the flight control system. The non-connectivity and fast transmission characteristics of the UDP protocol enable it to maintain a high transmission efficiency in an unreliable network environment. Even if data packets are lost occasionally, the UDP protocol will not affect the overall performance of the UAV flight control system, because the flight correction parameters are sent continuously, and subsequent data packets can quickly supplement the previous lost information to ensure that the control end can continue to obtain the latest correction instructions. By using the UDP protocol for direct transmission, the additional overhead caused by establishing connections and data confirmation can also be avoided, ensuring the timeliness of UAV flight control.

[0098] In one embodiment of the present invention, after the embedded computing device sends the first correction parameter to the control end, the time when the embedded computing device sends the first correction parameter can be recorded. If the embedded computing device does not send a new first correction parameter to the control end within a preset time after the time, it indicates that the flight mission of the drone has been completed and the drone has flown to the end of the predetermined route. Alternatively, if a problem occurs in the process of calculating the first correction parameter by the embedded computing device, the embedded computing device instructs the control end to control the drone to perform a landing or return operation. In this way, the flight mission failure or drone out-of-control problem caused by signal interruption or data loss is avoided. The stability and safety of the drone flight process can be guaranteed.

[0099] In one embodiment of the present invention, the control end receives the first correction parameter at the current moment, and obtains the target correction parameter at the previous moment, i.e., the second correction parameter, from its own storage space. The control end may perform a weighted operation on the first correction parameter and the second correction parameter to obtain the target correction parameter at the current moment. Specifically, the control end may perform a weighted operation on the first correction parameter and the second correction parameter based on an exponential moving average (EMA) algorithm, and the calculation formula thereof is as follows:

[0100] EMA t =α·Data t +(1-α) EMA t-1 (1)

[0101] Among them, Data t Indicates the first correction parameter at the current moment, EMA t-1 EMA of the flight correction parameters at the last moment

[0102] Filter value, i.e. the second correction parameter, EMA t It represents the filter value at the current moment, that is, the target correction parameter at the current moment, and α is the filter weight coefficient, which ranges from 0 to 1.

[0103] In an embodiment of the present invention, the control end can also dynamically adjust α according to the complexity of the UAV's flight route. In a high-noise environment, the weight of the historical moment correction parameter is increased, that is, the value of (1-α) is increased; in a low-noise environment, the weight of the historical moment correction parameter is reduced, that is, the value of (1-α) is reduced, thereby improving the flexibility of the UAV flight control.

[0104] In another embodiment of the present invention, the control end may perform an average operation on the first correction parameter and the second correction parameter to obtain the target correction parameter at the current moment.

[0105] The embodiment of the present invention does not specifically limit the manner in which the control end determines the target correction parameter at the current moment according to the first correction parameter and the second correction parameter.

[0106] In one embodiment of the present invention, after obtaining the target correction parameter at the current moment, the control end can convert the pixel coordinates into the geodetic coordinates of the actual flight of the UAV when the target correction parameter is expressed in the pixel coordinates in the image coordinate system, and then input the target correction parameter after the coordinate conversion into the Proportional Integration Differentiation (PID) controller, and obtain the flight control instructions of the UAV based on the PID control algorithm. The PID control algorithm is a traditional control algorithm that adjusts the output of the controller by comparing the error between the target value and the actual value.

[0107] Among them, the formula of PID control algorithm is as follows:

[0108]

[0109] Among them, u(t) is the final flight control instruction, K p is the proportional gain, K i is the integral gain, K d is the differential gain, e(t) is the current error, that is, the target correction parameter at the current moment. According to the proportional value (ProportionalTerm, P) of the current error K p e(t), the cumulative value of the current error (Integral Term, I) K i ∫e(t)dt and the rate of change of the current error (Derivative Term, D) Calculate the final flight control instruction u(t), where t represents the current moment.

[0110] In another embodiment of the present invention, after obtaining the target correction parameter at the current moment, the control end can input the target correction parameter into the adaptive control model, and obtain the flight control instruction of the UAV based on the model predictive control algorithm. The model predictive control algorithm is a control algorithm based on a dynamic model, which calculates the controller output by predicting the future state and control amount.

[0111] After generating the flight control instructions, the controller sends the flight control instructions to the drone, thereby controlling the drone to fly according to the established route.

[0112] In the technical solution provided by the embodiment of the present invention, during the flight of the drone, the first correction parameter at the current moment and the correct correction parameter at the previous moment are calculated to obtain the target correction parameter at the current moment. The target correction parameter at the current moment is regulated by the correct correction parameter at the previous moment. When the drone flies into the wrong position by mistake, the target correction parameter at the current moment is effectively regulated by the accurate correction parameter at the previous moment to avoid excessive correction amplitude caused by instantaneous noise of the drone, thereby accurately controlling the drone to fly according to the established route.

[0113] based on Figure 2 The embodiment shown in the figure, the embodiment of the present invention also provides a method for controlling the flight of an unmanned aerial vehicle, see Figure 3 , which is a flow chart of a second UAV flight control method provided in an embodiment of the present invention, the method includes steps S301-S305, wherein S305 is the same as the aforementioned S202, and S301-S304 are detailed steps of the aforementioned S201.

[0114] S301, obtaining an image to be detected.

[0115] The above-mentioned images to be detected are images collected by the drone in real time during flight, and the images to be detected may be road images, etc. The drone may be provided with a shooting device, and after the drone takes off, the drone may turn on the shooting device to collect images in real time, or after the drone receives an image collection instruction sent by the control end, the drone may turn on the shooting device to collect images in real time.

[0116] S302: Determine a target object from the image to be detected.

[0117] The above-mentioned target object can be understood as an object that often appears in the flight route of the drone and can indicate the trajectory of the drone's flight route. For example, when the detection image is a road image, the target object can be the center line of the highway in the middle of the road, so that the drone can fly along the trajectory of the center line of the highway in the middle of the road. When the detection object is a city image, the target object can be a street lamp, so that the drone can fly along the trajectory of the street lamp.

[0118] In one embodiment of the present invention, the embedded computing device may use an existing image recognition algorithm to determine a target object from an image to be detected.

[0119] In another embodiment of the present invention, a feature extraction model may be deployed on an embedded computing device, the image to be detected may be input into the feature extraction model, and the output of the feature extraction model may be determined as the target object.

[0120] In the embodiment of the present invention, there is no specific limitation on the method of determining the target object from the image to be detected.

[0121] S303: Determine the target position of the drone based on the position of the target object in the image to be detected.

[0122] The target position of the above-mentioned drone is the position where the drone should be located at the next moment. After the embedded computing device determines the target object from the image to be detected, it can also determine the position coordinates of the target object in the image coordinate system accordingly. Since the target object can indicate the trajectory of the drone's flight route, and the target object is determined from the image to be detected collected by the drone in real time, the embedded device can use the position coordinates of the target object in the image coordinate system as the target position of the drone.

[0123] S304, calculating a first correction parameter at the current moment according to the actual position of the UAV and the target position of the UAV.

[0124] In the embodiment of the present invention, the embedded computing device can determine the current position coordinates of the drone in the image coordinate system and use the position coordinates as the actual position of the drone.

[0125] In one embodiment of the present invention, when the flight route of the UAV is a straight path, the UAV may deviate from the target object during the flight of the straight path. At this time, the first correction parameter at the current moment can be specifically an offset correction parameter, that is, the offset between the horizontal coordinate of the actual position of the UAV and the horizontal coordinate of the target position. The flight path is corrected by calculating the first correction parameter to keep the UAV at the center of the target object.

[0126] See also Figure 4 , which is a schematic diagram of calculating the first correction parameter under a straight path provided by an embodiment of the present invention. After the embedded computing device detects and tracks the position of the target object (the center line of the highway) in real time and determines the target position of the drone and the current actual position of the drone, it obtains the horizontal coordinate x of the actual position of the drone at the current moment. t , and the target position of the drone at the next moment, that is, the horizontal coordinate x of the position where the drone should be located after correction at the next moment t+1 , the first correction parameter is calculated based on the horizontal coordinates of these two positions, and its calculation formula is as follows:

[0127] L= |x t+1 -x t | (3)

[0128] Among them, x t is the horizontal coordinate of the current position of the drone, x t+1 is the horizontal coordinate of the target position of the UAV at the next moment, and L represents the lateral offset distance that the UAV needs to correct, that is, the first correction parameter.

[0129] In another embodiment of the present invention, when the flight route of the UAV is a curved path, the UAV may deviate from the target object during the flight of the curved path. At this time, the first correction parameter at the current moment can be specifically a yaw correction parameter, that is, the offset angle between the actual position of the UAV and the target position. The flight path is corrected by calculating the first correction parameter so that the UAV remains at the center of the target object.

[0130] See also Figure 5 , which is a schematic diagram of calculating the first correction parameter under a curved path provided by an embodiment of the present invention. After the embedded device detects and tracks the position of the target object (highway centerline) in real time and determines the target position of the drone and the current actual position of the drone, it obtains the coordinates of the actual position of the drone at the current moment (x t ,y t ), and the target position of the drone at the next moment, that is, the coordinates (x t+1 ,y t+1 ), the first correction parameter is calculated based on the horizontal coordinates of these two positions, and its calculation formula is as follows:

[0131]

[0132] Among them, x t+1 -x t is the difference between the target position of the drone and the current actual position, y t+1 -y t is the difference between the target position of the UAV and the current actual position ordinate, and θ is the yaw correction angle between the current moment and the next moment of the UAV position, that is, the first correction parameter.

[0133] In one embodiment of the present invention, the flight correction parameters calculated by the embedded computing device may be expressed in the form of pixel coordinates in the image coordinate system. However, the pixel coordinates cannot be directly mapped to the position and motion trajectory of the drone in the actual geographic space. Therefore, the embedded computing device can convert the pixel coordinates into the geodetic coordinates of the actual flight of the drone. When the first correction parameter is the offset correction parameter L, the conversion formula is as shown in equations (5) and (6):

[0134]

[0135] L g = L × GSD level (6)

[0136] Where R is the horizontal size of the complementary metal oxide semiconductor (CMOS) of the image acquisition device, that is, the horizontal pixel size, H is the altitude of the drone above the ground, f is the focal length of the image acquisition device, n is the number of pixels in the horizontal direction of the image transmitted in real time over the network, L is the offset correction parameter (the first correction parameter), GSD level (Ground Sample Distance) is the ground sample distance in the horizontal direction, indicating the actual size of each pixel on the ground, which is determined by the physical characteristics of the image acquisition device (CMOS size and focal length) and the flight altitude of the drone, L g It is the offset correction parameter of the UAV in the geodetic coordinate system after coordinate transformation.

[0137] When the first correction parameter is the yaw correction parameter θ, since θ directly specifies the yaw angle of the drone, indicating the yaw direction that the drone currently needs to adjust, regardless of the current position or flight altitude of the drone, the angle is always a relative quantity without a unit, so no further conversion or complex processing is required. The conversion formula is as follows:

[0138] θ g =θ (7)

[0139] Among them, θ is the yaw correction parameter (the first correction parameter), θ g It is the UAV yaw correction parameter in the geodetic coordinate system after coordinate transformation.

[0140] As can be seen from the above embodiments, in the embodiments of the present invention, the target position of the drone is determined through the image collected in real time by the drone, and then the first correction parameter at the current moment is calculated according to the actual position of the drone and the target position of the drone. In this way, the embedded computing device can calculate the first correction parameter at the current moment more quickly, further improving the accuracy of the drone flight control.

[0141] based on Figure 3 The embodiment shown in the figure, the embodiment of the present invention also provides a method for controlling the flight of an unmanned aerial vehicle, see Figure 6 , which is a flow chart of a third UAV flight control method provided in an embodiment of the present invention, the method comprises steps S601-S605, wherein S601 is the same as the aforementioned step S301, S603-S605 are the same as the aforementioned S303-S305, and S602 is a detailed step of the aforementioned S302.

[0142] S602: extracting the target object in the image to be detected according to a preset feature extraction model.

[0143] Among them, the above-mentioned preset feature extraction model includes a backbone network, a neck network and an output network connected in sequence, the backbone network includes multiple CBL units, the CBL unit is composed of a convolutional layer, a batch normalization layer and an activation function, a CBL unit in the backbone network is directly connected to the output network, and the CBL unit is added with a residual connection, and other CBL units in the backbone network except the CBL unit directly connected to the output network are connected to the neck network, and the neck network includes a spatial pyramid pooling SPP unit and a cross-region segmented CSP unit.

[0144] In an embodiment of the present invention, a preset feature extraction model may be deployed on an embedded computing device, and a backbone network, a neck network and an output network of the preset feature extraction model are connected in sequence.

[0145] The backbone network is responsible for the preliminary extraction of basic features in the image to be detected. It is composed of CBL (i.e., convolutional layer Convolutional + batch normalization layer Batch Normalization + activation function Leaky ReLU) units composed of multiple convolutional layers, batch normalization layers and activation functions, to achieve multi-level step-by-step extraction of feature information of different scales. Multiple CBL units in the backbone network can be connected in series, and the embodiment of the present invention does not specifically limit the connection relationship between multiple CBL units. In the backbone network part, the traditional maximum pooling layer of the first preset number of layers is replaced with a CBL unit, and the CBL unit includes a convolutional layer with a preset step size. For example, the traditional maximum pooling layer of the first three layers of the backbone network is replaced with a convolutional layer with a step size of 2. The core purpose of this change is to reduce the loss of feature information in the pooling process and improve the granularity and accuracy of feature extraction. Although the pooling layer can reduce the amount of calculation, it will also lose some image details, while the convolutional layer can more effectively retain these details, which is helpful for the accuracy and robustness of target detection.

[0146] Too few pooling layers will make the model too large, and the number of parameters and the amount of calculation will increase dramatically, resulting in the inability to run efficiently on embedded airborne devices with low computing power. In order to further improve the detection accuracy and control the computing cost, in an embodiment of the present invention, an SPPCSP structure is introduced in the neck network of the preset feature extraction model. The structure can pool the feature map at multiple scales by introducing maximum pooling layers of different sizes, such as 5×5, 9×9, and 13×13 maximum pooling layers. Specifically, the SPPCSP structure can aggregate and fuse input features from different scales, effectively retaining important features of each scale. This multi-scale pooling strategy not only improves the richness and diversity of feature representation, but also reduces the computational complexity, thereby reducing the overall computational amount while ensuring detection accuracy. Multi-scale feature fusion enables the model to detect and classify more accurately when facing complex scenes and changing targets. In addition, the SPPCSP structure optimizes the feature transfer path through the CSP connection method, alleviates the gradient disappearance problem, and further improves the learning efficiency and feature extraction ability of the network. It can be seen that the embodiment of the present invention balances the efficiency and accuracy of feature extraction by improving the preset feature extraction model structure, ensuring that the image feature extraction model has both high real-time performance and can run efficiently on low-computing-power embedded airborne devices, thereby realizing a detection model that is both real-time and high-precision on embedded airborne devices, thereby meeting the actual needs of UAV flight missions.

[0147] On the basis of optimizing the structure of the image feature extraction model, in order to avoid the influence of excessive complexity on the training effect and operation efficiency of the model, and to ensure that the important basic features are transmitted to the output network without distortion, in the embodiment of the present invention, a special processing design is performed on the backbone network. Specifically, a convolutional layer (CBL unit) in the backbone network is directly connected to the output network. Since the basic features extracted by the backbone network are particularly important in the entire detection process, in addition to the complex feature transmission and fusion processing of the neck network, it is also necessary to ensure that a part of the key features on the backbone network can be directly and losslessly transmitted to the output network. This direct path ensures the integrity and timeliness of feature transmission, helps to improve the detection effect, and optimizes the computational complexity of the model. At the same time, residual connections are introduced on the units directly connected to the output network. Residual connections can effectively retain global features, enhance information flow, prevent gradient disappearance, and thus improve the training performance and convergence speed of the model. Not only does it solve the degradation problem in deep networks, but it also enhances the learning ability and expression ability of the network.

[0148] In the output network part, the information from the backbone network and the neck network is combined to achieve high-precision target detection and classification. In addition, since drones often encounter complex environments during flight, such as background interference such as roadbeds and flower beds, these interferences will affect the image feature extraction model's extraction and attention to the target object. Therefore, in an embodiment of the present invention, a spatial attention mechanism (Spatial Attention Mechanism, SAM) can also be introduced in the output network part. By introducing the SAM mechanism, these interferences can be reduced and the attention to the target object in the image to be detected can be enhanced.

[0149] It can be seen from the above embodiments that in the embodiments of the present invention, the structure of the image feature extraction model is optimized to balance the efficiency and accuracy of feature extraction, ensuring that the image feature extraction model has both high real-time performance and can run efficiently on low-computing-power embedded airborne devices, thereby realizing the deployment of real-time and high-precision image feature extraction models on embedded airborne devices, thereby meeting the actual needs of UAV flight missions and further improving the accuracy of UAV flight control.

[0150] based on Figure 3 The embodiment shown in the figure, the embodiment of the present invention also provides a method for controlling the flight of an unmanned aerial vehicle, see Figure 7 , is a flow chart of a fourth UAV flight control method provided by an embodiment of the present invention, the method comprising steps S701-S7010, wherein S701-S703 are the same as the above steps S301-S303, S709-S7010 are the same as the above steps S304-S305, and Figure 3 Compared with the illustrated embodiment, the method further includes steps S704-S708 after step S703.

[0151] S704: Obtain position information of a first detection frame surrounding the target object.

[0152] The first detection frame is a detection frame that surrounds the target object in the image to be detected. When the embedded computing device determines the target object from the image to be detected, it can use the detection frame to surround the target object. The detection frame is the first detection frame that surrounds the target object. The position information of the first detection frame can be represented in the form of coordinates.

[0153] In one embodiment of the present invention, the embedded computing device may obtain the vertex coordinates of the first detection frame as the position information of the first detection frame. In another embodiment of the present invention, the embedded computing device may obtain the center point coordinates of the first detection frame as the position information of the first detection frame.

[0154] When the embedded computing device determines the target object from the image to be detected, it may determine multiple target objects, that is, multiple first detection frames will appear. Therefore, in one embodiment of the present invention, the embedded computing device can filter out multiple first detection frames. Specifically, the embedded computing device can use the non-maximum suppression (NMS) algorithm to filter out redundant detection frames. In addition, since during the flight of the drone, the target objects usually appear in sequence relative to the direction of movement of the drone, for example, the target object is the center line of the highway in the middle of the road, and these highway center lines appear in sequence relative to the direction of movement of the drone. For example, the target object is the street lamp on the city road, and these street lamps appear in sequence relative to the direction of movement of the drone. Therefore, in an embodiment of the present invention, the embedded computing device determines the position of the first detection frame of the target object determined from the continuous images to be detected, by comparing the vertical coordinate y of the center point of the detection frame obtained at the current moment. t And the ordinate y of the center point of the new detection frame obtained at the next moment t+1 , if y t+1 >y t , a new ID is assigned to the target object surrounded by the new detection frame, that is, the logic of assigning the ID number of the target object is to assign it in the order from the near drone end to the far drone end, and the transmission of the correction parameters depends on the order of the target object ID number, which can ensure that the drone inspects in the order of the target objects.

[0155] S705: Estimate position information of a second detection frame in a next frame of the image to be detected.

[0156] In addition to determining the target object from the image to be detected, the determined target object can also be tracked, thus saving the process of determining the target object from each frame of the image to be detected, thereby saving computing resources. Based on this, in one embodiment of the present invention, the embedded computing device can estimate the position information of the second detection frame in the next frame of the image to be detected. Specifically, the embedded computing device can use Kalman filtering to estimate the position information of the second detection frame in the next frame of the image to be detected.

[0157] S706, matching the position information of the first detection frame with the position information of the second detection frame;

[0158] In one embodiment of the present invention, the embedded computing device may specifically use the Hungarian algorithm to match the position information of the current frame with the position information of the second detection frame, that is, to match the target object surrounded by the first detection frame with the object surrounded by the second detection frame. Since the number of initial second detection frames may be multiple, the embedded computing device may select the second detection frames with the top preset number of confidence rankings as candidate second detection frames according to the confidence of the matching results of the first detection frame and the multiple second detection frames, and then use the Hungarian algorithm to calculate the matching distance between the first detection frame and the multiple candidate second detection frames, and determine the second detection frame that matches the first detection frame from the multiple candidate second detection frames according to the matching distance. The specific matching method may use methods such as Mahalanobis distance and cosine distance.

[0159] S707: If the match is successful, determine whether the IOU of the first detection frame and the second detection frame exceeds a preset threshold.

[0160] Because in complex dynamic scenes, after the above-mentioned preliminary matching, some erroneous detection frames may not be effectively removed, thereby affecting the accuracy of target matching and the overall stability of the flight control system. In order to prevent the drone from tracking the wrong target, an Intersection Over Union (IOU) judgment step is once again introduced in an embodiment of the present invention. When the preliminary matching is successful, that is, the embedded computing device determines the second detection frame that matches the first detection frame from multiple candidate second detection frames, the IOU value between the first detection frame and the second detection frame is calculated. If the IOU value does not exceed the preset threshold, the second detection frame is considered to be unreliable and the second detection frame is removed. The calculation method of IOU is already relatively well known and will not be repeated here.

[0161] S708: If the value exceeds the preset threshold, the object surrounded by the second detection frame is determined to be the updated target object, and the target position of the drone is updated according to the position information of the second detection frame.

[0162] In an embodiment of the present invention, when the IOU value between the first detection frame and the second detection frame exceeds a preset threshold, the embedded computing device can determine that the object enclosed by the second detection frame is an updated target object, update the target position of the drone according to the position information of the second detection frame, and then calculate a new first correction parameter based on the updated target position of the drone.

[0163] It can be seen from the above embodiments that in the embodiments of the present invention, when tracking the target object, after undergoing a matching process, an intersection-over-union judgment step is introduced to further improve the accuracy of the matching result, thereby improving the accuracy of target tracking and ensuring the accuracy and continuity of the UAV flight control process.

[0164] In order to achieve safety and continuity during the flight of the drone, in one embodiment of the present invention, a dynamic power monitoring mechanism is designed on the embedded computing device to control the flight status of the drone according to the different battery power states. During the flight of the drone, the embedded computing device obtains the battery status information of the drone and detects whether the battery status information is abnormal. For example, the embedded computing device can detect whether the battery status of the drone is in a low power state. When it is in a low power state, it can be determined that the battery status information of the drone is abnormal. The embedded computing device can also detect whether the battery status of the drone is consumed at a rate greater than a preset threshold. When the consumption rate is greater than the preset threshold, it can be determined that the battery status information of the drone is abnormal.

[0165] When the embedded computing device determines that the battery status information of the drone is normal, it instructs the control end to control the drone to continue flying; when the embedded computing device determines that the battery status information of the drone is abnormal, it calculates whether the battery power status of the drone can return to the take-off point. If the battery power status of the drone can return to the take-off point, the embedded computing device instructs the control end to control the drone to return from the current position to the take-off point. When the battery power status of the drone cannot return to the take-off point, the control end is instructed to control the drone to perform a landing operation. At this time, the embedded computing device or the control end can also record the GPS coordinate information of the current position. In this way, not only the safety of the drone during the inspection process is ensured, but also reliable location information is provided for the next flight mission, ensuring the seamless connection of the two flight missions.

[0166] It can be seen from the above embodiments that in the embodiments of the present invention, by real-time monitoring of the power status of the drone, when the battery power is lower than a preset threshold, the drone is promptly controlled to perform landing or return operations to ensure the safety and continuity of the drone's flight mission.

[0167] See also Figure 8 , is a flow chart of a fifth UAV flight control method provided in an embodiment of the present invention. The method is applied to the control end in the above-mentioned UAV flight control system. The above-mentioned UAV flight control system also includes a UAV. The above-mentioned method includes steps S801-S802.

[0168] S801, receiving a first correction parameter at the current moment sent by an embedded computing device on the drone;

[0169] S802, determining the target correction parameter at the current moment according to the first correction parameter and the second correction parameter, generating a flight control instruction according to the target correction parameter at the current moment, and sending the flight control instruction to the UAV to control the UAV to fly.

[0170] Among them, the second correction parameter is the target correction parameter of the previous moment; the first correction parameter and the target correction parameter are used to instruct the UAV to fly to the target position, and the target position is the position where the UAV should be located at the next moment.

[0171] For this embodiment, since it is basically similar to the above method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the above method embodiment.

[0172] In the technical solution provided by the embodiment of the present invention, during the flight of the drone, the first correction parameter at the current moment and the correction parameter at the previous moment are calculated to obtain the target correction parameter at the current moment. The target correction parameter at the current moment is not only affected by the first correction parameter, but also regulated by the correction parameter at the previous moment. When the drone is affected by instantaneous noise factors such as environmental noise and visual noise, resulting in a large error in the first correction parameter, the target correction parameter at the current moment is effectively regulated by the accurate correction parameter that may not have been affected at the previous moment, thereby avoiding excessive correction amplitude caused by instantaneous noise, thereby accurately controlling the drone to fly according to the established route.

[0173] Next, based on the specific application scenario of UAV road inspection, the UAV flight control method provided by the present invention is described. Fig. 9 , is a schematic flow chart of a sixth UAV flight control method provided in an embodiment of the present invention, the method comprising steps S901-S9012.

[0174] S901, the control end sends an image acquisition instruction to the drone.

[0175] S902, the UAV sends the road image collected in real time to the embedded computing device.

[0176] In one embodiment of the present invention, the drone can also store the road images collected in real time. After the flight mission is completed, the ground workstation can read the road images stored by the drone and perform road damage detection on the road images. Specifically, the road images can be input into the trained road defect detection model to perform road damage detection on the road images.

[0177] S903: The embedded computing device determines a target object from the road image and tracks the target object.

[0178] Specifically, the target object may be a highway centerline in a road image.

[0179] S904, the embedded computing device calculates the first correction parameter of the drone at the current moment.

[0180] S905, the embedded computing device sends the first correction parameter to the control end based on the UDP protocol.

[0181] S906, the control end performs EMA filtering based on the first correction parameter and the second correction parameter to obtain the target correction parameter at the current moment, and generates a flight control instruction based on the target correction parameter.

[0182] S907, the UAV starts highway inspection based on the flight control command.

[0183] S908, the embedded computing device determines whether the battery information is abnormal.

[0184] If abnormal, execute S909; if normal, execute S9012.

[0185] S909, the embedded computing device determines whether the drone has enough power to return to the take-off point.

[0186] If the drone's battery power is not enough to return to the take-off point, execute S9010. If the drone's battery power is enough to return to the take-off point, execute S9011.

[0187] S9010, the control end controls the drone to perform landing operations.

[0188] S9011, the control end controls the drone to perform the return operation and records the GPS information of the return point.

[0189] S9012, the control end determines whether the first correction parameter is received within a preset time period.

[0190] If yes, execute S906; if no, execute S909.

[0191] It should be noted that the embodiment of the present invention does not specifically limit the execution order of some of the above steps, and for this embodiment, since it is basically similar to the above method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the above method embodiment.

[0192] In the technical solution provided by the embodiment of the present invention, during the flight of the drone, the first correction parameter at the current moment and the correct correction parameter at the previous moment are calculated to obtain the target correction parameter at the current moment. The target correction parameter at the current moment is regulated by the correct correction parameter at the previous moment. When the drone flies into the wrong position by mistake, the target correction parameter at the current moment is effectively regulated by the accurate correction parameter at the previous moment to avoid excessive correction amplitude caused by instantaneous noise of the drone, thereby accurately controlling the drone to fly according to the established route.

[0193] See also Fig.10, is a schematic diagram of the structure of a first UAV flight control device provided by an embodiment of the present invention, which is an embedded computing device used in a UAV flight control system. The UAV flight control system also includes a control terminal. The device includes:

[0194] A first correction parameter acquisition module 1001 is used to acquire a first correction parameter at a current moment;

[0195] The first correction parameter sending module 1002 is used to send the first correction parameter to the control end, so that the control end determines the target correction parameter at the current moment according to the first correction parameter and the second correction parameter, and the second correction parameter is the target correction parameter at the previous moment; generate a flight control instruction according to the target correction parameter at the current moment, and send the flight control instruction to the UAV to control the UAV to fly, the first correction parameter and the target correction parameter are used to instruct the UAV to fly to the target position, and the target position is the position where the UAV should be located at the next moment.

[0196] In the technical solution provided by the embodiment of the present invention, during the flight of the drone, the first correction parameter at the current moment and the correction parameter at the previous moment are calculated to obtain the target correction parameter at the current moment. The target correction parameter at the current moment is not only affected by the first correction parameter, but also regulated by the correction parameter at the previous moment. When the drone is affected by instantaneous noise factors such as environmental noise and visual noise, resulting in a large error in the first correction parameter, the target correction parameter at the current moment is effectively regulated by the accurate correction parameter that may not have been affected at the previous moment, thereby avoiding excessive correction amplitude caused by instantaneous noise, thereby accurately controlling the drone to fly according to the established route.

[0197] In one embodiment of the present invention, the first correction parameter acquisition module 1001 includes:

[0198] The image acquisition submodule is used to acquire the image to be detected, wherein the image to be detected is an image collected in real time by the UAV during flight;

[0199] A target object submodule, used to determine the target object from the image to be detected;

[0200] A target position determination submodule, used to determine the target position of the drone based on the position of the target object in the image to be detected;

[0201] The first correction parameter calculation submodule is used to calculate the first correction parameter at the current moment according to the actual position of the drone and the target position of the drone.

[0202] As can be seen from the above embodiments, in the embodiments of the present invention, the target position of the drone is determined through the image collected in real time by the drone, and then the first correction parameter at the current moment is calculated according to the actual position of the drone and the target position of the drone. In this way, the embedded computing device can calculate the first correction parameter at the current moment more quickly, further improving the accuracy of the drone flight control.

[0203] In one embodiment of the present invention, the target object determination submodule is specifically used to:

[0204] According to a preset feature extraction model, the target object in the image to be detected is extracted; wherein the preset feature extraction model includes a backbone network, a neck network and an output network connected in sequence, the backbone network includes a plurality of standard convolution CBL units, the CBL unit is composed of a convolution layer, a batch normalization layer and an activation function, a CBL unit in the backbone network is directly connected to the output network, and the CBL unit is added with a residual connection, and other CBL units in the backbone network except the CBL unit directly connected to the output network are connected to the neck network, and the neck network includes a spatial pyramid pooling SPP unit and a cross-region segmentation CSP unit.

[0205] It can be seen from the above embodiments that in the embodiments of the present invention, the structure of the image feature extraction model is optimized to balance the efficiency and accuracy of feature extraction, ensuring that the image feature extraction model has both high real-time performance and can run efficiently on low-computing-power embedded airborne devices, thereby realizing the deployment of real-time and high-precision image feature extraction models on embedded airborne devices, thereby meeting the actual needs of UAV flight missions and further improving the accuracy of UAV flight control.

[0206] In one embodiment of the present invention, the first correction parameter acquisition module 1001 further includes:

[0207] A position information acquisition submodule, configured to acquire position information of a first detection frame surrounding the target object after determining the target position of the drone based on the position of the target object in the image to be detected;

[0208] A position information estimation submodule, used to estimate the position information of a second detection frame in the next frame image of the image to be detected;

[0209] A position information matching submodule, used to match the position information of the first detection frame with the position information of the second detection frame;

[0210] A preset threshold judgment submodule is used to judge whether the intersection over union (IOU) of the first detection frame and the second detection frame exceeds a preset threshold if the match is successful;

[0211] The target position updating submodule is used to determine that the object surrounded by the second detection frame is an updated target object if it exceeds a preset threshold, and update the target position of the drone according to the position information of the second detection frame.

[0212] It can be seen from the above embodiments that in the embodiments of the present invention, when tracking the target object, after undergoing a matching process, an intersection-over-union judgment step is introduced to further improve the accuracy of the matching result, thereby improving the accuracy of target tracking and ensuring the accuracy and continuity of the UAV flight control process.

[0213] In one embodiment of the present invention, the first correction parameter sending module 1002 is specifically used for:

[0214] sending the first correction parameter to the control end;

[0215] When no new first correction parameter is sent to the control end within a preset time period after the moment when the first correction parameter is sent, the control end is instructed to control the drone to perform a landing or return operation.

[0216] As can be seen from the above embodiments, the embodiments of the present invention avoid the problem of flight mission failure or loss of control of the drone due to signal interruption or data loss, and can ensure the stability and safety of the drone flight process.

[0217] In one embodiment of the present invention, the device further comprises:

[0218] A battery status information acquisition module, used to acquire the battery status information of the drone, wherein the battery status information includes the battery charge status;

[0219] A battery status information detection module, used to detect whether the battery status information is abnormal;

[0220] A flight control module, used for instructing the control terminal to control the UAV to continue flying when the battery status information is normal;

[0221] The battery power calculation module is used to calculate whether the battery power status of the drone is capable of returning to the take-off point when the battery status information is abnormal; when the battery power status of the drone is capable of returning to the take-off point, instruct the control end to control the drone to return from the current position to the take-off point; when the battery power status of the drone is not capable of returning to the take-off point, instruct the control end to control the drone to perform a landing operation.

[0222] It can be seen from the above embodiments that in the embodiments of the present invention, by real-time monitoring of the power status of the drone, when the battery power is lower than a preset threshold, the drone is promptly controlled to perform landing or return operations to ensure the safety and continuity of the drone's flight mission.

[0223] See also Fig.11 , is a structural schematic diagram of a second UAV flight control device provided by an embodiment of the present invention, which is applied to a control end in a UAV flight control system, wherein the UAV flight control system also includes a UAV, and the device includes:

[0224] The first correction parameter receiving module 1101 is used to receive the first correction parameter at the current moment sent by the embedded computing device on the drone;

[0225] The target correction parameter determination module 1102 is used to determine the target correction parameter at the current moment based on the first correction parameter and the second correction parameter, where the second correction parameter is the target correction parameter at the previous moment; generate a flight control instruction based on the target correction parameter at the current moment, and send the flight control instruction to the UAV to control the UAV to fly, where the first correction parameter and the target correction parameter are used to instruct the UAV to fly to a target position, where the target position is the position where the UAV should be at the next moment.

[0226] In the technical solution provided by the embodiment of the present invention, during the flight of the drone, the first correction parameter at the current moment and the correction parameter at the previous moment are calculated to obtain the target correction parameter at the current moment. The target correction parameter at the current moment is not only affected by the first correction parameter, but also regulated by the correction parameter at the previous moment. When the drone is affected by instantaneous noise factors such as environmental noise and visual noise, resulting in a large error in the first correction parameter, the target correction parameter at the current moment is effectively regulated by the accurate correction parameter that may not have been affected at the previous moment, thereby avoiding excessive correction amplitude caused by instantaneous noise, thereby accurately controlling the drone to fly according to the established route.

[0227] The embodiment of the present invention further provides an electronic device, such as Fig.12 As shown, it includes a processor 1201 , a communication interface 1202 , a memory 1203 and a communication bus 1204 , wherein the processor 1201 , the communication interface 1202 , and the memory 1203 communicate with each other via the communication bus 1204 .

[0228] Memory 1203, used for storing computer programs;

[0229] The processor 1201 is used to implement any of the above-mentioned UAV flight control methods when executing the program stored in the memory 1203.

[0230] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0231] The communication interface is used for communication between the above electronic device and other devices.

[0232] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0233] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0234] In another embodiment provided by the present invention, a computer-readable storage medium is also provided, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned UAV flight control methods are implemented.

[0235] In another embodiment provided by the present invention, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute any of the UAV flight control methods in the above embodiments.

[0236] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.

[0237] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0238] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for system embodiments, device embodiments, electronic device embodiments, and computer storage medium embodiments, since they are basically similar to method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0239] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A UAV flight control method, characterized in that: An embedded computing device applied to a drone in a drone flight control system, wherein the drone flight control system further includes a control terminal, and the method includes: Get the first correction parameter at the current moment; The first correction parameter is sent to the control end so that the control end determines the target correction parameter at the current moment according to the first correction parameter and the second correction parameter, where the second correction parameter is the target correction parameter at the previous moment; a flight control instruction is generated according to the target correction parameter at the current moment, and the flight control instruction is sent to the UAV to control the UAV to fly, where the first correction parameter and the target correction parameter are used to instruct the UAV to fly to a target position, where the target position is the position where the UAV should be located at the next moment.

2. The method according to claim 1, characterized in that The obtaining of the first correction parameter at the current moment comprises: Acquire an image to be detected, where the image to be detected is an image collected in real time by the drone during flight; Determine a target object from the image to be detected; Determining a target position of the drone based on a position of the target object in the image to be detected; A first correction parameter at a current moment is calculated according to the actual position of the drone and the target position of the drone.

3. The method according to claim 2, characterized in that The step of determining the target object from the image to be detected comprises: According to a preset feature extraction model, the target object in the image to be detected is extracted; wherein the preset feature extraction model includes a backbone network, a neck network and an output network connected in sequence, the backbone network includes a plurality of standard convolution CBL units, the CBL unit is composed of a convolution layer, a batch normalization layer and an activation function, a CBL unit in the backbone network is directly connected to the output network, and the CBL unit is added with a residual connection, and other CBL units in the backbone network except the CBL unit directly connected to the output network are connected to the neck network, and the neck network includes a spatial pyramid pooling SPP unit and a cross-region segmentation CSP unit.

4. The method according to claim 2, characterized in that: After determining the target position of the drone based on the position of the target object in the image to be detected, the method further includes: Acquire position information of a first detection frame surrounding the target object; estimating position information of a second detection frame in a next frame image of the image to be detected; Matching the position information of the first detection frame with the position information of the second detection frame; If the match is successful, determine whether the intersection over union (IOU) of the first detection frame and the second detection frame exceeds a preset threshold; If it exceeds the preset threshold, it is determined that the object surrounded by the second detection frame is the updated target object, and the target position of the drone is updated according to the position information of the second detection frame.

5. The method according to any one of claims 1 to 4, characterized in that The sending the first correction parameter to the control end includes: sending the first correction parameter to the control end; When no new first correction parameter is sent to the control end within a preset time period after the moment when the first correction parameter is sent, the control end is instructed to control the drone to perform a landing or return operation.

6. The method according to any one of claims 1 to 4, characterized in that The method further comprises: Acquire battery status information of the drone, wherein the battery status information includes battery charge status; Detecting whether the battery status information is abnormal; If not, instruct the control terminal to control the drone to continue flying; If so, calculating whether the battery power status of the drone is sufficient to return to the take-off point; When the battery power of the drone is sufficient to return to the take-off point, instruct the control terminal to control the drone to return from the current position to the take-off point; When the battery power status of the drone is not enough to return to the take-off point, the control end is instructed to control the drone to perform a landing operation.

7. A UAV flight control method, characterized in that: A control terminal applied to a UAV flight control system, wherein the UAV flight control system also includes a UAV, and the method includes: Receiving a first correction parameter at a current moment sent by an embedded computing device on the drone; According to the first correction parameter and the second correction parameter, the target correction parameter at the current moment is determined, and the second correction parameter is the target correction parameter at the previous moment; a flight control instruction is generated according to the target correction parameter at the current moment, and the flight control instruction is sent to the UAV to control the UAV to fly, and the first correction parameter and the target correction parameter are used to instruct the UAV to fly to a target position, and the target position is the position where the UAV should be located at the next moment.

8. A UAV flight control system, characterized in that: The UAV flight control system includes a control terminal and a UAV. The embedded computing device on the drone is used to obtain the first correction parameter at the current moment and send the first correction parameter to the control end; The control end is used to receive the first correction parameter, and determine the target correction parameter at the current moment according to the first correction parameter and the second correction parameter, wherein the second correction parameter is the target correction parameter at the previous moment; Generate a flight control instruction according to the target correction parameter at the current moment, and send the flight control instruction to the UAV to control the UAV to fly, wherein the first correction parameter and the target correction parameter are used to instruct the UAV to fly to a target position, and the target position is a position where the UAV should be located at the next moment; The UAV is used to receive the flight control instruction and fly based on the flight control instruction.

9. A UAV flight control device, characterized in that: An embedded computing device used in a UAV flight control system, wherein the UAV flight control system further comprises a UAV and a control terminal, and the device comprises: A first correction parameter acquisition module, used to acquire the first correction parameter at the current moment; The first correction parameter sending module is used to send the first correction parameter to the control end, so that the control end determines the target correction parameter at the current moment according to the first correction parameter and the second correction parameter, and the second correction parameter is the target correction parameter at the previous moment; a flight control instruction is generated according to the target correction parameter at the current moment, and the flight control instruction is sent to the UAV to control the UAV to fly, and the first correction parameter and the target correction parameter are used to instruct the UAV to fly to a target position, and the target position is the position where the UAV should be located at the next moment.

10. A UAV flight control device, characterized in that: A control terminal used in a UAV flight control system, wherein the UAV flight control system also includes a UAV, and the device includes: A first correction parameter receiving module, used for receiving a first correction parameter at a current moment sent by an embedded computing device on the drone; The target correction parameter determination module is used to determine the target correction parameter at the current moment according to the first correction parameter and the second correction parameter, wherein the second correction parameter is the target correction parameter at the previous moment; generate a flight control instruction according to the target correction parameter at the current moment, and send the flight control instruction to the UAV to control the UAV to fly, wherein the first correction parameter and the target correction parameter are used to instruct the UAV to fly to a target position, wherein the target position is the position where the UAV should be located at the next moment.

11. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, for implementing the method steps described in any one of claims 1-6 or 7 when executing a program stored in a memory.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any one of claims 1-6 or 7 are implemented.

Images