Yaw control method, device and equipment of unmanned aerial vehicle, storage medium and product

Abstract

This application discloses a yaw control method, apparatus, device, storage medium, and product for a drone, relating to the field of drone control technology. It acquires real-time views from a binocular camera and real-time inertial measurement data, calculates the drone's real-time pose data, constructs a real-time 3D occupancy grid map based on the binocular camera's real-time view, obtains a target point, adjusts the drone's yaw heading based on the target point and real-time pose data, and determines the drone's flight trajectory based on the target point and the 3D occupancy grid map. Specifically, in this embodiment, by decoupling attitude control and position control during the drone control process, the drone's current heading is adjusted first, and the yaw heading is directed towards the desired target point before path planning control is performed. This avoids drone attitude loss of control and collisions with obstacles due to failure to update the field of view in time, significantly improving the stability and reliability of the drone's autonomous flight.

Description

Technical Field

[0001] This application relates to the field of unmanned aerial vehicle (UAV) control technology, and in particular to a method, apparatus, device, storage medium, and product for UAV yaw control. Background Technology

[0002] With the development of technology, drones are being used in increasingly wider fields, and corresponding autonomous flight technology for drones has also flourished. Autonomous flight of drones typically employs multi-source sensor fusion algorithms to construct the drone's positioning information, such as visual-inertial odometry fusion algorithms. Then, depth information sensors are used to obtain the distribution of obstacles in the environment, enabling real-time dynamic obstacle avoidance autonomous flight planning without GPS navigation. The corresponding planning results are then converted into control commands and sent to the drone's control system, thus achieving autonomous flight.

[0003] However, in current systems that use visual-inertial odometry fusion algorithms to obtain UAV positioning information, UAVs are prone to losing control when facing large-angle yaw maneuvers during autonomous flight. For example, when a UAV turns 180° to fly backward, yaw control and position control are coupled, causing errors in UAV attitude calculation. This can lead to UAV attitude loss or the UAV's field of vision not being straightened, making it unable to update the distance relationship with obstacles behind it, resulting in a direct collision with the obstacles and a crash. This greatly reduces the stability and reliability of UAVs during autonomous flight.

[0004] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0005] The main purpose of this application is to provide a yaw control method, device, equipment, storage medium and product for unmanned aerial vehicles (UAVs), aiming to solve the technical problem that the coupling between attitude control and position control during UAV yaw operation leads to UAV loss of control.

[0006] To achieve the above objectives, this application proposes a yaw control method for an unmanned aerial vehicle (UAV), the yaw control method comprising:

[0007] Acquire real-time view of the UAV's binocular camera and real-time inertial measurement data, and calculate the UAV's real-time pose data based on the binocular camera view and the real-time inertial measurement data;

[0008] A real-time 3D occupancy grid map is constructed based on the real-time view of the binocular camera.

[0009] The target point of the UAV is obtained, and the yaw and heading of the UAV are adjusted according to the target point and the real-time pose data.

[0010] The flight trajectory of the UAV after yaw heading adjustment is determined based on the target point and the three-dimensional occupancy grid map.

[0011] Optionally, the step of calculating the real-time pose data of the UAV based on the real-time view of the binocular camera and the real-time inertial measurement data includes:

[0012] The inertial measurement real-time data and the binocular camera real-time view are time-synchronized to obtain inertial measurement aligned data that is aligned with the timestamp of each frame of the binocular camera real-time view.

[0013] Feature points are obtained by performing feature processing on the real-time view of the binocular camera, wherein the feature processing includes feature extraction and feature matching;

[0014] The initial coordinates of the feature points are calculated based on binocular parallax.

[0015] Based on feature tracking and the initial coordinates, feature point reprojection error is established in consecutive frames of the real-time view of the binocular camera;

[0016] Based on the inertial measurement alignment data and the feature point reprojection error, the real-time pose data of the UAV is determined.

[0017] Optionally, the step of determining the real-time pose data of the UAV based on the inertial measurement alignment data and the feature point reprojection error includes:

[0018] The inertial measurement alignment data is integrated to obtain the relative rotation, velocity increment, and position increment;

[0019] Determine the optimization objective function based on the system state vector;

[0020] The relative rotation, velocity increment, position increment, and feature point reprojection error are optimized according to the optimization objective function to obtain the real-time pose data of the UAV, which includes attitude, position, and real-time velocity.

[0021] Optionally, the step of obtaining the target point of the UAV and adjusting the yaw and heading of the UAV based on the target point and the real-time pose data includes:

[0022] The orientation relationship between the UAV and the target point is calculated based on the real-time pose data, and the desired azimuth angle is calculated based on the orientation relationship.

[0023] The yaw angle of the UAV is calculated based on the real-time pose data, and the yaw difference between the desired azimuth angle and the yaw angle is calculated.

[0024] Determine whether the yaw difference is less than or equal to a preset threshold;

[0025] If the yaw difference is greater than the preset threshold, then the yaw parameter of the UAV is set, and the attitude of the UAV is adjusted based on the yaw parameter to obtain new real-time pose data.

[0026] Based on the new real-time pose data, return to the step of calculating the yaw angle of the UAV based on the real-time pose data;

[0027] If the yaw difference is less than or equal to the preset threshold, the flight trajectory of the UAV after yaw heading adjustment is determined based on the target point and the three-dimensional occupied grid map.

[0028] Optionally, the step of setting the yaw parameters of the UAV and adjusting the attitude of the UAV based on the yaw parameters to obtain new real-time pose data includes:

[0029] The target point is set as the current position of the UAV, and the three-axis speed of the UAV is set as the yaw adjustment value;

[0030] The yaw rate direction of the UAV is determined based on the current position and the yaw adjustment value;

[0031] The attitude of the UAV is adjusted according to the yaw rate direction and the yaw angle.

[0032] Optionally, the step of determining the flight trajectory of the UAV after yaw heading adjustment based on the target point and the three-dimensional occupancy grid map includes:

[0033] The flight trajectory of the UAV is determined based on the target point;

[0034] The system detects whether there are overlapping obstacles on the flight path based on the three-dimensional occupancy grid map. If there are no overlapping obstacles, the system drives the UAV to fly autonomously along the flight path.

[0035] If there are overlapping obstacles, the flight path will be replanned.

[0036] Furthermore, to achieve the above objectives, this application also proposes a yaw control device for an unmanned aerial vehicle (UAV), the yaw control device comprising:

[0037] The acquisition module is used to acquire the real-time view of the binocular camera and the real-time inertial measurement data of the UAV, and to calculate the real-time pose data of the UAV based on the real-time view of the binocular camera and the real-time inertial measurement data.

[0038] The construction module is used to construct a real-time 3D occupancy grid map based on the real-time view of the binocular camera;

[0039] An adjustment module is used to acquire the target point of the UAV and adjust the yaw and heading of the UAV based on the target point and the real-time pose data.

[0040] The determination module is used to determine the flight trajectory of the UAV after yaw heading adjustment based on the target point and the three-dimensional occupied grid map.

[0041] In addition, to achieve the above objectives, this application also proposes an electronic device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the yaw control method for a UAV as described above.

[0042] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the yaw control method for the UAV as described above.

[0043] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the yaw control method for a UAV as described above.

[0044] One or more technical solutions proposed in this application have at least the following technical effects:

[0045] In this embodiment, the real-time view of the UAV's binocular camera and real-time inertial measurement data are acquired. The real-time pose data of the UAV is calculated based on the binocular camera view and the real-time inertial measurement data. A real-time 3D occupancy grid map is constructed based on the binocular camera view to obtain the UAV's target point. The UAV's yaw heading is adjusted based on the target point and the real-time pose data. The flight trajectory of the UAV after yaw heading adjustment is determined based on the target point and the 3D occupancy grid map. In other words, this embodiment decouples attitude control and position control during the UAV control process. Specifically, the current heading of the UAV is adjusted first, avoiding positioning divergence caused by rapid movement along the planned path during rapid rotation, which could lead to attitude loss. The yaw heading is then directed towards the desired target point before path planning control is performed. This avoids collisions with obstacles and crashes caused by the UAV's inability to update distance relationships with obstacles in time during yaw control due to an out-of-focus field of view, significantly improving the stability and reliability of the UAV's autonomous flight. Attached Figure Description

[0046] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0047] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is a flowchart illustrating the first embodiment of the yaw control method for the UAV in this application;

[0049] Figure 2 This is a system architecture flowchart of the yaw control method for the UAV in this application;

[0050] Figure 3 This is a flowchart of the UAV state transition in the UAV yaw control method of this application;

[0051] Figure 4 This is a flowchart illustrating the second embodiment of the yaw control method for the UAV of this application.

[0052] Figure 5 An autonomous flight trajectory diagram of an unmanned aerial vehicle (UAV) provided in an embodiment of this application;

[0053] Figure 6 This is a schematic diagram of the module structure of the yaw control device for an unmanned aerial vehicle (UAV) according to an embodiment of this application;

[0054] Figure 7 This is a schematic diagram of the device structure of the hardware operating environment involved in the yaw control method of the UAV in the embodiments of this application.

[0055] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0056] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0057] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0058] The main solution of this application embodiment is as follows: acquire the real-time view of the binocular camera and the real-time inertial measurement data of the UAV; calculate the real-time pose data of the UAV based on the real-time view of the binocular camera and the real-time inertial measurement data; construct a real-time three-dimensional occupancy grid map based on the real-time view of the binocular camera; acquire the target point of the UAV; adjust the yaw and heading of the UAV based on the target point and the real-time pose data; and determine the flight trajectory of the UAV after the yaw and heading adjustment based on the target point and the three-dimensional occupancy grid map.

[0059] With the development of technology, drones are being used in increasingly wider fields, and corresponding autonomous flight technology for drones has also flourished. Autonomous flight of drones typically employs multi-source sensor fusion algorithms to construct the drone's positioning information, such as visual-inertial odometry fusion algorithms. Then, depth information sensors are used to obtain the distribution of obstacles in the environment, enabling real-time dynamic obstacle avoidance autonomous flight planning without GPS navigation. The corresponding planning results are then converted into control commands and sent to the drone's control system, thus achieving autonomous flight.

[0060] However, in current systems that use visual-inertial odometry fusion algorithms to obtain UAV positioning information, UAVs are prone to losing control when facing large-angle yaw maneuvers during autonomous flight. For example, when a UAV turns 180° to fly backward, yaw control and position control are coupled, causing errors in UAV attitude calculation. This can lead to UAV attitude loss or the UAV's field of vision not being straightened, making it unable to update the distance relationship with obstacles behind it, resulting in a direct collision with the obstacles and a crash. This greatly reduces the stability and reliability of UAVs during autonomous flight.

[0061] This application provides a solution whereby, by acquiring real-time views from a binocular camera and real-time inertial measurement data of a UAV, the real-time pose data of the UAV can be calculated. A real-time 3D occupancy grid map of the UAV is then constructed using the binocular camera's real-time view. The UAV's heading is then skewed towards a desired target point using the real-time pose data. Finally, obstacle avoidance path planning is performed on the UAV's autonomous flight trajectory using the desired target point and the 3D occupancy grid map. Understandably, in this application embodiment, by decoupling the UAV's attitude control and position control, and setting a maximum yaw speed, the yaw speed during the yaw process is limited, preventing the positioning algorithm from diverging due to excessive speed. Furthermore, once the heading corresponds to the desired target point, obstacle information within the UAV's field of view is updated promptly, preventing collisions and crashes caused by simultaneous changes in pose with obstacles outside the field of view. This significantly improves the reliability and stability of the UAV during autonomous flight.

[0062] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, computer, mobile phone, etc., or an electronic device capable of realizing the above functions.

[0063] Based on this, embodiments of this application provide a yaw control method for an unmanned aerial vehicle (UAV), referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the yaw control method for the UAV of this application.

[0064] In this embodiment, the yaw control method for the UAV includes steps S10 to S40:

[0065] Step S10: Obtain the real-time view of the binocular camera and the real-time inertial measurement data of the UAV, and calculate the real-time pose data of the UAV based on the real-time view of the binocular camera and the real-time inertial measurement data.

[0066] It should be noted that a binocular camera is a device that uses bionic principles to mimic human binocular vision. The real-time view of a binocular camera is a view captured from different angles by the two cameras within the camera. The binocular camera can be an Intel RealSense depth camera. Real-time inertial measurement data refers to acceleration and angular velocity data collected in real time by the inertial measurement unit (IMU) onboard the drone. Real-time pose data of the drone refers to its real-time position and attitude data in three-dimensional space.

[0067] Step S20: Construct a real-time 3D occupancy grid map based on the real-time view of the binocular camera;

[0068] It should be noted that a real-time 3D occupancy raster map is a probabilistic map based on a 3D raster grid to represent the environment, describing the feasibility of space through occupied / idle / unknown states.

[0069] For example, a real-time view is acquired through a binocular camera. This view undergoes image preprocessing, and binocular camera distortion is corrected in real time based on calibration parameters. Grayscale enhancement and noise reduction are performed on the left and right views to improve matching accuracy. A fast matching algorithm is employed to calculate the disparity map frame by frame. Combined with camera intrinsic parameters such as the binocular camera baseline and focal length, the depth point cloud is projected onto the world coordinate system of the UAV through coordinate transformation. Isotope filtering is used for downsampling to reduce data volume and improve computational efficiency. In terms of spatial representation, a 3D occupancy grid map is used to record the distribution of environmental obstacles. The resulting real-time 3D occupancy grid map accurately depicts the geometry and distribution of obstacles in the space surrounding the UAV, providing precise environmental perception data for subsequent UAV trajectory planning and obstacle avoidance.

[0070] Step S30: Obtain the target point of the UAV, and adjust the yaw and heading of the UAV according to the target point and the real-time pose data;

[0071] It should be noted that the target point refers to the coordinates of the target location that the user expects the drone to reach. It can be understood that the yaw heading adjustment refers to the drone adjusting its heading to deviate from its existing flight path after receiving the target point input by the user in real time.

[0072] For example, by acquiring a target point, the desired azimuth angle of the UAV toward that target point is calculated, and the UAV's heading is adjusted based on the desired azimuth angle and real-time pose data. The process of adjusting the UAV's heading based on the desired azimuth angle can include small-angle adjustments, such as right-angle turns; and large-angle adjustments, such as 180° deflections.

[0073] Step S40: Determine the flight trajectory of the UAV after yaw heading adjustment based on the target point and the three-dimensional occupied grid map;

[0074] It should be noted that the flight trajectory refers to the flight path planned by the UAV after adjusting its heading based on the target point, using the target point and real-time pose data. Understandably, the presence of obstacles along this flight path can be determined using a 3D occupancy grid map, thus enabling obstacle avoidance control.

[0075] For example, see Figure 2 , Figure 2 This application provides a flowchart of a yaw control system architecture for an unmanned aerial vehicle (UAV). A binocular camera acquisition unit acquires the UAV's current real-time view to construct a 3D occupancy grid map. An inertial measurement unit reads the UAV's real-time pose data. Based on the real-time pose data, the 3D occupancy grid map, and the target point, the UAV's flight trajectory is planned. Once the flight trajectory is safe, reliable, and meets dynamic constraints, the trajectory points, combined with the UAV's current position, velocity, acceleration, and yaw rate, are converted into corresponding position, velocity, acceleration, and yaw rate commands, which are then sent to an external control command signal processing unit for further control operations.

[0076] In this embodiment, real-time views from the binocular camera and real-time inertial measurement data of the UAV are acquired. Real-time pose data of the UAV is calculated based on the binocular camera view and the real-time inertial measurement data. A real-time 3D occupancy grid map is constructed based on the binocular camera view to obtain the target point of the UAV. The UAV's yaw heading is adjusted based on the target point and the real-time pose data. The flight trajectory of the UAV after yaw heading adjustment is determined based on the target point and the 3D occupancy grid map. In other words, this embodiment decouples attitude control and position control during the UAV control process. Specifically, the current heading of the UAV is adjusted first, avoiding positioning divergence caused by rapid movement along the planned path during rapid rotation, which could lead to attitude loss. The yaw heading is then directed towards the desired target point before path planning control is performed. This avoids collisions with obstacles and crashes caused by the UAV's inability to update distance relationships with obstacles in time during yaw control due to an out-of-focus field of view, significantly improving the stability and reliability of the UAV's autonomous flight.

[0077] In one feasible implementation, the step of calculating the real-time pose data of the UAV based on the real-time view of the binocular camera and the real-time inertial measurement data includes steps S11 to S15:

[0078] Step S11: Perform time synchronization processing on the inertial measurement real-time data and the binocular camera real-time view to obtain inertial measurement aligned data that is aligned with the timestamp of each frame of the binocular camera real-time view.

[0079] Step S12: Perform feature processing on the real-time view of the binocular camera to obtain feature points, wherein the feature processing includes feature extraction and feature matching;

[0080] Step S13: Calculate the initial coordinates of the feature points based on binocular parallax;

[0081] Step S14: Based on feature tracking and the initial coordinates, establish feature point reprojection error in consecutive frames of the real-time view of the binocular camera;

[0082] Step S15: Based on the inertial measurement alignment data and the feature point reprojection error, determine the real-time pose data of the UAV.

[0083] For example, inertial measurement alignment data is read, wherein the inertial measurement alignment data includes triaxial acceleration. and triaxial angular velocity ,in, , ,in, Let x be the acceleration along the X-axis. The acceleration along the Y-axis, Let Z be the acceleration along the Z-axis. Let be the angular velocity along the X-axis. The angular velocity along the Y-axis. The angular velocity is denoted by Z. Feature processing is performed on the real-time view from the binocular camera to select high-quality matching feature points. Utilizing the principle of binocular parallax and combining intrinsic parameters such as the binocular camera baseline and focal length, the initial coordinates of the matching feature points are calculated. This forms the initial point cloud. Based on the current camera pose... and location The three-dimensional feature points are projected using the projection function π(·). Projecting onto the image plane yields the predicted pixel coordinates. Reprojection error. ,in, Let π(·) represent the pixel positions actually observed by the binocular camera, and let π(·) be the projection function. For camera pose, For camera position, The coordinates of the 3D feature points in the world coordinate system are given, and the error value directly reflects the accuracy of the pose estimation of the binocular camera.

[0084] In one feasible implementation, the step of determining the real-time pose data of the UAV based on the inertial measurement alignment data and the feature point reprojection error includes steps S151-S153:

[0085] Step S151: Integrate the inertial measurement alignment data to obtain relative rotation, velocity increment, and position increment;

[0086] Step S152: Determine the optimization objective function based on the system state vector;

[0087] Step S153: Optimize the relative rotation, velocity increment, position increment, and feature point reprojection error according to the optimization objective function to obtain the real-time pose data of the UAV. The real-time pose data includes attitude, position, and real-time velocity.

[0088] It should be noted that relative rotation is used to establish rotational constraints between keyframes, reflecting the motion trend measured by the inertial measurement unit. Velocity increment reflects the acceleration mapping from the UAV coordinate system to the world coordinate system. Position increment converts the high-frequency motion information of inertial measurement into position constraints between keyframes, making up for the time interval between visual frames.

[0089] For example, between two keyframes from a stereo camera, the inertial measurement alignment data is integrated, and the integration formula is as follows:

[0090]

[0091]

[0092]

[0093] Where i is the i-th camera keyframe, j is the j-th camera keyframe, and k is the k-th keyframe between the i-th and j-th camera keyframes. Let be the angular velocity of the k-th frame. For gyroscope bias For the acceleration of the k-th frame, For accelerometer bias, For time step.

[0094] The system state vector is represented as: ,in, Let i be the pose of the camera in the i-th frame. Let i be the position of the camera in the i-th frame. Let be the speed of the camera in the i-th frame. For biasing the inertial measurement gyroscope, For inertial measurement accelerometer bias, The coordinates of the map feature points are given. The objective function for optimization is then obtained:

[0095]

[0096] in, For the pre-integration error of inertial measurement, This is due to binocular reprojection error. These are the weight matrices for the pre-integration error and the binocular reprojection error, respectively.

[0097] By minimizing the weighted sum of the two types of residuals, complementary visual and inertial measurement information is achieved. After optimization, the camera's attitude in the world coordinate system at the current moment is obtained. ,Location And velocity, where attitude is composed of quaternions, denoted by q, such as The position is represented by p, such as .

[0098] In one feasible implementation, the step of acquiring the target point of the UAV and adjusting the yaw and heading of the UAV based on the target point and the real-time pose data includes steps S31 to S36:

[0099] Step S31: Calculate the orientation relationship between the UAV and the target point based on the real-time pose data, and calculate the desired azimuth angle based on the orientation relationship;

[0100] Step S32: Calculate the yaw angle of the UAV based on the real-time pose data, and calculate the yaw difference between the desired azimuth angle and the yaw angle.

[0101] Step S33: Determine whether the yaw difference is less than or equal to a preset threshold;

[0102] Step S34: If the yaw difference is greater than the preset threshold, then set the yaw parameters of the UAV, adjust the attitude of the UAV based on the yaw parameters, and obtain new real-time pose data.

[0103] Step S35: Based on the new real-time pose data, return to the step of calculating the yaw angle of the UAV based on the real-time pose data;

[0104] Step S36: If the yaw difference is less than or equal to the preset threshold, then the flight trajectory of the UAV after yaw heading adjustment is determined based on the target point and the three-dimensional occupied grid map.

[0105] In one feasible implementation, the step of setting the yaw parameters of the UAV and adjusting the yaw heading of the UAV according to the yaw parameters includes steps S351~S353:

[0106] Step S351: Set the target point as the current position of the UAV, and set the three-axis speed of the UAV as the yaw adjustment value;

[0107] Step S352: Determine the yaw rate direction of the UAV based on the current position and the yaw adjustment value;

[0108] Step S353: Adjust the attitude of the UAV according to the yaw rate direction and the yaw angle.

[0109] For further explanation, please refer to [link / reference]. Figure 3 , Figure 3The UAV state transition flowchart provided in this application embodiment, before the steps of calculating the azimuth relationship between the UAV and the target point based on the real-time pose data, and calculating the desired azimuth angle based on the azimuth relationship, need to determine the UAV's operating state. The types of operating states include waiting for a target state, planning a trajectory state, replanning a trajectory state, and tracking a trajectory state. In the waiting for a target state, the UAV hovers, waiting for the target point input. Once the target point is received, the UAV's heading is adjusted, and it is determined whether the UAV's heading is pointing towards the target point. If it is, it enters the trajectory planning state. In the trajectory planning state, after receiving the new target point instruction and completing the yaw heading adjustment, the UAV begins planning a trajectory from its current position to the target point and detects whether there are obstacles on the path. If there are obstacles, it enters the replanning trajectory state; if there are no obstacles, it enters the tracking trajectory state. In the replanning trajectory state, the UAV detects interference between the initially planned trajectory and environmental obstacles, and re-plans the trajectory until the trajectory meets the requirements. In the tracking trajectory state, the UAV flies autonomously according to the planned flight trajectory.

[0110] For example, the formula for calculating the desired azimuth angle is:

[0111]

[0112]

[0113]

[0114] in, This represents the distance difference between the target point and the drone along the X-axis. This represents the distance difference between the target point and the drone along the Y-axis.

[0115] The formula for calculating the yaw angle is:

[0116]

[0117] in, , , and Let be the attitude quaternion of the drone.

[0118] Normalize the yaw difference to the range [-π, π], such as:

[0119]

[0120]

[0121] in, For the yaw difference, when A value greater than 0 indicates that the drone's current heading is deviated to the right relative to the target direction. When the value is less than 0, it indicates that the drone's current heading is deviated to the left relative to the target direction. In this case, it is determined whether the yaw difference is less than or equal to a preset threshold. If the yaw difference is greater than the preset threshold, the yaw parameters of the drone are set, and the drone's yaw heading is adjusted according to the yaw parameters.

[0122] It should be noted that the yaw parameters include the target point coordinates and the three-axis speed of the UAV.

[0123] For example, the target point is set to the drone's current location. Set the drone's three-axis speed to 0.

[0124] Update the drone's current yaw angle:

[0125]

[0126] in, , , and This is the quaternion of the drone's current attitude. If the yaw difference is positive, the drone is controlled to yaw counterclockwise; otherwise, it is controlled to yaw clockwise. The desired yaw angle is obtained by adding the yaw rate direction to the yaw angle, calculated using the following formula:

[0127]

[0128] in, For the desired yaw angle, Yaw angle This is the yaw difference. The speed factor is used to control the speed of rotation. It is adjusted according to the divergence of the positioning algorithm. If the positioning robustness is good, the speed can be increased to speed up the adjustment; otherwise, it is decreased to ensure the stability of the positioning algorithm.

[0129] Understandably, when a drone needs to adjust its heading, it is first determined that the drone is in a target waiting state. The drone's current heading is adjusted first, and the yaw is directed towards the desired target point before trajectory planning is performed. At this time, the drone does not need to handle yaw control and position control at the same time. Decoupling the two improves flight stability. At the same time, a maximum rotation speed limit is set to limit the speed of the drone's yaw movement, so as to avoid the positioning algorithm diverging due to excessive rotation speed. Furthermore, obstacle information in the field of view can be updated in a timely manner after the heading is adjusted, so as to avoid collisions with obstacles outside the field of view caused by simultaneous changes in pose.

[0130] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description, and will not be repeated hereafter. On this basis, refer to Figure 4 , Figure 4 This is a flowchart illustrating the second embodiment of the yaw control method for a UAV in this application. The step of determining the flight trajectory of the UAV after yaw heading adjustment based on the target point and the three-dimensional occupancy grid map includes steps S41 to S43:

[0131] Step S41: Determine the flight trajectory of the UAV based on the target point;

[0132] Step S42: Detect whether there are overlapping obstacles on the flight trajectory based on the three-dimensional occupancy grid map. If there are no overlapping obstacles, drive the UAV to fly autonomously according to the flight trajectory.

[0133] Step S43: If there are overlapping obstacles, the flight trajectory is replanned.

[0134] For example, by selecting a third-order B-spline, the flight trajectory is ensured to be smooth and the second derivative is continuous, satisfying the dynamic constraints. The control point interval is set according to the flight speed, such as 0.5m for low-speed flight and 1m for high-speed flight. A grid sampling verification method is used, sampling trajectory points at 0.1m intervals along the B-spline trajectory, checking whether each sampling point falls within the three-dimensional occupied grid map. If the flight trajectory is detected to overlap with an obstacle, the flight trajectory is replanned to avoid the obstacle. If the flight trajectory is not detected to overlap with an obstacle, the UAV is driven to fly autonomously according to the flight trajectory.

[0135] Understandable, see Figure 5 This application provides an embodiment of an autonomous flight trajectory map for a drone, showing the drone turning around normally during its return journey. x, y, and z represent the drone's coordinates in a three-dimensional grid map. By decoupling attitude control and position control during the drone's control process, when the drone is in a target waiting state and receives a return command, it first adjusts its current heading. This avoids positioning divergence caused by the drone rapidly moving along the planned path during rapid rotation, which could lead to attitude instability. The yaw heading is then aligned with the desired target point before path planning control is performed. This prevents the drone from colliding directly with obstacles and crashing due to an out-of-focus view during yaw control, thus ensuring the drone completes its return journey.

[0136] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the yaw control method of the UAV in this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0137] This application also provides a yaw control device for an unmanned aerial vehicle (UAV), please refer to... Figure 6 The yaw control device of the UAV includes:

[0138] The acquisition module 10 is used to acquire the real-time view of the binocular camera and the real-time inertial measurement data of the UAV, and to calculate the real-time pose data of the UAV based on the real-time view of the binocular camera and the real-time inertial measurement data.

[0139] Module 20 is used to construct a real-time 3D occupancy grid map based on the real-time view of the binocular camera;

[0140] Adjustment module 30 is used to acquire the target point of the UAV and adjust the yaw and heading of the UAV according to the target point and the real-time pose data;

[0141] The determination module 40 is used to determine the flight trajectory of the UAV after yaw heading adjustment based on the target point and the three-dimensional occupied grid map.

[0142] Optionally, the acquisition module 10 is further configured to:

[0143] The inertial measurement real-time data and the binocular camera real-time view are time-synchronized to obtain inertial measurement aligned data that is aligned with the timestamp of each frame of the binocular camera real-time view.

[0144] Feature points are obtained by performing feature processing on the real-time view of the binocular camera, wherein the feature processing includes feature extraction and feature matching;

[0145] The initial coordinates of the feature points are calculated based on binocular parallax.

[0146] Based on feature tracking and the initial coordinates, feature point reprojection error is established in consecutive frames of the real-time view of the binocular camera;

[0147] Based on the inertial measurement alignment data and the feature point reprojection error, the real-time pose data of the UAV is determined.

[0148] Optionally, the acquisition module 10 is further configured to:

[0149] The inertial measurement alignment data is integrated to obtain the relative rotation, velocity increment, and position increment;

[0150] Determine the optimization objective function based on the system state vector;

[0151] The relative rotation, velocity increment, position increment, and feature point reprojection error are optimized according to the optimization objective function to obtain the real-time pose data of the UAV, which includes attitude, position, and real-time velocity.

[0152] Optionally, the adjustment module 30 is further configured to:

[0153] The orientation relationship between the UAV and the target point is calculated based on the real-time pose data, and the desired azimuth angle is calculated based on the orientation relationship.

[0154] The first yaw angle of the UAV is calculated based on the real-time pose data, and the first yaw difference between the desired azimuth angle and the first yaw angle is calculated.

[0155] Determine whether the first yaw difference is less than or equal to a preset threshold;

[0156] If the yaw difference is greater than the preset threshold, then the yaw parameter of the UAV is set, and the attitude of the UAV is adjusted based on the yaw parameter to obtain new real-time pose data.

[0157] Based on the new real-time pose data, return to the step of calculating the yaw angle of the UAV based on the real-time pose data;

[0158] If the yaw difference is less than or equal to the preset threshold, the flight trajectory of the UAV after yaw heading adjustment is determined based on the target point and the three-dimensional occupied grid map.

[0159] Optionally, the adjustment module 30 is further configured to:

[0160] The target point is set as the current position of the UAV, and the three-axis speed of the UAV is set as the yaw adjustment value;

[0161] The yaw rate direction of the UAV is determined based on the current position and the yaw adjustment value;

[0162] The attitude of the UAV is adjusted according to the yaw rate direction and the yaw angle.

[0163] Optionally, the determining module 40 is further configured to:

[0164] The flight trajectory of the UAV is determined based on the target point;

[0165] The system detects whether there are overlapping obstacles on the flight path based on the three-dimensional occupancy grid map. If there are no overlapping obstacles, the system drives the UAV to fly autonomously along the flight path.

[0166] If there are overlapping obstacles, the flight path will be replanned.

[0167] The yaw control device for unmanned aerial vehicles (UAVs) provided in this application adopts the yaw control method for UAVs in the above embodiments, aiming to solve the technical problem of UAV loss of control due to the coupling of attitude control and position control during yaw operation. Compared with the prior art, the beneficial effects of the yaw control device for UAVs provided in this application are the same as those of the yaw control method for UAVs provided in the above embodiments, and other technical features in the yaw control device for UAVs are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0168] This application provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the yaw control method for the UAV in Embodiment 1 described above.

[0169] The following is for reference. Figure 7 The diagram illustrates a structural schematic of an electronic device suitable for implementing embodiments of this application. The electronic devices in these embodiments may include, but are not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 7 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0170] like Figure 7As shown, the electronic device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the electronic device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication device 1009 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. While electronic devices with various systems are shown in the figures, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.

[0171] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0172] The electronic device provided in this application employs the yaw control method for UAVs described in the above embodiments, aiming to solve the technical problem of UAV loss of control due to the coupling of attitude control and position control during yaw operation. Compared with the prior art, the beneficial effects of the electronic device provided in this application are the same as those of the yaw control method for UAVs provided in the above embodiments, and other technical features of this electronic device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0173] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0174] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0175] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the yaw control method of the UAV in the above embodiments.

[0176] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0177] The aforementioned computer-readable storage medium may be included in an electronic device or may exist independently without being assembled into an electronic device.

[0178] The aforementioned computer-readable storage medium carries one or more programs that, when executed by an electronic device, cause the electronic device to: acquire a real-time view from a binocular camera and real-time inertial measurement data of the UAV; calculate the real-time pose data of the UAV based on the real-time view from the binocular camera and the real-time inertial measurement data; construct a real-time three-dimensional occupancy grid map based on the real-time view from the binocular camera; acquire a target point of the UAV; adjust the yaw and heading of the UAV based on the target point and the real-time pose data; and determine the flight trajectory of the UAV after yaw and heading adjustment based on the target point and the three-dimensional occupancy grid map.

[0179] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0180] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0181] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0182] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the yaw control method of the aforementioned UAV. This aims to solve the technical problem of UAV loss of control due to the coupling of attitude control and position control during yaw operation. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the yaw control method of the UAV provided in the above embodiments, and will not be repeated here.

[0183] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the yaw control method for a UAV as described above.

[0184] The computer program product provided in this application aims to solve the technical problem of drone loss of control due to the coupling of attitude control and position control during yaw operation. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the drone yaw control method provided in the above embodiments, and will not be repeated here.

[0185] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A yaw control method for an unmanned aerial vehicle (UAV), characterized in that, The method includes: Acquire real-time view of the UAV's binocular camera and real-time inertial measurement data, and calculate the UAV's real-time pose data based on the binocular camera view and the real-time inertial measurement data; A real-time 3D occupancy grid map is constructed based on the real-time view of the binocular camera. The system acquires the target point of the UAV and adjusts its yaw heading based on the target point and the real-time pose data. It calculates the azimuth relationship between the UAV and the target point based on the real-time pose data and calculates the desired azimuth angle based on this relationship. It then calculates the yaw angle of the UAV based on the real-time pose data and calculates the yaw difference between the desired azimuth angle and the yaw angle. The system determines whether the yaw difference is less than or equal to a preset threshold. If the yaw difference is greater than the preset threshold, it sets the yaw parameters of the UAV and adjusts the UAV's attitude based on these parameters to obtain a new real-time pose. Pose data; based on the new real-time pose data, return to execute the step of calculating the yaw angle of the UAV based on the real-time pose data; if the yaw difference is less than or equal to the preset threshold, then determine the flight trajectory of the UAV after yaw heading adjustment based on the target point and the three-dimensional occupied grid map; set the target point as the current position point of the UAV, and set the three-axis velocity of the UAV as the yaw adjustment value; determine the yaw angular velocity direction of the UAV based on the current position point and the yaw adjustment value; adjust the attitude of the UAV based on the yaw angular velocity direction and the yaw angle; The flight trajectory of the UAV after yaw heading adjustment is determined based on the target point and the three-dimensional occupancy grid map.

2. The yaw control method for a UAV as described in claim 1, characterized in that, The step of calculating the real-time pose data of the UAV based on the real-time view of the binocular camera and the real-time inertial measurement data includes: The inertial measurement real-time data and the binocular camera real-time view are time-synchronized to obtain inertial measurement aligned data that is aligned with the timestamp of each frame of the binocular camera real-time view. Feature points are obtained by performing feature processing on the real-time view of the binocular camera, wherein the feature processing includes feature extraction and feature matching; The initial coordinates of the feature points are calculated based on binocular parallax. Based on feature tracking and the initial coordinates, feature point reprojection error is established in consecutive frames of the real-time view of the binocular camera; Based on the inertial measurement alignment data and the feature point reprojection error, the real-time pose data of the UAV is determined.

3. The yaw control method for a UAV as described in claim 2, characterized in that, The step of determining the real-time pose data of the UAV based on the inertial measurement alignment data and the feature point reprojection error includes: The inertial measurement alignment data is integrated to obtain the relative rotation, velocity increment, and position increment; Determine the optimization objective function based on the system state vector; The relative rotation, velocity increment, position increment, and feature point reprojection error are optimized according to the optimization objective function to obtain the real-time pose data of the UAV, which includes attitude, position, and real-time velocity.

4. The yaw control method for a UAV as described in claim 1, characterized in that, The step of determining the flight trajectory of the UAV after yaw heading adjustment based on the target point and the three-dimensional occupied grid map includes: The flight trajectory of the UAV is determined based on the target point; The system detects whether there are overlapping obstacles on the flight path based on the three-dimensional occupancy grid map. If there are no overlapping obstacles, the system drives the UAV to fly autonomously along the flight path. If there are overlapping obstacles, the flight path will be replanned.

5. A yaw control device for an unmanned aerial vehicle (UAV), characterized in that, The device includes: The acquisition module is used to acquire the real-time view of the binocular camera and the real-time inertial measurement data of the UAV, and to calculate the real-time pose data of the UAV based on the real-time view of the binocular camera and the real-time inertial measurement data. The construction module is used to construct a real-time 3D occupancy grid map based on the real-time view of the binocular camera; An adjustment module is used to acquire the target point of the UAV, adjust the yaw heading of the UAV based on the target point and the real-time pose data; calculate the azimuth relationship between the UAV and the target point based on the real-time pose data, and calculate the desired azimuth angle based on the azimuth relationship; calculate the yaw angle of the UAV based on the real-time pose data, and calculate the yaw difference between the desired azimuth angle and the yaw angle; determine whether the yaw difference is less than or equal to a preset threshold; if the yaw difference is greater than the preset threshold, set the yaw parameters of the UAV, and adjust the attitude of the UAV based on the yaw parameters to obtain a new yaw heading. The system retrieves real-time pose data; based on the new real-time pose data, it returns and executes the step of calculating the yaw angle of the UAV based on the real-time pose data; if the yaw difference is less than or equal to the preset threshold, it determines the flight trajectory of the UAV after yaw heading adjustment based on the target point and the three-dimensional occupied grid map; it sets the target point as the current position point of the UAV and sets the three-axis velocity of the UAV as the yaw adjustment value; it determines the yaw angular velocity direction of the UAV based on the current position point and the yaw adjustment value; and it adjusts the attitude of the UAV based on the yaw angular velocity direction and the yaw angle. The determination module is used to determine the flight trajectory of the UAV after yaw heading adjustment based on the target point and the three-dimensional occupied grid map.

6. An electronic device, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the yaw control method for the UAV as described in any one of claims 1 to 4.

7. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the yaw control method for the UAV as described in any one of claims 1 to 4.

8. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the yaw control method for the UAV as described in any one of claims 1 to 4.

Images